diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-05-19 07:33:21 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-05-19 07:33:21 +0000 |
| commit | 36a59d088eca61b834191dacea009677a96c052f (patch) | |
| tree | e4f33972dab5d8ef79e3944a9f403035fceea43f /doc/user | |
| parent | a1761f15ec2cae7c7f7bbda39a75494add0dfd6f (diff) | |
| download | gitlab-ce-36a59d088eca61b834191dacea009677a96c052f.tar.gz | |
Add latest changes from gitlab-org/gitlab@15-0-stable-eev15.0.0-rc42
Diffstat (limited to 'doc/user')
235 files changed, 3411 insertions, 6651 deletions
diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index 69cb2f04c4d..9d6dcf30908 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -20,7 +20,7 @@ Broadcast messages can be managed using the [broadcast messages API](../../api/b Banners are shown on the top of a page and in Git remote responses. - + ```shell $ git push @@ -66,18 +66,13 @@ To add a broadcast message: - `padding` - `margin` - `text-decoration` -1. Select one of the suggested background colors, or add the hex code of a different color. The default color is orange. +1. Select a **Theme**. The default theme is `indigo`. 1. Select the **Dismissable** checkbox to enable users to dismiss the broadcast message. 1. Optional. Select **Target roles** to only show the broadcast message to users with the selected roles. The message displays on group, subgroup, and project pages, and does not display in Git remote responses. 1. If required, add a **Target Path** to only show the broadcast message on URLs matching that path. You can use the wildcard character `*` to match multiple URLs, for example `mygroup/myproject*`. 1. Select a date for the message to start and end. 1. Select **Add broadcast message**. -NOTE: -The **Background color** field expects the value to be a hexadecimal code because -the form uses the [color_field](https://api.rubyonrails.org/v6.0.3.4/classes/ActionView/Helpers/FormHelper.html#method-i-color_field) -helper method, which generates the proper HTML to render. - When a broadcast message expires, it no longer displays in the user interface but is still listed in the list of broadcast messages. diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 21ac0f720ec..4308b45df78 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -40,14 +40,11 @@ To access the Credentials inventory: If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: -| Token state | [Token expiration enforced?](settings/account_and_limit_settings.md#allow-expired-personal-access-tokens-to-be-used-deprecated) | Show Revoke button? | Comments | -|-------------|------------------------|--------------------|----------------------------------------------------------------------------| -| Active | Yes | Yes | Allows administrators to revoke the PAT, such as for a compromised account | -| Active | No | Yes | Allows administrators to revoke the PAT, such as for a compromised account | -| Expired | Yes | No | PAT expires automatically | -| Expired | No | Yes | The administrator may revoke the PAT to prevent indefinite use | -| Revoked | Yes | No | Not applicable; token is already revoked | -| Revoked | No | No | Not applicable; token is already revoked | +| Token state | Show Revoke button? | Comments | +|-------------|---------------------|----------------------------------------------------------------------------| +| Active | Yes | Allows administrators to revoke the PAT, such as for a compromised account | +| Expired | No | Not applicable; token is already expired | +| Revoked | No | Not applicable; token is already revoked | When a PAT is revoked from the credentials inventory, the instance notifies the user by email. diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md index b3b2c14adbd..43dce1921f4 100644 --- a/doc/user/admin_area/geo_nodes.md +++ b/doc/user/admin_area/geo_nodes.md @@ -12,7 +12,7 @@ You can configure various settings for GitLab Geo sites. For more information, s On either the primary or secondary site: 1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Geo > Nodes**. +1. On the left sidebar, select **Geo > Sites**. ## Common settings diff --git a/doc/user/admin_area/img/broadcast_messages_banner_v12_10.png b/doc/user/admin_area/img/broadcast_messages_banner_v12_10.png Binary files differdeleted file mode 100644 index 2e893476bc6..00000000000 --- a/doc/user/admin_area/img/broadcast_messages_banner_v12_10.png +++ /dev/null diff --git a/doc/user/admin_area/img/broadcast_messages_banner_v15_0.png b/doc/user/admin_area/img/broadcast_messages_banner_v15_0.png Binary files differnew file mode 100644 index 00000000000..e1b350142b3 --- /dev/null +++ b/doc/user/admin_area/img/broadcast_messages_banner_v15_0.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index f57672d3d36..262bb2cc931 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -25,7 +25,7 @@ The Admin Area is made up of the following sections: | Section | Description | |:-----------------------------------------------|:------------| | **{overview}** [Overview](#overview-section) | View your GitLab [Dashboard](#admin-area-dashboard), and administer [projects](#administering-projects), [users](#administering-users), [groups](#administering-groups), [topics](#administering-topics), [jobs](#administering-jobs), [runners](#administering-runners), and [Gitaly servers](#administering-gitaly-servers). | -| **{monitor}** Monitoring | View GitLab [system information](#system-information), and information on [background jobs](#background-jobs), [logs](#logs), [health checks](monitoring/health_check.md), [requests profiles](#requests-profiles), and [audit events](#audit-events). | +| **{monitor}** Monitoring | View GitLab [system information](#system-information), and information on [background jobs](#background-jobs), [logs](#logs), [health checks](monitoring/health_check.md), and [audit events](#audit-events). | | **{messages}** Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | **{hook}** System Hooks | Configure [system hooks](../../administration/system_hooks.md) for many events. | | **{applications}** Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | @@ -184,7 +184,7 @@ The following data is included in the export: - Type - Path - Access level ([Project](../permissions.md#project-members-permissions) and [Group](../permissions.md#group-members-permissions)) -- Date of last activity ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345388) in GitLab 14.6). For a list of activities that populate this column, see the [Users API documentation](../../api/users.md#get-user-activities-admin-only). +- Date of last activity ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345388) in GitLab 14.6). For a list of activities that populate this column, see the [Users API documentation](../../api/users.md#get-user-activities-administrator-only). Only the first 100,000 user accounts are exported. @@ -326,7 +326,7 @@ To search runners' descriptions: 1. In the **Search or filter results...** field, type the description of the runner you want to find. -1. Press Enter. +1. Press <kbd>Enter</kbd>. You can also filter runners by status, type, and tag. To filter: @@ -430,10 +430,6 @@ For details of these log files and their contents, see [Log system](../../admini The content of each log file is listed in chronological order. To minimize performance issues, a maximum 2000 lines of each log file are shown. -### Requests Profiles - -The **Requests Profiles** page contains the token required for profiling. For more details, see [Request Profiling](../../administration/monitoring/performance/request_profiling.md). - ### Audit Events **(PREMIUM SELF)** The **Audit Events** page lists changes made within the GitLab server. With this information you can control, analyze, and track every change. diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md index 5999e774d26..ff9e87680f9 100644 --- a/doc/user/admin_area/license_file.md +++ b/doc/user/admin_area/license_file.md @@ -18,8 +18,7 @@ Otherwise, to add your license: 1. Sign in to GitLab as an administrator. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > General**. -1. In the **License file** area, select **Add a license**. -1. Add a license by either uploading the file or pasting the key. +1. In the **Add License** area, add a license by either uploading the file or entering the key. 1. Select the **Terms of Service** checkbox. 1. Select **Add license**. diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index e8db319df77..53c08d8cbc1 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -15,7 +15,7 @@ users. A user in _pending approval_ state requires action by an administrator. A user sign up can be in a pending approval state because an administrator has enabled any of the following options: -- [Require admin approval for new sign-ups](settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) setting. +- [Require administrator approval for new sign-ups](settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) setting. - [User cap](settings/sign_up_restrictions.md#user-cap). - [Block auto-created users (OmniAuth)](../../integration/omniauth.md#configure-initial-settings) - [Block auto-created users (LDAP)](../../administration/auth/ldap/index.md#basic-configuration-settings) diff --git a/doc/user/admin_area/monitoring/background_migrations.md b/doc/user/admin_area/monitoring/background_migrations.md index 726827054da..b666c0c5ad2 100644 --- a/doc/user/admin_area/monitoring/background_migrations.md +++ b/doc/user/admin_area/monitoring/background_migrations.md @@ -190,7 +190,7 @@ sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackground In GitLab 14.8, the `BackfillNamespaceIdForNamespaceRoute` batched background migration job may fail to complete. When retried, a `500 Server Error` is returned. This issue was -[resolved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82387) in GitLab 14.9. +[resolved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82387) in GitLab 14.9. To resolve this issue, [upgrade GitLab](../../../update/index.md) from 14.8 to 14.9. You can ignore the failed batch migration until after you update to GitLab 14.9. diff --git a/doc/user/admin_area/reporting/spamcheck.md b/doc/user/admin_area/reporting/spamcheck.md index 559235fe322..b1ec203cffc 100644 --- a/doc/user/admin_area/reporting/spamcheck.md +++ b/doc/user/admin_area/reporting/spamcheck.md @@ -65,4 +65,4 @@ Spamcheck service on its own can not communicate directly over TLS with GitLab. However, Spamcheck can be deployed behind a reverse proxy which performs TLS termination. In such a scenario, GitLab can be made to communicate with Spamcheck over TLS by specifying `tls://` scheme for the external Spamcheck URL -instead of `grpc://` in the Admin settings. +instead of `grpc://` in the Admin Area settings. diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index e6d8107ed9b..9905298784a 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -51,12 +51,14 @@ For GitLab.com repository size limits, read [accounts and limit settings](../../ ## Max push size -You can change the maximum push size for your repository: +You can change the maximum push size for your instance: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. 1. Increase or decrease by changing the value in **Maximum push size (MB)**. +For GitLab.com application limits, read [GitLab application limits](../../../administration/instance_limits.md#max-push-size). + NOTE: When you [add files to a repository](../../project/repository/web_editor.md#create-a-file) through the web UI, the maximum **attachment** size is the limiting factor, @@ -64,9 +66,19 @@ because the [web server](../../../development/architecture.md#components) must receive the file before GitLab can generate the commit. Use [Git LFS](../../../topics/git/lfs/index.md) to add large files to a repository. +## Max export size + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86124) in GitLab 15.0. + +To modify the maximum file size for exports in GitLab: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > General**, then expand **Account and limit**. +1. Increase or decrease by changing the value in **Maximum export size (MB)**. + ## Max import size -> [Modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. To modify the maximum file size for imports in GitLab: @@ -158,22 +170,6 @@ wiki, packages, or snippets. The repository size limit applies to both private a For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). -## Troubleshooting - -### 413 Request Entity Too Large - -When attaching a file to a comment or reply in GitLab displays a `413 Request Entity Too Large` -error, the [max attachment size](#max-attachment-size) -is probably larger than the web server's allowed value. - -To increase the max attachment size to 200 MB in a -[Omnibus GitLab](https://docs.gitlab.com/omnibus/) install, you may need to -add the line below to `/etc/gitlab/gitlab.rb` before increasing the max attachment size: - -```ruby -nginx['client_max_body_size'] = "200m" -``` - ## Customize session duration for Git Operations when 2FA is enabled **(PREMIUM SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296669) in GitLab 13.9. @@ -229,35 +225,28 @@ Once a lifetime for SSH keys is set, GitLab: NOTE: When a user's SSH key becomes invalid they can delete and re-add the same key again. -## Allow expired SSH keys to be used (DEPRECATED) **(ULTIMATE SELF)** +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> +## Allow expired SSH keys to be used (removed) **(ULTIMATE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250480) in GitLab 13.9. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/320970) in GitLab 14.0. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 15.0. -WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8. +This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 15.0. +<!--- end_remove --> -By default, expired SSH keys **are not usable**. - -To allow the use of expired SSH keys: - -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand the **Account and limit** section. -1. Uncheck the **Enforce SSH key expiration** checkbox. - -Disabling SSH key expiration immediately enables all expired SSH keys. - -## Limit the lifetime of personal access tokens **(ULTIMATE SELF)** +## Limit the lifetime of access tokens **(ULTIMATE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) in GitLab 12.6. Users can optionally specify a lifetime for -[personal access tokens](../../profile/personal_access_tokens.md). +access tokens, this includes [personal](../../profile/personal_access_tokens.md), +[group](../../group/settings/group_access_tokens.md), and [project](../../project/settings/project_access_tokens.md) access tokens. This lifetime is not a requirement, and can be set to any arbitrary number of days. -Personal access tokens are the only tokens needed for programmatic access to GitLab. +Access tokens are the only tokens needed for programmatic access to GitLab. However, organizations with security requirements may want to enforce more protection by requiring the regular rotation of these tokens. @@ -266,15 +255,15 @@ requiring the regular rotation of these tokens. Only a GitLab administrator can set a lifetime. Leaving it empty means there are no restrictions. -To set a lifetime on how long personal access tokens are valid: +To set a lifetime on how long access tokens are valid: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand the **Account and limit** section. -1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. +1. Fill in the **Maximum allowable lifetime for access tokens (days)** field. 1. Click **Save changes**. -Once a lifetime for personal access tokens is set, GitLab: +Once a lifetime for access tokens is set, GitLab: - Applies the lifetime for new personal access tokens, and require users to set an expiration date and a date no later than the allowed lifetime. @@ -282,23 +271,17 @@ Once a lifetime for personal access tokens is set, GitLab: allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, or remove it, before revocation takes place. -## Allow expired Personal Access Tokens to be used (DEPRECATED) **(ULTIMATE SELF)** +<!-- start_remove The following content will be removed on remove_date: '2022-08-22' --> +## Allow expired access tokens to be used (removed) **(ULTIMATE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214723) in GitLab 13.1. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/296881) in GitLab 13.9. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 14.8. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 15.0. -WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 14.8. - -By default, expired personal access tokens (PATs) **are not usable**. - -To allow the use of expired PATs: - -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand the **Account and limit** section. -1. Uncheck the **Enforce personal access token expiration** checkbox. +This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351962) in GitLab 15.0. +<!-- end_remove --> ## Disable user profile name changes **(PREMIUM SELF)** @@ -314,5 +297,35 @@ To do this: NOTE: When this ability is disabled, GitLab administrators can still use the -[Admin UI](../index.md#administering-users) or the +[Admin Area](../index.md#administering-users) or the [API](../../../api/users.md#user-modification) to update usernames. + +## Troubleshooting + +### 413 Request Entity Too Large + +When attaching a file to a comment or reply in GitLab displays a `413 Request Entity Too Large` +error, the [max attachment size](#max-attachment-size) +is probably larger than the web server's allowed value. + +To increase the max attachment size to 200 MB in a +[Omnibus GitLab](https://docs.gitlab.com/omnibus/) install, you may need to +add the line below to `/etc/gitlab/gitlab.rb` before increasing the max attachment size: + +```ruby +nginx['client_max_body_size'] = "200m" +``` + +### This repository has exceeded its size limit + +If you receive intermittent push errors in your [Rails exceptions log](../../../administration/logs.md#exceptions_jsonlog), like this: + +```plaintext +Your push has been rejected, because this repository has exceeded its size limit. +``` + +[Housekeeping](../../../administration/housekeeping.md) tasks may be causing your repository size to grow. +To resolve this problem, either of these options helps in the short- to middle-term: + +- Increase the [repository size limit](#repository-size-limit). +- [Reduce the repo size](../../project/repository/reducing_the_repo_size_using_git.md). diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 683c07460ee..170d3cf4c90 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -149,7 +149,7 @@ As an administrator you can set either a global or namespace-specific limit on t ## Archive jobs -Archiving jobs is useful for reducing the CI/CD footprint on the system by removing some +Archiving jobs is useful for reducing the CI/CD footprint on the system by removing some of the capabilities of the jobs (metadata stored in the database needed to run the job), but persisting the traces and artifacts for auditing purposes. @@ -170,7 +170,7 @@ For the value set for GitLab.com, see [Scheduled job archiving](../../gitlab_com ## Protect CI/CD variables by default To set all new [CI/CD variables](../../../ci/variables/index.md) as -[protected](../../../ci/variables/index.md#protect-a-cicd-variable) by default: +[protected](../../../ci/variables/index.md#protected-cicd-variables) by default: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. @@ -224,13 +224,9 @@ To enable or disable the banner: 1. Select or clear the **Enable pipeline suggestion banner** checkbox. 1. Select **Save changes**. -## Required pipeline configuration **(PREMIUM SELF)** +## Required pipeline configuration **(ULTIMATE SELF)** -WARNING: -Required pipeline configurations is in its end-of-life process for Premium users. It's -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) in GitLab 14.8, -and planned to be unavailable for Premium users in GitLab 15.0. This feature is planned to continue -to be available for Ultimate users. Ultimate users are not impacted by this deprecation and removal. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0. NOTE: An alternative [compliance solution](../../project/settings/index.md#compliance-pipeline-configuration) diff --git a/doc/user/admin_area/settings/files_api_rate_limits.md b/doc/user/admin_area/settings/files_api_rate_limits.md index 7305e49b0d2..544c81e0583 100644 --- a/doc/user/admin_area/settings/files_api_rate_limits.md +++ b/doc/user/admin_area/settings/files_api_rate_limits.md @@ -8,7 +8,7 @@ type: reference # Rate limits on Repository files API **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68561) in GitLab 14.3. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75918) in GitLab 14.6. [Feature flag files_api_throttling](https://gitlab.com/gitlab-org/gitlab/-/issues/338903) removed. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75918) in GitLab 14.6. [Feature flag `files_api_throttling`](https://gitlab.com/gitlab-org/gitlab/-/issues/338903) removed. The [Repository files API](../../../api/repository_files.md) enables you to fetch, create, update, and delete files in your repository. To improve the security diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 052b6e26c07..970897fd8da 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -112,8 +112,6 @@ The **Metrics and profiling** settings contain: - [Self monitoring](../../../administration/monitoring/gitlab_self_monitoring_project/index.md#create-the-self-monitoring-project) - Enable or disable instance self monitoring. - [Usage statistics](usage_statistics.md) - Enable or disable version check and Service Ping. -- [Pseudonymizer data collection](../../../administration/pseudonymizer.md) - - Enable or disable the Pseudonymizer data collection. ### Network @@ -179,8 +177,10 @@ The **Repository** settings contain: - Repository maintenance: - [Repository checks](../../../administration/repository_checks.md) - Configure automatic Git checks on repositories. - - [Housekeeping](../../../administration/housekeeping.md)). Configure automatic + - [Housekeeping](../../../administration/housekeeping.md). Configure automatic Git housekeeping on repositories. + - [Inactive project deletion](../../../administration/inactive_project_deletion.md). Configure inactive + project deletion. - [Repository static objects](../../../administration/static_objects_external_storage.md) - Serve repository static objects (for example, archives and blobs) from an external storage (for example, a CDN). diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md index 50dd24de3fb..6c0c15243da 100644 --- a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -15,7 +15,7 @@ To can change its value: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Issues Rate Limits**. -1. Under **Max requests per minute per user**, enter the new value. +1. Under **Max requests per minute**, enter the new value. 1. Select **Save changes**. For example, if you set a limit of 300, requests using the diff --git a/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md new file mode 100644 index 00000000000..2819a18d361 --- /dev/null +++ b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md @@ -0,0 +1,33 @@ +--- +type: reference +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Rate limits on pipeline creation **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362475) in GitLab 15.0. + +You can set a limit so that users and processes can't request more than a certain number of pipelines each minute. This limit can help save resources and improve stability. + +For example, if you set a limit of `10`, and `11` requests are sent to the [trigger API](../../../ci/triggers/) within one minute, +the eleventh request is blocked. Access to the endpoint is allowed again after one minute. + +This limit is: + +- Applied independently per project, user, and commit. +- Not applied per IP address. +- Disabled by default. + +Requests that exceed the limit are logged in the `application_json.log` file. + +## Set a pipeline request limit + +To limit the number of pipeline requests: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Network**. +1. Expand **Pipelines Rate Limits**. +1. Under **Max requests per minute**, enter a value greater than `0`. +1. Select **Save changes**. diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index c63cd88eeb4..7316b1bdbb8 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -66,7 +66,7 @@ Git clients, and access RESTful API endpoints as administrators, without additio authentication steps. We may address these limitations in the future. For more information see the following epic: -[Admin mode for GitLab Administrators](https://gitlab.com/groups/gitlab-org/-/epics/2158). +[Admin Mode for GitLab Administrators](https://gitlab.com/groups/gitlab-org/-/epics/2158). ### Troubleshooting Admin Mode diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 8ce3b4f1c18..534450c1871 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -60,7 +60,7 @@ To enforce confirmation of the email address used for new sign ups: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > General**, and expand **Sign-up restrictions**. -1. Select the **Enable email restrictions for sign ups** checkbox, then select **Save changes**. +1. Select the **Send confirmation email on sign-up** checkbox, then select **Save changes**. ## User cap diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 923ea9e19c1..ce949999fb8 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -1,8 +1,7 @@ --- -stage: none -group: unassigned +stage: Growth +group: Product Intelligence info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Usage statistics **(FREE SELF)** @@ -15,13 +14,54 @@ All usage statistics are [opt-out](#enable-or-disable-usage-statistics). ## Service Ping Service Ping is a process that collects and sends a weekly payload to GitLab Inc. -For more information, see the [Service Ping guide](../../../development/service_ping/index.md). +For more information, see the [Service Ping guide](../../../development/service_ping/index.md). When Service Ping is enabled, GitLab gathers data from other instances and enables certain [instance-level analytics features](../analytics/index.md) +that are dependent on Service Ping. -### Instance-level analytics availability +### Why enable Service Ping? -When Service Ping is enabled, GitLab gathers data from other instances and -enables certain [instance-level analytics features](../analytics/index.md) -that are dependent on Service Ping. +The main purpose of Service Ping is to build a better GitLab. We collect data about how GitLab is used +to understand feature or stage adoption and usage. This data gives an insight into how GitLab adds +value and helps our team understand the reasons why people use GitLab, and with this knowledge we're able to make better product decisions. + +There are several other benefits to enabling Service Ping: + +- Analyze the users' activities over time of your GitLab installation. +- A [DevOps Score](../analytics/dev_ops_reports.md#devops-score) to give you an overview of your entire instance's adoption of concurrent DevOps from planning to monitoring. +- More proactive support (assuming that our TAMs and support organization used the data to deliver more value). +- Insight and advice into how to get the most value out of your investment in GitLab. +- Reports that show how you compare against other similar organizations (anonymized), with specific advice and recommendations on how to improve your DevOps processes. +- Participation in our [Registration Features Program](#registration-features-program) to receive free paid features. + +## Registration Features Program + +> Introduced in GitLab 14.1. + +In GitLab versions 14.1 and later, GitLab Free customers with a self-managed instance running +GitLab Enterprise Edition can receive paid features by registering with GitLab and sending us +activity data through Service Ping. Features introduced here do not remove the feature from its paid +tier. Users can continue to access the features in a paid tier without sharing usage data. + +### Features available in 14.1 and later + +- [Email from GitLab](../email_from_gitlab.md). + +### Features available in 14.4 and later + +- [Repository size limit](../settings/account_and_limit_settings.md#repository-size-limit). +- [Restrict group access by IP address](../../group/index.md#restrict-group-access-by-ip-address). + +NOTE: +Registration is not yet required for participation, but may be added in a future milestone. + +### Enable registration features + +1. Sign in as a user with administrator access. +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling**. +1. Expand the **Usage statistics** section. +1. If not enabled, select the **Enable Service Ping** checkbox. +1. Select the **Enable Registration Features** checkbox. +1. Select **Save changes**. ## Version check @@ -79,6 +119,81 @@ To enable or disable Service Ping and version check: 1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. 1. Select **Save changes**. +## Disable usage statistics with the configuration file + +NOTE: +The method to disable Service Ping in the GitLab configuration file does not work in +GitLab versions 9.3 to 13.12.3. For more information about how to disable it, see [troubleshooting](../../../development/service_ping/troubleshooting.md#cannot-disable-service-ping-with-the-configuration-file). + +To disable Service Ping and prevent it from being configured in the future through +the Admin Area: + +**For installations using the Linux package:** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['usage_ping_enabled'] = false + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +**For installations from source:** + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: &base + # ... + gitlab: + # ... + usage_ping_enabled: false + ``` + +1. Restart GitLab: + + ```shell + sudo service gitlab restart + ``` + +## View the Service Ping payload + +You can view the exact JSON payload sent to GitLab Inc. in the Admin Area. To view the payload: + +1. Sign in as a user with administrator access. +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Metrics and profiling**. +1. Expand the **Usage statistics** section. +1. Select **Preview payload**. + +For an example payload, see [Example Service Ping payload](../../../development/service_ping/index.md#example-service-ping-payload). + +## Manually upload Service Ping payload + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7388) in GitLab 14.8 with a flag named `admin_application_settings_service_usage_data_center`. Disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83265) in GitLab 14.10. + +You can upload the Service Ping payload to GitLab even if your instance doesn't have internet access, +or if the Service Ping [cron job](../../../development/service_ping/index.md#how-service-ping-works) is not enabled. + +To upload the payload manually: + +1. Sign in as a user with administrator access. +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Service** usage data. +1. Select **Download payload**. +1. Save the JSON file. +1. Visit [Service usage data center](https://version.gitlab.com/usage_data/new). +1. Select **Choose file** and choose the file from p5. +1. Select **Upload**. + +The uploaded file is encrypted and sent using secure HTTPS protocol. HTTPS creates a secure +communication channel between web browser and the server, and protects transmitted data against man-in-the-middle attacks. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md index 56e240a8d39..bb3ee64abac 100644 --- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -104,7 +104,7 @@ attached into the response headers. | Header | Example | Description | |:----------------------|:--------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the admin area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | +| `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the Admin Area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | | `RateLimit-Name` | `throttle_authenticated_web` | Name of the throttle blocking the requests. | | `RateLimit-Observed` | `67` | Number of requests associated to the client in the time window. | | `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Observed`. | diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index 704476cdc90..017a0c46570 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -127,12 +127,6 @@ To retrieve metrics for change failure rate, use the [GraphQL](../../api/graphql We use the following terms to describe GitLab analytics: -- **Cycle time:** The duration of only the execution work. Cycle time is often displayed in combination with the lead time, which is longer than the cycle time. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. GitLab displays cycle time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](../analytics/value_stream_analytics.md). -- **Deploys:** The total number of successful deployments to production in the given time frame (across all applicable projects). GitLab displays deploys in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md) and [project-level Value Stream Analytics](value_stream_analytics.md). -- **Lead time:** The duration of your value stream, from start to finish. Different to -[Lead time for changes](#lead-time-for-changes). Often displayed in combination with "cycle time," -which is shorter. GitLab measures lead time from issue creation to issue close. GitLab displays lead -time in [group-level Value Stream Analytics](../group/value_stream_analytics/index.md). - **Mean Time to Change (MTTC):** The average duration between idea and delivery. GitLab measures MTTC from issue creation to the issue's latest related merge request's deployment to production. - **Mean Time to Detect (MTTD):** The average duration that a bug goes undetected in production. @@ -142,11 +136,6 @@ merge request creation to merge request merge (and closed/un-merged merge reques For more information, see [Merge Request Analytics](merge_request_analytics.md). - **Mean Time to Recover/Repair/Resolution/Resolve/Restore (MTTR):** The average duration that a bug is not fixed in production. GitLab measures MTTR from deployment of bug to deployment of fix. -- **Throughput:** The number of issues closed or merge requests merged (not closed) in a period of -time. Often measured per sprint. GitLab displays merge request throughput in [Merge Request Analytics](merge_request_analytics.md). -- **Value Stream:** The entire work process that is followed to deliver value to customers. For example, -the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts -with "plan" and ends with "monitor". GitLab helps you track your value stream using [Value Stream Analytics](value_stream_analytics.md). - **Velocity:** The total issue burden completed in some period of time. The burden is usually measured in points or weight, often per sprint. For example, your velocity may be "30 points per sprint". GitLab measures velocity as the total points or weight of issues closed in a given period of time. diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index 06774c3f16a..29f2ec79800 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -48,7 +48,8 @@ To view the number of merge requests merged per month: - In the **From** field, select a start date. - In the **To** field, select an end date. -The **Throughput** chart shows the number of merge requests merged per month. +The **Throughput** chart shows issues closed or merge requests merged (not closed) over a period of +time. The table shows up to 20 merge requests per page, and includes information about each merge request. @@ -58,9 +59,10 @@ information about each merge request. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229389) in GitLab 13.9. Use the number in **Mean time to merge** to view the average time between when a merge request is -created and when it's merged. +created and when it's merged. Closed and un-merged merge requests are not included. To view **Mean time to merge**: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Merge request**. +1. On the left sidebar, select **Analytics > Merge request**. The **Mean time to merge** number +is shown on the dashboard. diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 92c4d447ed9..039d33a1ad8 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -12,6 +12,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w Value stream analytics provides metrics about each stage of your software development process. +A **value stream** is the entire work process that delivers value to customers. For example, +the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts +with the Manage stage and ends with the Protect stage. + Use value stream analytics to identify: - The amount of time it takes to go from an idea to production. @@ -74,7 +78,7 @@ To view the median time spent in each stage: Value stream analytics shows the lead time and cycle time for issues in your project: - Lead time: Median time from when the issue was created to when it was closed. -- Cycle time: Median time from first commit to issue closed. Commits are associated with issues when users [cross-link them in the commit message](../project/issues/crosslinking_issues.md#from-commit-messages). +- Cycle time: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a [linked issue's merge request](../project/issues/crosslinking_issues.md) to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation is always later than commit time. To view the lead time and cycle time for issues: diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index ed94686b7a3..f97e09f33bb 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -139,7 +139,7 @@ OpenAPI 2.x lets you specify the accepted media types globally or per operation, - In [GitLab 14.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/333304), the default behavior is to select one of the supported media types to use. The first supported media type is chosen from the list. This behavior is configurable. - In GitLab 14.9 and earlier, the default behavior is to perform testing using all supported media types. This means if two media types are listed (for example, `application/json` and `application/xml`), tests are performed using JSON, and then the same tests using XML. -Testing the same operation (for example, `POST /user`) using different media types (for example, `application/json` and `application/xml`) is not always desirable. +Testing the same operation (for example, `POST /user`) using different media types (for example, `application/json` and `application/xml`) is not always desirable. For example, if the target application executes the same code regardless of the request content type, it will take longer to finish the test session, and it may report duplicate vulnerabilities related to the request body depending on the target app. The environment variable `FUZZAPI_OPENAPI_ALL_MEDIA_TYPES` lets you specify whether or not to use all supported media types instead of one when generating requests for a given operation. When the environmental variable `FUZZAPI_OPENAPI_ALL_MEDIA_TYPES` is set to any value, API Fuzzing will try to generate requests for all supported media types instead of one in a given operation. This will cause testing to take longer as testing is repeated for each provided media type. @@ -1087,7 +1087,7 @@ You can provide the following properties to exclude specific parameters during t - `body-json`: Use this property to exclude specific JSON nodes from a request that uses the media type `application/json`. The property's value is an array, each entry of the array is a [JSON Path](https://goessner.net/articles/JsonPath/) expression. - `body-xml`: Use this property to exclude specific XML nodes from a request that uses media type `application/xml`. The property's value is an array, each entry of the array is a [XPath v2](https://www.w3.org/TR/xpath20/) expression. -The following JSON document is an example of the expected structure to exclude parameters. +The following JSON document is an example of the expected structure to exclude parameters. ```json { @@ -1155,11 +1155,11 @@ To exclude the `password` field in a request that uses `application/x-www-form-u The exclude parameters uses `body-form` when the request uses a content type `application/x-www-form-urlencoded`. -##### Excluding a specific JSON nodes using JSON Path +##### Excluding a specific JSON nodes using JSON Path To exclude the `schema` property in the root object, set the `body-json` property's value to an array with the JSON Path expression `[ "$.schema" ]`. -The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. +The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. For instance, the JSON document looks like this: ```json @@ -1168,13 +1168,13 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path, characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path, characters like `$`, `*`, `.` among others have special meaning. -##### Excluding multiple JSON nodes using JSON Path +##### Excluding multiple JSON nodes using JSON Path To exclude the property `password` on each entry of an array of `users` at the root level, set the `body-json` property's value to an array with the JSON Path expression `[ "$.users[*].paswword" ]`. -The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. +The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. For instance, the JSON document looks like this: @@ -1184,7 +1184,7 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. ##### Excluding an XML attribute @@ -1196,17 +1196,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/@isEnabled" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions, characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions, characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element's text -To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. +To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. In the XPath expression `/credentials/username/text()`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. Last part has a `/` that refers to the current element, and uses a XPath function called `text()` which identifies the text of the current element. @@ -1214,17 +1214,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username/text()" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element -To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. +To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. In the XPath expression `/credentials/username`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. @@ -1232,7 +1232,7 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username" ] } @@ -1242,21 +1242,21 @@ The exclude parameters uses `body-xml` when the request uses a content type `app ##### Excluding an XML node with namespaces -To exclude a XML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. +To exclude a XML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. -In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. +In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. The namespace name should have been defined in the XML document which is part of the body request. You may check the namespace in the specification document HAR, OpenAPI, or Postman Collection file. ```json { - "body-xml": [ + "body-xml": [ "/credentials/s:login" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. #### Using a JSON string @@ -1294,7 +1294,7 @@ variables: FUZZAPI_EXCLUDE_PARAMETER_FILE: api-fuzzing-exclude-parameters.json ``` -The `api-fuzzing-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). +The `api-fuzzing-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). ### Exclude URLS @@ -1348,7 +1348,7 @@ variables: ##### Excluding URL using regular expressions -In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. +In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml variables: @@ -1467,7 +1467,7 @@ reported. Faults detected by API Fuzzing occur in the live web application, and require manual investigation to determine if they are vulnerabilities. Fuzzing faults are included as vulnerabilities with a severity of Unknown. To facilitate investigation of the fuzzing faults, detailed information is -provided about the HTTP messages sent and received along with a description of the modification(s) +provided about the HTTP messages sent and received along with a description of the modifications made. Follow these steps to view details of a fuzzing fault: diff --git a/doc/user/application_security/cluster_image_scanning/index.md b/doc/user/application_security/cluster_image_scanning/index.md index 293645b8de6..aba28a5ca89 100644 --- a/doc/user/application_security/cluster_image_scanning/index.md +++ b/doc/user/application_security/cluster_image_scanning/index.md @@ -28,10 +28,17 @@ To integrate GitLab with security scanners other than those listed here, see You can use cluster image scanning through the following methods: -- [The cluster image scanning analyzer](#use-the-cluster-image-scanning-analyzer) +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> +- [The cluster image scanning analyzer](#use-the-cluster-image-scanning-analyzer-removed) ([Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in GitLab 15.0. Use [the GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) instead.) +<!--- end_remove --> - [The GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) -## Use the cluster image scanning analyzer +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> + +## Use the cluster image scanning analyzer (removed) + +This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in GitLab 15.0. +Use [the GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) instead. You can use the cluster image scanning analyzer to run cluster image scanning with [GitLab CI/CD](../../../ci/index.md). To enable the cluster image scanning analyzer, [include the CI job](#configuration) @@ -277,6 +284,7 @@ Here's an example cluster image scanning report: } ``` +<!--- end_remove --> ## Cluster image scanning with the GitLab agent You can use the [GitLab agent](../../clusters/agent/index.md) to @@ -284,14 +292,12 @@ scan images from within your Kubernetes cluster and record the vulnerabilities i ### Prerequisites -- [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/installation/kubectl/) - installed and configured in your cluster. - [GitLab agent](../../clusters/agent/install/index.md) set up in GitLab, installed in your cluster, and configured using a configuration repository. ### Configuration -The agent runs the cluster image scanning once the `cluster_image_scanning` +The agent runs the cluster image scanning once the `starboard` directive is added to your [agent's configuration repository](../../clusters/agent/vulnerabilities.md). ## Security Dashboard @@ -304,9 +310,12 @@ the security vulnerabilities in your groups, projects, and pipelines. After you find a vulnerability, you can address it in the [vulnerability report](../vulnerabilities/index.md) or the [GitLab agent's](../../clusters/agent/vulnerabilities.md) details section. +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> ## Troubleshooting ### Getting warning message `gl-cluster-image-scanning-report.json: no matching files` For information on this error, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + +<!--- end_remove --> diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 64566e458ee..2828b56a5d1 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -5,7 +5,7 @@ group: Container Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Container Scanning **(ULTIMATE)** +# Container Scanning **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3672) in GitLab 10.4. @@ -44,6 +44,26 @@ information directly in the merge request.  +### Capabilities + +| Capability | In Free | In Ultimate | +| --- | ------ | ------ | +| [Configure Scanners](#configuration) | Yes | Yes | +| Customize Settings ([Variables](#available-cicd-variables), [Overriding](#overriding-the-container-scanning-template), [offline environment support](#running-container-scanning-in-an-offline-environment), etc) | Yes | Yes | +| [View JSON Report](#reports-json-format) as a CI job artifact | Yes | Yes | +| Generation of a JSON report of [dependencies](#dependency-list) as a CI job artifact | Yes | Yes | +| Ability to enable container scanning via an MR in the GitLab UI | Yes | Yes | +| [UBI Image Support](#fips-enabled-images) | Yes | Yes | +| Support for Trivy | Yes | Yes | +| Support for Grype | Yes | Yes | +| Inclusion of GitLab Advisory Database | Limited to the time-delayed content from GitLab [advisories-communities](https://gitlab.com/gitlab-org/advisories-community/) project | Yes - all the latest content from [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) | +| Presentation of Report data in Merge Request and Security tab of the CI pipeline job | No | Yes | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) such as merge request approvals | No | Yes | +| [Solutions for vulnerabilities (auto-remediation)](#solutions-for-vulnerabilities-auto-remediation) | No | Yes | +| Support for the [vulnerability allow list](#vulnerability-allowlisting) | No | Yes | +| [Access to Security Dashboard page](#security-dashboard) | No | Yes | +| [Access to Dependency List page](../dependency_list/) | No | Yes | + ## Requirements To enable container scanning in your pipeline, you need the following: @@ -164,8 +184,8 @@ include: The `CS_DISABLE_DEPENDENCY_LIST` CI/CD variable controls whether the scan creates a [Dependency List](../dependency_list/) -report. The variable's default setting of `false` causes the scan to create the report. To disable -the report, set the variable to `true`: +report. This variable is currently only supported when the `trivy` analyzer is used. The variable's default setting of `"false"` causes the scan to create the report. To disable +the report, set the variable to `"true"`: For example: @@ -205,8 +225,9 @@ container_scanning: When you enable this feature, you may see [duplicate findings](../terminology/#duplicate-finding) in the [Vulnerability Report](../vulnerability_report/) if [Dependency Scanning](../dependency_scanning/) -is enabled for your project. This happens because GitLab can't automatically deduplicate the -findings reported by the two different analyzers. +is enabled for your project. This happens because GitLab can't automatically deduplicate findings +across different types of scanning tools. Please reference [this comparison](../dependency_scanning/#dependency-scanning-compared-to-container-scanning) +between GitLab Dependency Scanning and Container Scanning for more details on which types of dependencies are likely to be duplicated. #### Available CI/CD variables @@ -217,7 +238,7 @@ You can [configure](#customizing-the-container-scanning-settings) analyzers by u | `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | All | | `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | All | | `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | All | -| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:4` | Docker image of the analyzer. | All | +| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:5` | Docker image of the analyzer. | All | | `CS_DEFAULT_BRANCH_IMAGE` | `""` | The name of the `DOCKER_IMAGE` on the default branch. See [Setting the default branch image](#setting-the-default-branch-image) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5. | All | | `CS_DISABLE_DEPENDENCY_LIST` | `"false"` | Disable Dependency Scanning for packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` | `"true"` | Disable scanning for language-specific packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | @@ -234,10 +255,24 @@ You can [configure](#customizing-the-container-scanning-settings) analyzers by u ### Supported distributions -Support depends on the scanner: - -- [Grype](https://github.com/anchore/grype#grype) -- [Trivy](https://aquasecurity.github.io/trivy/latest/docs/vulnerability/detection/os/) (Default). +Support depends on which scanner is used: + +| Distribution | Grype | Trivy | +| -------------- | ----- | ----- | +| Alma Linux | | ✅ | +| Alpine Linux | ✅ | | +| Amazon Linux | ✅ | ✅ | +| BusyBox | ✅ | | +| CentOS | ✅ | ✅ | +| CBL-Mariner | | ✅ | +| Debian | ✅ | ✅ | +| Distroless | ✅ | ✅ | +| Oracle Linux | ✅ | ✅ | +| Photon OS | | ✅ | +| Red Hat (RHEL) | ✅ | ✅ | +| Rocky Linux | | ✅ | +| SUSE | | ✅ | +| Ubuntu | ✅ | ✅ | #### FIPS-enabled images @@ -250,9 +285,9 @@ standard tag plus the `-fips` extension. | Scanner name | `CS_ANALYZER_IMAGE` | | --------------- | ------------------- | -| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:4-fips` | -| Grype | `registry.gitlab.com/security-products/container-scanning/grype:4-fips` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4-fips` | +| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:5-fips` | +| Grype | `registry.gitlab.com/security-products/container-scanning/grype:5-fips` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5-fips` | NOTE: Prior to GitLab 15.0, the `-ubi` image extension is also available. GitLab 15.0 and later only @@ -305,9 +340,9 @@ The following options are available: | Scanner name | `CS_ANALYZER_IMAGE` | | ------------ | ------------------- | -| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:4` | -| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:4` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4` | +| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:5` | +| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:5` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5` | If you're migrating from a GitLab 13.x release to a GitLab 14.x release and have customized the `container_scanning` job or its CI variables, you might need to perform these migration steps in @@ -320,7 +355,7 @@ your CI file: - `SECURE_ANALYZERS_PREFIX` 1. Review the `CS_ANALYZER_IMAGE` variable. It no longer depends on the variables above and its new - default value is `registry.gitlab.com/security-products/container-scanning:4`. If you have an + default value is `registry.gitlab.com/security-products/container-scanning:5`. If you have an offline environment, see [Running container scanning in an offline environment](#running-container-scanning-in-an-offline-environment). @@ -405,7 +440,7 @@ container_scanning: The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. -### Vulnerability allowlisting +### Vulnerability allowlisting **(ULTIMATE)** To allowlist specific vulnerabilities, follow these steps: @@ -532,9 +567,9 @@ For container scanning, import the following images from `registry.gitlab.com` i [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/container-scanning:4 -registry.gitlab.com/security-products/container-scanning/grype:4 -registry.gitlab.com/security-products/container-scanning/trivy:4 +registry.gitlab.com/security-products/container-scanning:5 +registry.gitlab.com/security-products/container-scanning/grype:5 +registry.gitlab.com/security-products/container-scanning/trivy:5 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -574,7 +609,7 @@ following `.gitlab-ci.yml` example as a template. ```yaml variables: - SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:4 + SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:5 TARGET_IMAGE: $CI_REGISTRY/namespace/gitlab-container-scanning image: docker:stable @@ -753,15 +788,38 @@ Here's an example container scanning report: The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all the security vulnerabilities in your groups, projects and pipelines. -## Vulnerabilities database update +## Vulnerabilities database All analyzer images are [updated daily](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/blob/master/README.md#image-updates). -The images include the latest advisory database available for their respective scanner. Each -scanner includes data from multiple sources: - -- [Grype](https://github.com/anchore/grype#grypes-database). -- [Trivy](https://aquasecurity.github.io/trivy/latest/docs/vulnerability/detection/data-source/). +The images use data from upstream advisory databases depending on which scanner is used: + +| Data Source | Trivy | Grype | +| ------------------------------ | ----- | ----- | +| AlmaLinux Security Advisory | ✅ | ✅ | +| Amazon Linux Security Center | ✅ | ✅ | +| Arch Linux Security Tracker | ✅ | | +| SUSE CVRF | ✅ | ✅ | +| CWE Advisories | ✅ | | +| Debian Security Bug Tracker | ✅ | ✅ | +| GitHub Security Advisory | ✅ | ✅ | +| Go Vulnerability Database | ✅ | | +| CBL-Mariner Vulnerability Data | ✅ | | +| NVD | ✅ | ✅ | +| OSV | ✅ | | +| Red Hat OVAL v2 | ✅ | ✅ | +| Red Hat Security Data API | ✅ | ✅ | +| Photon Security Advisories | ✅ | | +| Rocky Linux UpdateInfo | ✅ | | +| Ubuntu CVE Tracker (only data sources from mid 2021 and later) | ✅ | ✅ | + +In addition to the sources provided by these scanners, GitLab maintains the following vulnerability databases: + +- The proprietary +[GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). +- The open source [GitLab Advisory Database (Open Source Edition)](https://gitlab.com/gitlab-org/advisories-community). + +In the GitLab Ultimate tier, the data from the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is merged in to augment the data from the external sources. In the GitLab Premium and Free tiers, the data from the [GitLab Advisory Database (Open Source Edition)](https://gitlab.com/gitlab-org/advisories-community) is merged in to augment the data from the external sources. This augmentation currently only applies to the analyzer images for the Trivy scanner. Database update information for other analyzers is available in the [maintenance table](../index.md#vulnerability-scanner-maintenance). @@ -770,7 +828,7 @@ Database update information for other analyzers is available in the After a vulnerability is found, you can [address it](../vulnerabilities/index.md). -## Solutions for vulnerabilities (auto-remediation) +## Solutions for vulnerabilities (auto-remediation) **(ULTIMATE)** Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. @@ -827,6 +885,7 @@ For information on this, see the [general Application Security troubleshooting s as the default for container scanning, and also [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326279) an integration with [Grype](https://github.com/anchore/grype) as an alternative scanner. +- GitLab 15.0 changed the major analyzer version from `4` to `5`. Other changes to the container scanning analyzer can be found in the project's [changelog](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/blob/master/CHANGELOG.md). diff --git a/doc/user/application_security/dast/checks/16.3.md b/doc/user/application_security/dast/checks/16.3.md index e4fc2468dae..6f80a2a32c6 100644 --- a/doc/user/application_security/dast/checks/16.3.md +++ b/doc/user/application_security/dast/checks/16.3.md @@ -32,4 +32,4 @@ information from the `X-Powered-By` header. ## Links - [CWE](https://cwe.mitre.org/data/definitions/16.html) -- [PHP expose_php](https://www.php.net/manual/en/ini.core.php#ini.expose-php) +- [PHP `expose_php`](https://www.php.net/manual/en/ini.core.php#ini.expose-php) diff --git a/doc/user/application_security/dast/checks/16.5.md b/doc/user/application_security/dast/checks/16.5.md index 28bb9f7ee4b..e03da3043ef 100644 --- a/doc/user/application_security/dast/checks/16.5.md +++ b/doc/user/application_security/dast/checks/16.5.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -The target website returns AspNet header(s) and version information of this website. By +The target website returns AspNet headers and version information of this website. By exposing these values attackers may attempt to identify if the target software is vulnerable to known vulnerabilities, or catalog known sites running particular versions to exploit in the future when a vulnerability is identified in the particular version. diff --git a/doc/user/application_security/dast/checks/16.6.md b/doc/user/application_security/dast/checks/16.6.md index ddd3a10c5f8..9cbcde669a0 100644 --- a/doc/user/application_security/dast/checks/16.6.md +++ b/doc/user/application_security/dast/checks/16.6.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -The target website returns AspNet header(s) along with version information of this website. By +The target website returns AspNet headers along with version information of this website. By exposing these values attackers may attempt to identify if the target software is vulnerable to known vulnerabilities. Or catalog known sites running particular versions to exploit in the future when a vulnerability is identified in the particular version. diff --git a/doc/user/application_security/dast/checks/359.1.md b/doc/user/application_security/dast/checks/359.1.md new file mode 100644 index 00000000000..af1fdf8a596 --- /dev/null +++ b/doc/user/application_security/dast/checks/359.1.md @@ -0,0 +1,34 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of Private Personal Information (PII) to an unauthorized actor (credit card) + +## Description + +The target application was found to return credit card information in the response. Organizations +found returning such information may be in violation of industry regulations and could face fines. + +## Remediation + +PII such as credit cards should never be directly returned to the user. The majority of the information should masked except +the last few digits or characters of the identifier. For example, credit card numbers should +only return the last four digits: `****-****-****-1234`. Ensure this masking is done on the server +and only then send the masked data back to the client. Do not rely on client side JavaScript or other methods +to mask these values as the data could still be intercepted or unmasked. + +Additionally, credit card information should never be stored un-encrypted in files or databases. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 359.1 | true | 359 | Passive | Medium | + +## Links + +- [OWASP Top 10 A3 2017 - Sensitive Data Exposure](https://owasp.org/www-project-top-ten/2017/A3_2017-Sensitive_Data_Exposure) +- [CWE](https://cwe.mitre.org/data/definitions/359.html) +- [PCI-DSS](https://www.pcisecuritystandards.org/pdfs/pci_fs_data_storage.pdf) diff --git a/doc/user/application_security/dast/checks/359.2.md b/doc/user/application_security/dast/checks/359.2.md new file mode 100644 index 00000000000..beb99e26097 --- /dev/null +++ b/doc/user/application_security/dast/checks/359.2.md @@ -0,0 +1,34 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of Private Personal Information (PII) to an unauthorized actor (United States social security number) + +## Description + +The target application was found to return social security number (SSN) information in the response. Organizations +found returning such information may be in violation of (United States) state or federal laws and may face stiff penalties. + +## Remediation + +PII such as social security numbers should never be directly returned to the user. The majority of the information +should masked except the last few digits or characters of the identifier. For example, social security numbers +only be displayed with the last four digits: `***-**-1234`. Ensure this masking is done on the server +and only then send the masked data back to the client. Do not rely on client side JavaScript or other methods +to mask these values as the data could still be intercepted or unmasked. + +Additionally, social security numbers should never be stored un-encrypted in files or databases. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 359.2 | true | 359 | Passive | Medium | + +## Links + +- [OWASP Top 10 A3 2017 - Sensitive Data Exposure](https://owasp.org/www-project-top-ten/2017/A3_2017-Sensitive_Data_Exposure) +- [CWE](https://cwe.mitre.org/data/definitions/359.html) +- [Privacy Act (CMPPA)](https://www.ssa.gov/dataexchange/privacyinfo.html) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index 764e3c4a839..629ff1c3a8d 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -18,6 +18,8 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne | [16.5](16.5.md) | AspNet header exposes version information | Low | Passive | | [16.6](16.6.md) | AspNetMvc header exposes version information | Low | Passive | | [200.1](200.1.md) | Exposure of sensitive information to an unauthorized actor (private IP address) | Low | Passive | +| [359.1](359.1.md) | Exposure of Private Personal Information (PII) to an unauthorized actor (credit card) | Medium | Passive | +| [359.2](359.2.md) | Exposure of Private Personal Information (PII) to an unauthorized actor (United States social security number) | Medium | Passive | | [548.1](548.1.md) | Exposure of information through directory listing | Low | Passive | | [598.1](598.1.md) | Use of GET request method with sensitive query strings (session ID) | Medium | Passive | | [598.2](598.2.md) | Use of GET request method with sensitive query strings (password) | Medium | Passive | diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index ee57803dfc7..1389db65713 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -627,7 +627,7 @@ These CI/CD variables are specific to DAST. They can be used to customize the be | `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. | | `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080`. | | `DAST_API_OPENAPI` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | -| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290241) in GitLab 13.12 and replaced by `DAST_API_OPENAPI`. To be removed in GitLab 15.0. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290241)** in GitLab 15.0. Replaced by `DAST_API_OPENAPI`. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | | `DAST_AUTH_REPORT` <sup>2</sup> | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | | `DAST_AUTH_EXCLUDE_URLS` <sup>2</sup> | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | | `DAST_AUTH_URL` <sup>1,2</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. Example: `https://login.example.com`. | @@ -1156,6 +1156,7 @@ A site profile contains the following: - **Password**: The password used to authenticate to the website. - **Username form field**: The name of username field at the sign-in HTML form. - **Password form field**: The name of password field at the sign-in HTML form. + - **Submit form field**: The `id` or `name` of the element that when clicked submits the sign-in HTML form. When an API site type is selected, a [host override](#host-override) is used to ensure the API being scanned is on the same host as the target. This is done to reduce the risk of running an active scan against the wrong API. @@ -1199,7 +1200,14 @@ The site profile is created. #### Edit a site profile -To edit an existing site profile: +If a site profile is linked to a security policy, a user cannot edit the profile from this page. See +[Scan execution policies](../policies/scan-execution-policies.md) +for more information. + +When a validated site profile's file, header, or meta tag is edited, the site's +[validation status](#site-profile-validation) is revoked. + +To edit a site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. @@ -1207,42 +1215,37 @@ To edit an existing site profile: 1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the fields then select **Save profile**. -If a site profile is linked to a security policy, a user cannot edit the profile from this page. See -[Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Delete a site profile -To delete an existing site profile: +If a site profile is linked to a security policy, a user cannot delete the profile from this page. +See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + +To delete a site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. 1. Select the **Site Profiles** tab. -1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. +1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete** to confirm the deletion. -If a site profile is linked to a security policy, a user cannot delete the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Validate a site profile -Prerequisites: - -- A site profile. +Validating a site is required to run an active scan. To validate a site profile: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. -1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage scans**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. -1. In the profile's row select **Validate** or **Retry validation**. +1. In the profile's row, select **Validate**. 1. Select the validation method. 1. For **Text file validation**: 1. Download the validation file listed in **Step 2**. - 1. Upload the validation file to the host. Upload the file to the location in - **Step 3** or any location you prefer. + 1. Upload the validation file to the host, to the location in **Step 3** or any location you + prefer. + 1. If required, edit the file location in **Step 3**. 1. Select **Validate**. 1. For **Header validation**: 1. Select the clipboard icon in **Step 2**. @@ -1255,9 +1258,8 @@ To validate a site profile: 1. Select the input field in **Step 3** and enter the location of the meta tag. 1. Select **Validate**. -The site is validated and an active scan can run against it. - -If a validated site profile's target URL is edited, the site's validation status is revoked. +The site is validated and an active scan can run against it. A site profile's validation status is +revoked only when it's revoked manually, or its file, header, or meta tag is edited. #### Retry a failed validation @@ -1265,22 +1267,28 @@ If a validated site profile's target URL is edited, the site's validation status > - [Deployed behind the `dast_failed_site_validations` flag](../../../administration/feature_flags.md), enabled by default. > - [Feature flag `dast_failed_site_validations` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323961) in GitLab 14.4. -If a site profile's validation fails, you can retry it by selecting the **Retry validation** button -in the profiles list. +Failed site validation attempts are listed on the **Site profiles** tab of the **Manage profiles** +page. + +To retry a site profile's failed validation: -When loading the DAST profiles library, past failed validations are listed above the profiles -list. You can also retry the validation from there by selecting the **Retry validation** link in -the alert. You can also dismiss the alert to revoke failed validations. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select the **Site Profiles** tab. +1. In the profile's row, select **Retry validation**. #### Revoke a site profile's validation status -Note that all site profiles with the same URL have their validation status revoked. +WARNING: +When a site profile's validation status is revoked, all site profiles that share the same URL also +have their validation status revoked. To revoke a site profile's validation status: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. -1. Select **Revoke validation** beside the validated profile. +1. Beside the validated profile, select **Revoke validation**. The site profile's validation status is revoked. @@ -1348,40 +1356,40 @@ A scanner profile defines the scanner settings used to run an on-demand scan: To create a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row select **Manage**. +1. In the **DAST Profiles** row, select **Manage**. 1. Select **New > Scanner Profile**. 1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile). -1. Click **Save profile**. +1. Select **Save profile**. #### Edit a scanner profile +If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. +See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + To edit a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. +1. In the **DAST Profiles** row, select **Manage**. 1. Select the **Scanner Profiles** tab. -1. In the scanner's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. +1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the form. 1. Select **Save profile**. -If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Delete a scanner profile +If a scanner profile is linked to a security policy, a user cannot delete the profile from this +page. See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + To delete a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. +1. In the **DAST Profiles** row, select **Manage**. 1. Select the **Scanner Profiles** tab. -1. In the scanner's row select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. +1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete**. -If a scanner profile is linked to a security policy, a user cannot delete the profile from this -page. See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - ## Auditing > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index a4908204b60..a1b19c52b20 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -1041,7 +1041,7 @@ You can provide the following properties to exclude specific parameters during t - `body-json`: Use this property to exclude specific JSON nodes from a request that uses the media type `application/json`. The property's value is an array, each entry of the array is a [JSON Path](https://goessner.net/articles/JsonPath/) expression. - `body-xml`: Use this property to exclude specific XML nodes from a request that uses media type `application/xml`. The property's value is an array, each entry of the array is a [XPath v2](https://www.w3.org/TR/xpath20/) expression. -Thus, the following JSON document is an example of the expected structure to exclude parameters. +Thus, the following JSON document is an example of the expected structure to exclude parameters. ```json { @@ -1109,11 +1109,11 @@ To exclude the `password` field in a request that uses `application/x-www-form-u The exclude parameters uses `body-form` when the request uses a content type `application/x-www-form-urlencoded`. -##### Excluding a specific JSON nodes using JSON Path +##### Excluding a specific JSON nodes using JSON Path To exclude the `schema` property in the root object, set the `body-json` property's value to an array with the JSON Path expression `[ "$.schema" ]`. -The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. +The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. For instance, the JSON document looks like this: ```json @@ -1122,13 +1122,13 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. -##### Excluding multiple JSON nodes using JSON Path +##### Excluding multiple JSON nodes using JSON Path To exclude the property `password` on each entry of an array of `users` at the root level, set the `body-json` property's value to an array with the JSON Path expression `[ "$.users[*].paswword" ]`. -The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. +The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. For instance, the JSON document looks like this: @@ -1138,7 +1138,7 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. ##### Excluding a XML attribute @@ -1150,17 +1150,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/@isEnabled" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding a XML text's element -To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. +To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. In the XPath expression `/credentials/username/text()`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. Last part has a `/` that refers to the current element, and uses a XPath function called `text()` which identifies the text of the current element. @@ -1168,17 +1168,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username/text()" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element -To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. +To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. In the XPath expression `/credentials/username`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. @@ -1186,31 +1186,31 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML node with namespaces -To exclude anXML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. +To exclude anXML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. -In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. +In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. The namespace name should have been defined in the XML document which is part of the body request. You may check the namespace in the specification document HAR, OpenAPI, or Postman Collection file. ```json { - "body-xml": [ + "body-xml": [ "/credentials/s:login" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath, expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath, expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. #### Using a JSON string @@ -1248,7 +1248,7 @@ variables: DAST_API_EXCLUDE_PARAMETER_FILE: dast-api-exclude-parameters.json ``` -The `dast-api-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). +The `dast-api-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). ### Exclude URLS @@ -1302,7 +1302,7 @@ variables: ##### Excluding URL using regular expressions -In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. +In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml variables: diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 665d29c4017..acbc94cba47 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -20,11 +20,9 @@ This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/ Dependency Scanning supports the following official analyzers: -- [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) - [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) - [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) - [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python) -- [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) The analyzers are published as Docker images, which Dependency Scanning uses to launch dedicated containers for each analysis. @@ -34,11 +32,13 @@ The Dependency Scanning analyzers' current major version number is 2. Dependency Scanning is pre-configured with a set of **default images** that are maintained by GitLab, but users can also integrate their own **custom images**. -WARNING: -The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> -WARNING: -The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). +The [`bundler-audit`](https://gitlab.com/gitlab-org/gitlab/-/issues/289832) and [`retire.js`](https://gitlab.com/gitlab-org/gitlab/-/issues/350510) analyzers were deprecated +in GitLab 14.8 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86704) in 15.0. +Use Gemnasium instead. + +<!--- end_remove --> ## Official default analyzers @@ -67,7 +67,7 @@ the official analyzers. ### Disable specific analyzers You can select the official analyzers you don't want to run. Here's how to disable -`bundler-audit` and `gemnasium` analyzers. +the `gemnasium` analyzer. In `.gitlab-ci.yml` define: ```yaml @@ -75,7 +75,7 @@ include: template: Security/Dependency-Scanning.gitlab-ci.yml variables: - DS_EXCLUDED_ANALYZERS: "bundler-audit, gemnasium" + DS_EXCLUDED_ANALYZERS: "gemnasium" ``` ### Disabling default analyzers @@ -88,7 +88,7 @@ include: template: Security/Dependency-Scanning.gitlab-ci.yml variables: - DS_EXCLUDED_ANALYZERS: "gemnasium, gemnasium-maven, gemnasium-python, bundler-audit, retire.js" + DS_EXCLUDED_ANALYZERS: "gemnasium, gemnasium-maven, gemnasium-python" ``` This is used when one totally relies on [custom analyzers](#custom-analyzers). @@ -117,25 +117,25 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Analyzers data -The following table lists the data available for each official analyzer. - -| Property \ Tool | Gemnasium | bundler-audit | Retire.js | -|---------------------------------------|:------------------:|:------------------:|:------------------:| -| Severity | 𐄂 | ✓ | ✓ | -| Title | ✓ | ✓ | ✓ | -| File | ✓ | ⚠ | ✓ | -| Start line | 𐄂 | 𐄂 | 𐄂 | -| End line | 𐄂 | 𐄂 | 𐄂 | -| External ID (for example, CVE) | ✓ | ✓ | ⚠ | -| URLs | ✓ | ✓ | ✓ | -| Internal doc/explanation | ✓ | 𐄂 | 𐄂 | -| Solution | ✓ | ✓ | 𐄂 | -| Confidence | 𐄂 | 𐄂 | 𐄂 | -| Affected item (for example, class or package) | ✓ | ✓ | ✓ | -| Source code extract | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | 𐄂 | 𐄂 | -| Date | ✓ | 𐄂 | 𐄂 | -| Credits | ✓ | 𐄂 | 𐄂 | +The following table lists the data available for the Gemnasium analyzer. + +| Property \ Tool | Gemnasium | +|---------------------------------------|:------------------:| +| Severity | 𐄂 | +| Title | ✓ | +| File | ✓ | +| Start line | 𐄂 | +| End line | 𐄂 | +| External ID (for example, CVE) | ✓ | +| URLs | ✓ | +| Internal doc/explanation | ✓ | +| Solution | ✓ | +| Confidence | 𐄂 | +| Affected item (for example, class or package) | ✓ | +| Source code extract | 𐄂 | +| Internal ID | ✓ | +| Date | ✓ | +| Credits | ✓ | - ✓ => we have that data - ⚠ => we have that data, but it's partially reliable, or we need to extract that data from unstructured content diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 924e3838d91..87d49ffa324 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -17,9 +17,11 @@ aspects of inspecting the items your code uses. These items typically include ap dependencies that are almost always imported from external sources, rather than sourced from items you wrote yourself. +## Dependency Scanning compared to Container Scanning + GitLab offers both Dependency Scanning and Container Scanning to ensure coverage for all of these dependency types. To cover as much of your risk area as -possible, we encourage you to use all of our security scanners: +possible, we encourage you to use all of our security scanning tools: - Dependency Scanning analyzes your project and tells you which software dependencies, including upstream dependencies, have been included in your project, and what known @@ -41,6 +43,21 @@ possible, we encourage you to use all of our security scanners: efforts to de-duplicate these findings can be tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348655). +The following table summarizes which types of dependencies each scanning tool can detect: + +| Feature | Dependency Scanning | Container Scanning | +| ----------------------------------------------------------- | ------------------- | ------------------ | +| Identify the manifest, lock file, or static file that introduced the dependency | **{check-circle}** | **{dotted-circle}** | +| Development dependencies | **{check-circle}** | **{dotted-circle}** | +| Dependencies in a lock file committed to your repository | **{check-circle}** | **{check-circle}** <sup>1</sup> | +| Binaries built by Go | **{dotted-circle}** | **{check-circle}** <sup>2</sup> | +| Dynamically-linked language-specific dependencies installed by the Operating System | **{dotted-circle}** | **{check-circle}** | +| Operating system dependencies | **{dotted-circle}** | **{check-circle}** | +| Language-specific dependencies installed on the operating system (not built by your project) | **{dotted-circle}** | **{check-circle}** | + +1. Lock file must be present in the image to be detected. +1. Binary file must be present in the image to be detected. + ## Overview If you're using [GitLab CI/CD](../../../ci/index.md), you can use dependency scanning to analyze @@ -136,9 +153,9 @@ table.supported-languages ul { </thead> <tbody> <tr> - <td rowspan="2">Ruby</td> - <td rowspan="2">N/A</td> - <td rowspan="2"><a href="https://bundler.io/">Bundler</a></td> + <td>Ruby</td> + <td>N/A</td> + <td><a href="https://bundler.io/">Bundler</a></td> <td> <ul> <li><code>Gemfile.lock</code></li> @@ -149,11 +166,6 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td><code>Gemfile.lock</code></td> - <td><a href="https://github.com/rubysec/bundler-audit">bundler-audit</a></td> - <td>N</td> - </tr> - <tr> <td>PHP</td> <td>N/A</td> <td><a href="https://getcomposer.org/">Composer</a></td> @@ -200,9 +212,9 @@ table.supported-languages ul { <td>N</td> </tr> <tr> - <td rowspan="3">JavaScript</td> - <td rowspan="2">N/A</td> - <td rowspan="2"><a href="https://www.npmjs.com/">npm</a></td> + <td rowspan="2">JavaScript</td> + <td>N/A</td> + <td><a href="https://www.npmjs.com/">npm</a></td> <td> <ul> <li><code>package-lock.json</code></li> @@ -213,11 +225,6 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td><code>package.json</code></td> - <td><a href="https://retirejs.github.io/retire.js/">Retire.js</a></td> - <td>N</td> - </tr> - <tr> <td>N/A</td> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> <td><code>yarn.lock</code></td> @@ -236,8 +243,8 @@ table.supported-languages ul { <td>C#</td> </tr> <tr> - <td rowspan="3">Python</td> - <td rowspan="3">3.6</td> + <td rowspan="4">Python</td> + <td rowspan="4">3.9</td> <td><a href="https://setuptools.readthedocs.io/en/latest/">setuptools</a></td> <td><code>setup.py</code></td> <td><a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a></td> @@ -267,6 +274,12 @@ table.supported-languages ul { <td>N</td> </tr> <tr> + <td><a href="https://python-poetry.org/">Poetry</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></td> + <td><code>poetry.lock</code></td> + <td><a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a></td> + <td>N</td> + </tr> + <tr> <td>Scala</td> <td>N/A</td> <td><a href="https://www.scala-sbt.org/">sbt</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-3">3</a></b></sup></td> @@ -305,6 +318,14 @@ table.supported-languages ul { Support for <a href="https://www.scala-sbt.org/">sbt</a> 1.3 and above was added in GitLab 13.9. </p> </li> + <li> + <a id="notes-regarding-supported-languages-and-package-managers-4"></a> + <p> + Support for <a href="https://python-poetry.org/">Poetry</a> projects with a <code>poetry.lock</code> file was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/7006">added in GitLab 15.0</a>. + Support for projects without a <code>poetry.lock</code> file is tracked in issue: + <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/32774">Poetry's pyproject.toml support for dependency scanning.</a> + </p> + </li> </ol> <!-- markdownlint-enable MD044 --> @@ -321,13 +342,14 @@ The following package managers use lockfiles that GitLab analyzers are capable o | Package Manager | Supported File Format Versions | Tested Versions | | ------ | ------ | ------ | -| Bundler | N/A | [1.17.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/ruby-bundler/main/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | -| Composer | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/php-composer/main/composer.lock) | -| Conan | 0.4 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/c-conan/main/conan.lock) | -| Go | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/main/go.sum) | -| NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/main/src/web.api/packages.lock.json#L2) | -| npm | v1, v2 | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/main/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4) | -| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/main/yarn.lock#L2) | +| Bundler | N/A | [1.17.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/ruby-bundler/default/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | +| Composer | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/php-composer/default/composer.lock) | +| Conan | 0.4 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/c-conan/default/conan.lock) | +| Go | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/default/go.sum) | +| NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | +| npm | v1, v2 | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/default/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4) | +| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/default/yarn.lock#L2) | +| Poetry | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v3/qa/fixtures/python-poetry/default/poetry.lock) | #### Obtaining dependency information by running a package manager to generate a parsable file @@ -338,25 +360,18 @@ To support the following package managers, the GitLab analyzers proceed in two s | Package Manager | Pre-installed Versions | Tested Versions | | ------ | ------ | ------ | -| Bundler | [2.1.4](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/blob/v2.11.3/Dockerfile#L15)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [1.17.3](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/master/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | | sbt | [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L443-447), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L449-453), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L455-459), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L461-465), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L467-471), [1.5.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L473-477), [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L479-483) | | Maven | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L95-97) | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L95-97) | -| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.26.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup> | [5.6.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L319-323), [6.7](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L286-288)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup>, [6.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L331-335), [7.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L300-302)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup> | +| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup>, [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.26.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [5.6.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L319-323), [6.7](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L286-288)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [6.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L331-335), [7.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L300-302)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup> | | setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L224-247) | | pip | [20.2.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L26) | [20.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L77-91) | -| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L168-191)<sup><b><a href="#exported-dependency-information-notes-4">4</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L143-166) | +| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L168-191)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L143-166) | <!-- markdownlint-disable MD044 --> <ol> <li> <a id="exported-dependency-information-notes-1"></a> <p> - The pre-installed and tested version of <code>Bundler</code> is only used for the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit">bundler-audit</a> analyzer, and is not used for <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">gemnasium</a>. - </p> - </li> - <li> - <a id="exported-dependency-information-notes-2"></a> - <p> Different versions of Java require different versions of Gradle. The versions of Gradle listed in the above table are pre-installed in the analyzer image. The version of Gradle used by the analyzer depends on whether your project uses a <code>gradlew</code> (Gradle wrapper) file or not: @@ -383,13 +398,13 @@ To support the following package managers, the GitLab analyzers proceed in two s </ul> </li> <li> - <a id="exported-dependency-information-notes-3"></a> + <a id="exported-dependency-information-notes-2"></a> <p> These tests confirm that if a <code>gradlew</code> file does not exist, the version of <code>Gradle</code> pre-installed in the analyzer image is used. </p> </li> <li> - <a id="exported-dependency-information-notes-4"></a> + <a id="exported-dependency-information-notes-3"></a> <p> This test confirms that if a <code>Pipfile.lock</code> file is found, it will be used by <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a> to scan the exact package versions listed in this file. </p> @@ -411,35 +426,28 @@ NOTE: If you've run into problems while scanning multiple files, please contribute a comment to [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337056). -#### Ruby - -The following analyzers are executed, each of which have different behavior when processing multiple files: - -- [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) - - Supports multiple lockfiles. - -- [bundler-audit](https://github.com/rubysec/bundler-audit) - - Does not support multiple lockfiles. When multiple lockfiles exist, `bundler-audit` - analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. +#### Python -WARNING: -The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). +We only execute one installation in the directory where either a requirements file or a lock file has been detected. Dependencies are only analyzed by `gemnasium-python` for the first file that is detected. Files are searched for in the following order: -#### Python +1. `requirements.txt`, `requirements.pip`, or `requires.txt` for projects using Pip. +1. `Pipfile` or `Pipfile.lock` for projects using Pipenv. +1. `poetry.lock` for projects using Poetry. +1. `setup.py` for project using Setuptools. -We only execute one installation in the directory where a requirements file has been detected, such as `requirements.txt` or any -variation of this file (for example, `requirements.pip` or `requires.txt`). +The search begins with the root directory and then continues with subdirectories if no builds are found in the root directory. Consequently a Poetry lock file in the root directory would be detected before a Pipenv file in a subdirectory. #### Java and Scala -We only execute one build in the directory where a build file has been detected, such as `build.sbt` or `build.gradle`. -Please note, we support the following types of Java project structures: +We only execute one build in the directory where a build file has been detected. For large projects that include +multiple Gradle, Maven, or sbt builds, or any combination of these, `gemnasium-maven` only analyzes dependencies for the first build file +that is detected. Build files are searched for in the following order: + +1. `build.gradle` or `build.gradle.kts` for single or [multi-project](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) Gradle builds. +1. `pom.xml` for single or [multi-module](https://maven.apache.org/pom.html#Aggregation) Maven projects. +1. `build.sbt` for single or [multi-project](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) sbt builds. -- [multi-project sbt builds](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) -- [multi-project Gradle builds](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) -- [multi-module maven projects](https://maven.apache.org/pom.html#Aggregation) +The search begins with the root directory and then continues with subdirectories if no builds are found in the root directory. Consequently an sbt build file in the root directory would be detected before a Gradle build file in a subdirectory. #### JavaScript @@ -454,26 +462,20 @@ The following analyzers are executed, each of which have different behavior when Does not support multiple lockfiles. When multiple lockfiles exist, `Retire.js` analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. -From GitLab 14.8 the `Gemnasium` analyzer scans supported JavaScript projects for vendored libraries +From GitLab 14.8 the `gemnasium` analyzer scans supported JavaScript projects for vendored libraries (that is, those checked into the project but not managed by the package manager). -WARNING: -The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). -We execute both analyzers because they use different sources of vulnerability data. The result is more comprehensive analysis than if only one was executed. - -#### PHP, Go, C, C++, .NET, C# +#### PHP, Go, C, C++, .NET, C#, Ruby, JavaScript The analyzer for these languages supports multiple lockfiles. -### Support for additional languages +#### Support for additional languages Support for additional languages, dependency managers, and dependency files are tracked in the following issues: | Package Managers | Languages | Supported files | Scan tools | Issue | | ------------------- | --------- | --------------- | ---------- | ----- | -| [Poetry](https://python-poetry.org/) | Python | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | - -For workarounds, see the [Troubleshooting section](#troubleshooting). +| [Poetry](https://python-poetry.org/) | Python | `pyproject.toml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | [GitLab#32774](https://gitlab.com/gitlab-org/gitlab/-/issues/32774) | ## Contribute your scanner @@ -506,7 +508,7 @@ always take the latest dependency scanning artifact available. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4908) in GitLab 14.1 [with a flag](../../../administration/feature_flags.md) named `sec_dependency_scanning_ui_enable`. Enabled by default. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/282533) in GitLab 14.1. -> - [Feature flag sec_dependency_scanning_ui_enable removed](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. +> - [Feature flag `sec_dependency_scanning_ui_enable` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. To enable Dependency Scanning in a project, you can create a merge request: @@ -577,9 +579,9 @@ The following variables allow configuration of global dependency scanning settin | ----------------------------|------------ | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | | `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). | -| `DS_DEFAULT_ANALYZERS` | ([**DEPRECATED - use `DS_EXCLUDED_ANALYZERS` instead**](https://gitlab.com/gitlab-org/gitlab/-/issues/287691)) Override the names of the official default images. For more information, see [Dependency Scanning Analyzers](analyzers.md). | +| `DS_DEFAULT_ANALYZERS` | This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in 15.0. Use `DS_EXCLUDED_ANALYZERS` instead. | | `DS_EXCLUDED_PATHS` | Exclude files and directories from the scan based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. | -| `DS_IMAGE_SUFFIX` | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10. | +| `DS_IMAGE_SUFFIX` | Suffix added to the image name. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10.) Automatically set to `"-fips"` when FIPS mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info`. | @@ -589,16 +591,13 @@ The following variables are used for configuring specific analyzers (used for a | CI/CD variable | Analyzer | Default | Description | |--------------------------------------| ------------------ | ---------------------------- |------------ | -| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Use if you're running dependency scanning in an offline, air-gapped environment.| -| `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | -| `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | | `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local Gemnasium database. | | `GEMNASIUM_DB_UPDATE_DISABLED` | `gemnasium` | `"false"` | Disable automatic updates for the `gemnasium-db` advisory database (For usage see: [examples](#hosting-a-copy-of-the-gemnasium_db-advisory-database))| | `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | | `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | | `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. | -| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. | +| `DS_JAVA_VERSION` | `gemnasium-maven` | `17` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | @@ -607,9 +606,6 @@ The following variables are used for configuring specific analyzers (used for a | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | -| `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | -| `RETIREJS_NODE_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/npmrepository.json` | Path or URL to `retire.js` node vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | -| `RETIREJS_ADVISORY_DB_INSECURE` | `retire.js` | `false` | Enable fetching remote JS and Node vulnerability data files (defined by the `RETIREJS_JS_ADVISORY_DB` and `RETIREJS_NODE_ADVISORY_DB` variables) from hosts using an insecure or self-signed SSL (TLS) certificate. | #### Other variables @@ -667,32 +663,10 @@ Read more on [how to use private Maven repositories](../index.md#using-private-m GitLab also offers [FIPS-enabled Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) versions of the Gemnasium images. You can therefore replace standard images with FIPS-enabled images. -To use FIPS-enabled images, set the `DS_IMAGE_SUFFIX` to `-fips`, -and set `DS_EXCLUDED_ANALYZERS` to `bundler-audit, retire.js` -to exclude the analyzers that don't support FIPS. +Gemnasium scanning jobs automatically use FIPS-enabled image when FIPS mode is enabled in the GitLab instance. +([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) -```yaml -variables: - DS_IMAGE_SUFFIX: "-fips" - DS_EXCLUDED_ANALYZERS: "bundler-audit, retire.js" -``` - -If you want to execute `bundler-audit` or `retire.js` in your project pipeline, you can override the -Gemnasium scanning jobs, and set `DS_IMAGE_SUFFIX` to `-fips` only for those jobs. - -```yaml -gemnasium-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" - -gemnasium-maven-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" - -gemnasium-python-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" -``` +To manually switch to FIPS-enabled images, set the variable `DS_IMAGE_SUFFIX` to `"-fips"`. ## Interacting with the vulnerabilities @@ -944,9 +918,6 @@ Here are the requirements for using dependency scanning in an offline environmen This advisory database is constantly being updated, so you must periodically sync your local copy with GitLab. -- _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). -- _Only if scanning npm/yarn projects_: Host an offline copy of the [`retire.js`](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [`js`](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. - Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) @@ -961,11 +932,9 @@ import the following default dependency scanning analyzer images from `registry. your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/gemnasium:2 -registry.gitlab.com/security-products/gemnasium-maven:2 -registry.gitlab.com/security-products/gemnasium-python:2 -registry.gitlab.com/security-products/retire.js:2 -registry.gitlab.com/security-products/bundler-audit:2 +registry.gitlab.com/security-products/gemnasium:3 +registry.gitlab.com/security-products/gemnasium-maven:3 +registry.gitlab.com/security-products/gemnasium-python:3 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -987,8 +956,6 @@ Support for custom certificate authorities was introduced in the following versi | `gemnasium` | [v2.8.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/releases/v2.8.0) | | `gemnasium-maven` | [v2.9.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/releases/v2.9.0) | | `gemnasium-python` | [v2.7.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/releases/v2.7.0) | -| `retire.js` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js/-/releases/v2.4.0) | -| `bundler-audit` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/releases/v2.4.0) | ### Set dependency scanning CI/CD job variables to use local dependency scanning analyzers @@ -1121,22 +1088,6 @@ intended to obtain a private package from a private index. This only affects use requires that the package does not already exist in the public index (and thus the attacker can put the package there with an arbitrary version number). -## Limitations - -### Referencing local dependencies using a path in JavaScript projects - -The [Retire.js](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) analyzer -doesn't support dependency references made with [local paths](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#local-paths) -in the `package.json` of JavaScript projects. The dependency scan outputs the following error for -such references: - -```plaintext -ERROR: Could not find dependencies: <dependency-name>. You may need to run npm install -``` - -As a workaround, add the [`retire.js`](analyzers.md) analyzer to -[`DS_EXCLUDED_ANALYZERS`](#configuring-dependency-scanning). - ## Troubleshooting ### Working around missing support for certain languages or package managers @@ -1156,9 +1107,8 @@ Generally, the approach is the following: 1. Add [`dependencies: [<your-converter-job>]`](../../../ci/yaml/index.md#dependencies) to your `dependency_scanning` job to make use of the converted definitions files. -For example, the unsupported `poetry.lock` file can be -[converted](https://python-poetry.org/docs/cli/#export) -to the supported `requirements.txt` as follows. +For example, Poetry projects that _only_ have a `pyproject.toml` +file can generate the `poetry.lock` file as follows. ```yaml include: @@ -1167,15 +1117,11 @@ include: stages: - test -variables: - PIP_REQUIREMENTS_FILE: "requirements-converted.txt" - gemnasium-python-dependency_scanning: - # Work around https://gitlab.com/gitlab-org/gitlab/-/issues/7006 + # Work around https://gitlab.com/gitlab-org/gitlab/-/issues/32774 before_script: - - pip install poetry # Or via another method: https://python-poetry.org/docs/#installation - - poetry export --output="$PIP_REQUIREMENTS_FILE" - - rm poetry.lock pyproject.toml + - pip install "poetry>=1,<2" # Or via another method: https://python-poetry.org/docs/#installation + - poetry update --lock # Generates the lock file to be analyzed. ``` ### `Error response from daemon: error processing tar file: docker-tar: relocation error` @@ -1197,11 +1143,6 @@ syntax. This directive is limited to 10000 checks and always returns `true` afte number. Because of this, and depending on the number of files in your repository, a dependency scanning job might be triggered even if the scanner doesn't support your project. -### Issues building projects with npm or yarn packages relying on Python 2 - -[Python 2 was removed](https://www.python.org/doc/sunset-python-2/) from the `retire.js` analyzer in GitLab 13.7 (analyzer version 2.10.1). Projects using packages -with a dependency on this version of Python should use `retire.js` version 2.10.0 or lower (for example, `registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2.10.0`). - ### Error: `dependency_scanning is used for configuration only, and its script should not be executed` For information on this, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). @@ -1260,7 +1201,7 @@ We recommend committing the lock files, which prevents this warning. If you have manually set `DS_MAJOR_VERSION` or `DS_ANALYZER_IMAGE` for specific reasons, and now must update your configuration to again get the latest patched versions of our -analyzers, edit your `gitlab-ci.yml` file and either: +analyzers, edit your `.gitlab-ci.yml` file and either: - Set your `DS_MAJOR_VERSION` to match the latest version as seen in [our current Dependency Scanning template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml#L18). @@ -1290,5 +1231,22 @@ To work around this error, downgrade the analyzer's version of `setuptools` (e.g gemnasium-python-dependency_scanning: before_script: - pip install setuptools==57.5.0 - image: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 +``` + +### Dependency Scanning of projects using psycopg2 fails with `pg_config executable not found` error + +Scanning a Python project that depends on `psycopg2` can fail with this message: + +```plaintext +Error: pg_config executable not found. +``` + +[psycopg2](https://pypi.org/project/psycopg2/) depends on the `libpq-dev` Debian package, +which is not installed in the `gemnasium-python` Docker image. To work around this error, +install the `libpq-dev` package in a `before_script`: + +```yaml +gemnasium-python-dependency_scanning: + before_script: + - apt-get update && apt-get install -y libpq-dev ``` diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 3a6aa8e3485..b5d39f3b32a 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -10,7 +10,7 @@ GitLab can check your application for security vulnerabilities including: - Unauthorized access. - Data leaks. -- Denial of service attacks. +- Denial of Service (DoS) attacks. Statistics and details on vulnerabilities are included in the merge request. Providing actionable information _before_ changes are merged enables you to be proactive. @@ -19,9 +19,6 @@ GitLab also provides high-level statistics of vulnerabilities across projects an - The [Security Dashboard](security_dashboard/index.md) provides a high-level view of vulnerabilities detected in your projects, pipeline, and groups. -- The [Threat Monitoring](threat_monitoring/index.md) page provides runtime security metrics - for application environments. With the information provided, - you can immediately begin risk analysis and remediation. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview of GitLab application security, see [Shifting Security Left](https://www.youtube.com/watch?v=XnYstHObqlA&t). @@ -51,8 +48,8 @@ The following vulnerability scanners and their databases are regularly updated: | Secure scanning tool | Vulnerabilities database updates | |:----------------------------------------------------------------|:---------------------------------| -| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database-update). | -| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | +| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | +| [Dependency Scanning](dependency_scanning/index.md) | Relies on the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). It is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | | [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | | [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | @@ -145,7 +142,7 @@ Jobs pass if they are able to complete a scan. A _pass_ result does NOT indicate Jobs fail if they are unable to complete a scan. You can view the pipeline logs for more information. -All jobs are permitted to fail by default. This means that if they fail it do not fail the pipeline. +All jobs are permitted to fail by default. This means that if they fail, it does not fail the pipeline. If you want to prevent vulnerabilities from being merged, you should do this by adding [Security Approvals in Merge Requests](#security-approvals-in-merge-requests) which prevents unknown, high or critical findings from being merged without an approval from a specific group of people that you choose. @@ -174,7 +171,7 @@ reports are available to download. To download a report, select A merge request contains a security widget which displays a summary of the NEW results. New results are determined by comparing the current findings against existing findings in the target (default) branch (if there are prior findings). -We recommended you run a scan of the `default` branch before enabling feature branch scans for your developers. Otherwise, there is no base for comparison and all feature branches display the full scan results in the merge request security widget. +We recommend you run a scan of the `default` branch before enabling feature branch scans for your developers. Otherwise, there is no base for comparison and all feature branches display the full scan results in the merge request security widget. The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both NEW and EXISTING findings. @@ -204,56 +201,24 @@ By default, the vulnerability report does not show vulnerabilities of `dismissed ## Security approvals in merge requests -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9928) in GitLab 12.2. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9928) in GitLab 12.2. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357300) the Vulnerability-Check feature in GitLab 15.0. You can enforce an additional approval for merge requests that would introduce one of the following security issues: -- A security vulnerability. For more details, read - [Vulnerability-Check rule](#vulnerability-check-rule). +- A security vulnerability. For more details, read [Scan result policies](policies/scan-result-policies.md). - A software license compliance violation. For more details, read [Enabling license approvals within a project](../compliance/license_compliance/index.md#enabling-license-approvals-within-a-project). -### Vulnerability-Check rule +### Migration of existing Vulnerability-Check rules -WARNING: -This feature is in its end-of-life process. It is [deprecated](../../update/deprecations.md#vulnerability-check) -in GitLab 14.8, and is planned for removal in GitLab 15.0. Users should migrate to the new -[Security Approval Policies](policies/scan-result-policies.md). +If your projects have rules that have a security orchestration project, a new MR with +the existing rule's content is created automatically against the default branch belonging +to the security orchestration project. To maintain the same security approval rules you +had before GitLab 15.0, we recommend merging this new MR. -To prevent a merge request introducing a security vulnerability in a project, enable the -Vulnerability-Check rule. While this rule is enabled, additional merge request approval by -[eligible approvers](../project/merge_requests/approvals/rules.md#eligible-approvers) -is required when the latest security report in a merge request: - -- Contains vulnerabilities with states (for example, `previously detected`, `dismissed`) matching the rule's vulnerability states. Only `newly detected` are considered if the target branch differs from the project default branch. -- Contains vulnerabilities with severity levels (for example, `high`, `critical`, or `unknown`) - matching the rule's severity levels. -- Contains a vulnerability count higher than the rule allows. -- Is not yet generated (until pipeline completion). - -An approval is optional when the security report: - -- Contains only vulnerabilities with states (for example, `newly detected`, `resolved`) **NOT** matching the rule's vulnerability states. -- Contains only vulnerabilities with severity levels (for example, `low`, `medium`) **NOT** matching - the rule's severity levels. -- Contains a vulnerability count equal to or less than what the rule allows. - -Project members with at least the Maintainer role can enable or edit -the Vulnerability-Check rule. - -#### Enable the Vulnerability-Check rule - -To enable or edit the Vulnerability-Check rule: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge request approvals**. -1. Select **Activate** or **Edit** of the Vulnerability-Check. -1. Complete the fields. **Approvals required** must be at least 1. -1. Select **Add approval rule**. - -The approval rule is enabled for all merge requests. Any code changes reset the approvals required. +If your projects have rules without a security orchestration project, a new security orchestration project is created automatically with the content of the existing rule. No additional action is required. ## Using private Maven repositories @@ -443,7 +408,7 @@ You can interact with the results of the security scanning tools in several loca For more details about which findings or vulnerabilities you can view in each of those locations, select the respective link. Each page details the ways in which you can interact with the findings -and vulnerabilities. As an example, in most cases findings start out as _detected_ status. +and vulnerabilities. As an example, in most cases findings start out as a _detected_ status. You have the option to: @@ -507,7 +472,7 @@ Additional details about the differences between the two solutions are outlined | ------ | ------ | ------ | | **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, Secret Detection, and Container Scanning scans are supported. | | **Usability** | Requires knowledge of CI YAML. | Follows a `rules` and `actions`-based YAML structure. | -| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `gitlab-ci.yml` file. To include the project's `gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. | +| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `.gitlab-ci.yml` file. To include the project's `.gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. | | **Schedulable** | Can be scheduled through a scheduled pipeline on the group. | Can be scheduled natively through the policy configuration itself. | | **Separation of Duties** | Only group owners can create compliance framework labels. Only project owners can apply compliance framework labels to projects. The ability to make or approve changes to the compliance pipeline definition is limited to individuals who are explicitly given access to the project that contains the compliance pipeline. | Only project owners can define a linked security policy project. The ability to make or approve changes to security policies is limited to individuals who are explicitly given access to the security policy project. | | **Ability to apply one standard to multiple projects** | The same compliance framework label can be applied to multiple projects inside a group. | The same security policy project can be used for multiple projects across GitLab with no requirement of being located in the same group. | @@ -657,7 +622,7 @@ involve pinning to the previous template versions, for example: ``` Additionally, we provide a dedicated project containing the versioned legacy templates. -This can be used for offline setups or anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). +This can be used for offline setups or for anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). @@ -694,3 +659,6 @@ These security pages can be populated by running the jobs from the manual step o There is [an issue open to handle this scenario](https://gitlab.com/gitlab-org/gitlab/-/issues/346843). Please upvote the issue to help with prioritization, and [contributions are welcomed](https://about.gitlab.com/community/contribute/). + doc/user/project/merge_requests/approvals/settings.md ++ +0 diff --git a/doc/user/application_security/policies/img/policies_list_v14_3.png b/doc/user/application_security/policies/img/policies_list_v14_3.png Binary files differdeleted file mode 100644 index 7a24860d4a7..00000000000 --- a/doc/user/application_security/policies/img/policies_list_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/policies_list_v15_0.png b/doc/user/application_security/policies/img/policies_list_v15_0.png Binary files differnew file mode 100644 index 00000000000..4089c311fe4 --- /dev/null +++ b/doc/user/application_security/policies/img/policies_list_v15_0.png diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 81d24104340..f790164d9a0 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -19,7 +19,6 @@ GitLab supports the following security policies: - [Scan Execution Policy](scan-execution-policies.md) - [Scan Result Policy](scan-result-policies.md) -- [Container Network Policy](#container-network-policy) (DEPRECATED) ## Security policy project @@ -42,8 +41,9 @@ stored there. Examples and schema information are available for the following po - [Scan execution policy](scan-execution-policies.md#example-security-policies-project) - [Scan result policy](scan-result-policies.md#example-security-scan-result-policies-project) -Policies created in this project are applied through a background job that runs once every 10 -minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect. ### Managing the linked security policy project @@ -81,22 +81,7 @@ status), and create and edit deployed policies: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Policies**. - - -Network policies are fetched directly from the selected environment's -deployment platform while other policies are fetched from the project's -security policy project. Changes performed outside of this tab are -reflected upon refresh. - -By default, the policy list contains predefined network policies in a -disabled state. Once enabled, a predefined policy deploys to the -selected environment's deployment platform and you can manage it like -the regular policies. - -Note that if you're using [Auto DevOps](../../../topics/autodevops/index.md) -and change a policy in this section, your `auto-deploy-values.yaml` file doesn't update. Auto DevOps -users must make changes by following the -[Container Network Policy documentation](../../../topics/autodevops/stages.md#network-policy). + ## Policy editor @@ -144,111 +129,6 @@ See [Scan execution policies](scan-execution-policies.md). See [Scan result policies](scan-result-policies.md). -## Container Network Policy - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32365) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Policy is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The **Container Network Policy** section provides packet flow metrics for -your application's Kubernetes namespace. This section has the following -prerequisites: - -- Your project contains at least one [environment](../../../ci/environments/index.md). -- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). -- You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration). - -If you're using custom Helm values for Cilium, you must enable Hubble -with flow metrics for each namespace by adding the following lines to -your [Cilium values](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium): - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -The **Container Network Policy** section displays the following information -about your packet flow: - -- The total amount of the inbound and outbound packets -- The proportion of packets dropped according to the configured - policies -- The per-second average rate of the forwarded and dropped packets - accumulated over time window for the requested time interval - -If a significant percentage of packets is dropped, you should -investigate it for potential threats by -examining the Cilium logs: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -### Change the status - -To change a network policy's status: - -- Select the network policy you want to update. -- Select **Edit policy**. -- Select the **Policy status** toggle to update the selected policy. -- Select **Save changes** to deploy network policy changes. - -Disabled network policies have the `network-policy.gitlab.com/disabled_by: gitlab` selector inside -the `podSelector` block. This narrows the scope of such a policy and as a result it doesn't affect -any pods. The policy itself is still deployed to the corresponding deployment namespace. - -### Container Network Policy editor - -The policy editor only supports the [CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/policy/) -specification. Regular Kubernetes [NetworkPolicy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#networkpolicy-v1-networking-k8s-io) -resources aren't supported. - -Rule mode supports the following rule types: - -- [Labels](https://docs.cilium.io/en/v1.8/policy/language/#labels-based). -- [Entities](https://docs.cilium.io/en/v1.8/policy/language/#entities-based). -- [IP/CIDR](https://docs.cilium.io/en/v1.8/policy/language/#ip-cidr-based). Only - the `toCIDR` block without `except` is supported. -- [DNS](https://docs.cilium.io/en/v1.8/policy/language/#dns-based). -- [Level 4](https://docs.cilium.io/en/v1.8/policy/language/#layer-4-examples) - can be added to all other rules. - -Once your policy is complete, save it by selecting **Save policy** -at the bottom of the editor. Existing policies can also be -removed from the editor interface by selecting **Delete policy** -at the bottom of the editor. - -### Configure a Network Policy Alert - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3438) and [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/287676) in GitLab 13.9. -> - The feature flag was removed and the Threat Monitoring Alerts Project was [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/287676) in GitLab 14.0. - -You can use policy alerts to track your policy's impact. Alerts are only available if you've -[installed](../../clusters/agent/repository.md) -and [configured](../../clusters/agent/install/index.md#register-the-agent-with-gitlab) -an agent for this project. - -There are two ways to create policy alerts: - -- In the [policy editor UI](#container-network-policy-editor), - by clicking **Add alert**. -- In the policy editor's YAML mode, through the `metadata.annotations` property: - - ```yaml - metadata: - annotations: - app.gitlab.com/alert: 'true' - ``` - -Once added, the UI updates and displays a warning about the dangers of too many alerts. - ## Roadmap See the [Category Direction page](https://about.gitlab.com/direction/protect/security_orchestration/) diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 7e8e60768b9..aa23ad30a73 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -35,8 +35,9 @@ policy project is automatically created. Existing policies can also be removed from the editor interface by selecting **Delete policy** at the bottom of the editor. -All scan execution policy changes are applied through a background job that runs once every 10 -minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect.  @@ -84,9 +85,31 @@ This rule enforces the defined actions and schedules a scan on the provided date | `type` | `string` | `schedule` | The rule's type. | | `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | | `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | -| `clusters` | `object` | | The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | +| `agents` | `object` | | The name of the [GitLab agents](../../clusters/agent/index.md) where [cluster image scanning](../../clusters/agent/vulnerabilities.md) will run. The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | <!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> +| `clusters` (removed) | `object` | | This field was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. Use the `agents` field instead. The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | +<!--- end_remove --> -### `cluster` schema +GitLab supports the following types of CRON syntax for the `cadence` field: + +- A daily cadence of once per hour at a specified hour, for example: `0 18 * * *` +- A weekly cadence of once per week on a specified day and at a specified hour, for example: `0 13 * * 0` + +It is possible that other elements of the CRON syntax will work in the cadence field, however, GitLab does not officially test or support them. + +### `agent` schema + +Use this schema to define `agents` objects in the [`schedule` rule type](#schedule-rule-type). + +| Field | Type | Possible values | Description | +|--------------|---------------------|--------------------------|-------------| +| `namespaces` | `array` of `string` | | The namespace that is scanned. If empty, all namespaces will be scanned. | + +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> + +### `cluster` schema (removed) + +This schema was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. +Use the [`agent` schema](#agent-schema) instead. Use this schema to define `clusters` objects in the [`schedule` rule type](#schedule-rule-type). @@ -97,6 +120,8 @@ Use this schema to define `clusters` objects in the [`schedule` rule type](#sche | `namespaces` | `array` of `string` | | The namespace that is scanned (only the first value is currently supported). | | `kinds` | `array` of `string` | `deployment`/`daemonset` | The resource kind that should be scanned (only the first value is currently supported). | +<!--- end_remove --> + ## `scan` action type This action executes the selected `scan` with additional parameters when conditions for at least one diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index d2cce207bfd..232a5c9f91c 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -9,7 +9,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can use scan result policies to take action based on scan results. For example, one type of scan result policy is a security approval policy that allows approval to be required based on the findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning -job is fully executed. +job is fully executed. The following video gives you an overview of GitLab scan result policies: + +<div class="video-fallback"> + See the video: <a href="https://youtu.be/w5I9gcUgr9U">Overview of GitLab Scan Result Policies</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/w5I9gcUgr9U" frameborder="0" allowfullscreen="true"> </iframe> +</figure> ## Scan result policy editor @@ -25,8 +32,9 @@ If a security policy project doesn't link to your project, GitLab creates such a Existing policies can also be removed from the editor interface by selecting **Delete policy** at the bottom of the editor. -All scan result policy changes are applied through a background job that runs once every 10 minutes. -Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect. The [policy editor](index.md#policy-editor) supports YAML mode and rule mode. @@ -61,7 +69,7 @@ This rule enforces the defined actions based on the information provided. | Field | Type | Possible values | Description | |------------|------|-----------------|-------------| | `type` | `string` | `scan_finding` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `branches` | `array` of `string` | `[]` or the branch's name | Protected branches for this rule to consider. | | `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. | | `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | | `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical`| The severity levels for this rule to consider. | diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 7529bf90ccf..661f564828a 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -4,20 +4,25 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SAST Analyzers **(FREE)** +# SAST analyzers **(FREE)** > [Moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) from GitLab Ultimate to GitLab Free in 13.3. -SAST relies on underlying third party tools that are wrapped into what we call -"Analyzers". An analyzer is a -[dedicated project](https://gitlab.com/gitlab-org/security-products/analyzers) -that wraps a particular tool to: +Static Application Security Testing (SAST) uses analyzers +to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/#scanner), a third-party code analysis tool. -- Expose its detection logic. -- Handle its execution. -- Convert its output to the common format. +The analyzers are published as Docker images that SAST uses to launch dedicated containers for each +analysis. -This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/security-products/analyzers/common). +SAST default images are maintained by GitLab, but you can also integrate your own custom image. + +For each scanner, an analyzer: + +- Exposes its detection logic. +- Handles its execution. +- Converts its output to a [standard format](../terminology/#secure-report-format). + +## SAST analyzers SAST supports the following official analyzers: @@ -36,12 +41,6 @@ SAST supports the following official analyzers: - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) -The analyzers are published as Docker images that SAST uses to launch -dedicated containers for each analysis. - -SAST is pre-configured with a set of **default images** that are maintained by -GitLab, but users can also integrate their own **custom images**. - ## SAST analyzer features For an analyzer to be considered Generally Available, it is expected to minimally @@ -55,34 +54,140 @@ support the following features: - [Emits JSON report format](index.md#reports-json-format) - [SELinux support](index.md#running-sast-in-selinux) -## Official default analyzers +## Post analyzers + +Post analyzers enrich the report output by an analyzer. A post analyzer doesn't modify report +content directly. Instead, it enhances the results with additional properties, including: + +- CWEs. +- Location tracking fields. +- A means of identifying false positives or insignificant findings. **(ULTIMATE)** + +## Data provided by analyzers + +Each analyzer provides data about the vulnerabilities it detects. The following table details the +data available from each analyzer. The values provided by these tools are heterogeneous so they are sometimes +normalized into common values, for example, `severity` and `confidence`. + +| Property / tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Semgrep | Sobelow | +|--------------------------------|------|--------|----------|-----------------|----------|------------|-------|-----------------|-------|------------|-----------------------|---------------------------|---------|---------| +| Affected item (for example, class or package) | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| Confidence | ✗ | ✓ | ✓ | ✗ | ✓ | x | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✓ | +| Description | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | +| End column | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| End line | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| External ID (for example, CVE) | ✗ | ✗ | ⚠ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Internal doc/explanation | ✓ | ⚠ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | +| Severity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ⚠ | ✗ | +| Solution | ✓ | ✗ | ✗ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | +| Source code extract | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| Start column | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| URLs | ✓ | ✗ | ✓ | ✗ | ⚠ | ✗ | ⚠ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | + +- ✓ => Data is available. +- ⚠ => Data is available, but it's partially reliable, or it has to be extracted from unstructured content. +- ✗ => Data is not available or it would require specific, inefficient or unreliable, logic to obtain it. + +## Transition to Semgrep-based scanning + +SAST includes a [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) that covers [multiple languages](index.md#supported-languages-and-frameworks). +GitLab maintains the analyzer and writes detection rules for it. + +If you use the [GitLab-managed CI/CD template](index.md#configuration), the Semgrep-based analyzer operates alongside other language-specific analyzers. +It runs with GitLab-managed detection rules that mimic the other analyzers' detection rules. +Work to remove language-specific analyzers and replace them with the Semgrep-based analyzer is tracked in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/5245). + +You can choose to disable the other analyzers early and use Semgrep-based scanning for supported languages before the default behavior changes. If you do so: + +- You'll enjoy significantly faster scanning, reduced CI minutes usage, and more customizable scanning rules. +- However, vulnerabilities previously reported by language-specific analyzers will be reported again under certain conditions, including if you've dismissed the vulnerabilities before. The system behavior depends on: + - whether you've excluded the Semgrep-based analyzer from running in the past. + - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/). + +### Vulnerability translation + +When you switch analyzers for a language, vulnerabilities may not match up. + +The Vulnerability Management system automatically moves vulnerabilities from the old analyzer to Semgrep for certain languages: + +- For C, a vulnerability is moved if it has only ever been detected by Flawfinder in pipelines where Semgrep also detected it. Semgrep coverage for C was introduced by default into the CI/CD template in GitLab 14.4 (October 2021). +- For Go, a vulnerability is moved if it has only ever been detected by Gosec in pipelines where Semgrep also detected it. Semgrep coverage for Go was introduced by default into the CI/CD template in GitLab 14.2 (August 2021). +- For JavaScript and TypeScript, a vulnerability is moved if it has only ever been detected by ESLint in pipelines where Semgrep also detected it. Semgrep coverage for these languages was introduced into the CI/CD template in GitLab 13.12 (May 2021). + +However, you'll see old vulnerabilities re-created based on Semgrep results if: + +- A vulnerability was created by Bandit or SpotBugs and you disable those analyzers. We only recommend disabling Bandit and SpotBugs now if the analyzers aren’t working. Work to automatically translate Bandit and SpotBugs vulnerabilities to Semgrep is tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/328062). +- A vulnerability was created by ESLint, Gosec, or Flawfinder in a default-branch pipeline where Semgrep scanning did not run successfully (before Semgrep coverage was introduced for the language, because you disabled Semgrep explicitly, or because the Semgrep scan failed in that pipeline). We do not currently plan to combine these vulnerabilities if they already exist. -Any custom change to the official analyzers can be achieved by using a -[CI/CD variable in your `.gitlab-ci.yml`](index.md#available-cicd-variables). +When a vulnerability is re-created, the original vulnerability is marked as “no longer detected” in the Vulnerability Report. +A new vulnerability is then created based on the Semgrep finding. -### Using a custom Docker mirror +### Activating Semgrep-based scanning early -You can switch to a custom Docker registry that provides the official analyzer -images under a different prefix. For instance, the following instructs -SAST to pull `my-docker-registry/gl-images/sast/bandit` -instead of `registry.gitlab.com/security-products/sast/bandit`. -In `.gitlab-ci.yml` define: +You can choose to use Semgrep-based scanning instead of language-specific analyzers before the default behavior changes. + +We recommend taking this approach if any of these cases applies: + +- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/). +- You're having trouble configuring one of the analyzers whose coverage overlaps with Semgrep-based coverage. For example, you might have trouble setting up the SpotBugs-based analyzer to compile your code. +- You've already seen and dismissed vulnerabilities created by ESLint, Gosec, or Flawfinder scanning, and you've kept the re-created vulnerabilities created by Semgrep. + +You can make a separate choice for each of the language-specific analyzers, or you can disable them all. + +#### Activate Semgrep-based scanning + +To switch to Semgrep-based scanning early, you can: + +1. Create a merge request (MR) to set the [`SAST_EXCLUDED_ANALYZERS` CI/CD variable](#disable-specific-default-analyzers) to `"bandit,gosec,eslint"`. + - If you also want to disable SpotBugs scanning, add `spotbugs` to the list. We only recommend this for Java projects. SpotBugs is the only current analyzer that can scan Groovy, Kotlin, and Scala. + - If you also want to disable Flawfinder scanning, add `flawfinder` to the list. We only recommend this for C projects. Flawfinder is the only current analyzer that can scan C++. +1. Verify that scanning jobs succeed in the MR. You'll notice findings from the removed analyzers in _Fixed_ and findings from Semgrep in _New_. (Some findings may show different names, descriptions, and severities, since GitLab manages and edits the Semgrep rulesets.) +1. Merge the MR and wait for the default-branch pipeline to run. +1. Use the Vulnerability Report to dismiss the findings that are no longer detected by the language-specific analyzers. + +## Customize analyzers + +Use [CI/CD variables](index.md#available-cicd-variables) +in your `.gitlab-ci.yml` file to customize the behavior of your analyzers. + +### Use a custom Docker mirror + +You can use a custom Docker registry, instead of the GitLab registry, to host the analyzers' images. + +Prerequisites: + +- The custom Docker registry must provide images for all the official analyzers. + +NOTE: +This variable affects all Secure analyzers, not just the analyzers for SAST. + +To have GitLab download the analyzers' images from a custom Docker registry, define the prefix with +the `SECURE_ANALYZERS_PREFIX` CI/CD variable. + +For example, the following instructs SAST to pull `my-docker-registry/gitlab-images/bandit` instead +of `registry.gitlab.com/security-products/bandit`: ```yaml include: - template: Security/SAST.gitlab-ci.yml variables: - SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gitlab-images ``` -This configuration requires that your custom registry provides images for all -the official analyzers. +### Disable all default analyzers -### Disabling all default analyzers +You can disable all default SAST analyzers, leaving only [custom analyzers](#custom-analyzers) +enabled. -Setting `SAST_DISABLED` to `true` disables all the official -default analyzers. In `.gitlab-ci.yml` define: +To disable all default analyzers, set the CI/CD variable `SAST_DISABLED` to `true` in your +`.gitlab-ci.yml` file. + +Example: ```yaml include: @@ -92,13 +197,15 @@ variables: SAST_DISABLED: true ``` -That's needed when one totally relies on [custom analyzers](#custom-analyzers). +### Disable specific default analyzers + +Analyzers are run automatically according to the +source code languages detected. However, you can disable select analyzers. -### Disabling specific default analyzers +To disable select analyzers, set the CI/CD variable `SAST_EXCLUDED_ANALYZERS` to a comma-delimited +string listing the analyzers that you want to prevent running. -Set `SAST_EXCLUDED_ANALYZERS` to a comma-delimited string that includes the official -default analyzers that you want to avoid running. In `.gitlab-ci.yml` define the -following to prevent the `eslint` analyzer from running: +For example, to disable the `eslint` analyzer: ```yaml include: @@ -108,27 +215,21 @@ variables: SAST_EXCLUDED_ANALYZERS: "eslint" ``` -## Post Analyzers **(ULTIMATE)** +### Custom analyzers -While analyzers are thin wrappers for executing scanners, post analyzers work to -enrich the data generated within our reports. +You can provide your own analyzers by defining jobs in your CI/CD configuration. For +consistency with the default analyzers, you should add the suffix `-sast` to your custom +SAST jobs. -GitLab SAST post analyzers never modify report contents directly but work by -augmenting results with additional properties (such as CWEs), location tracking fields, -and a means of identifying false positives or insignificant findings. +For more details on integrating a custom security scanner into GitLab, see [Security Scanner Integration](../../../development/integrations/secure.md). -The implementation of post analyzers is determined by feature availability tiers, where -simple data enrichment may occur within our free tier and most advanced processing is split -into separate binaries or pipeline jobs. +#### Example custom analyzer -## Custom Analyzers +This example shows how to add a scanning job that's based on the Docker image +`my-docker-registry/analyzers/csharp`. It runs the script `/analyzer run` and outputs a SAST report +`gl-sast-report.json`. -You can provide your own analyzers by -defining CI jobs in your CI configuration. For consistency, you should suffix your custom -SAST jobs with `-sast`. Here's how to add a scanning job that's based on the -Docker image `my-docker-registry/analyzers/csharp` and generates a SAST report -`gl-sast-report.json` when `/analyzer run` is executed. Define the following in -`.gitlab-ci.yml`: +Define the following in your `.gitlab-ci.yml` file: ```yaml csharp-sast: @@ -140,33 +241,3 @@ csharp-sast: reports: sast: gl-sast-report.json ``` - -The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate custom security scanners into GitLab. - -## Analyzers Data - -| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Semgrep | Sobelow | -|--------------------------------|------|--------|----------|-----------------|----------|------------|-------|-----------------|-------|------------|-----------------------|---------------------------|---------|---------| -| Affected item (for example, class or package) | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| Confidence | ✗ | ✓ | ✓ | ✗ | ✓ | x | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✓ | -| Description | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | -| End column | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| End line | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| External ID (for example, CVE) | ✗ | ✗ | ⚠ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Internal doc/explanation | ✓ | ⚠ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | -| Severity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ⚠ | ✗ | -| Solution | ✓ | ✗ | ✗ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | -| Source code extract | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| Start column | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| URLs | ✓ | ✗ | ✓ | ✗ | ⚠ | ✗ | ⚠ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | - -- ✓ => we have that data -- ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content -- ✗ => we don't have that data or it would need to develop specific or inefficient/unreliable logic to obtain it. - -The values provided by these tools are heterogeneous so they are sometimes -normalized into common values (for example, `severity`, `confidence`, and so on). diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 8f006f258b6..38f26b7578d 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -13,12 +13,17 @@ The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab. explains how 4 of the top 6 attacks were application based. Download it to learn how to protect your organization. -If you're using [GitLab CI/CD](../../../ci/index.md), you can use Static Application Security -Testing (SAST) to check your source code for known vulnerabilities. -If the pipeline is associated with a merge request, the SAST analysis is compared with the results of -the target branch's analysis (if available). The results of that comparison are shown in the merge -request. If the pipeline is running from the default branch, the results of the SAST -analysis are available in the [security dashboards](../security_dashboard/index.md). +If you’re using [GitLab CI/CD](../../../ci/index.md), you can use Static Application Security +Testing (SAST) to check your source code for known vulnerabilities. You can run SAST analyzers in +any GitLab tier. The analyzers output JSON-formatted reports as job artifacts. + +With GitLab Ultimate, SAST results are also processed so you can: + +- See them in merge requests. +- Use them in approval workflows. +- Review them in the security dashboard. + +For more details, see the [Summary of features per tier](#summary-of-features-per-tier).  @@ -543,7 +548,7 @@ Several passthrouh types generate a configuration for the target analyzer: - Two `git` passthrough sections pull the head of branch `refs/remotes/origin/test` from the `myrules` Git repository, and revision - `97f7686` from the `sast-rules` Git repostory. From the `sast-rules` Git + `97f7686` from the `sast-rules` Git repository. From the `sast-rules` Git repository, only data from the `go` subdirectory is considered. - The `sast-rules` entry has a higher precedence because it appears later in the configuration. @@ -887,7 +892,7 @@ Some analyzers can be customized with CI/CD variables. | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | | `JAVA_OPTS` | SpotBugs | Additional arguments for the `java` executable. | | `JAVA_PATH` | SpotBugs | Path to the `java` executable. | -| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | +| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. [Starting in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/352549), supported versions are `11` and `17` (default). Before GitLab 15.0, supported versions are `8` (default) and `11`. | | `MAVEN_CLI_OPTS` | SpotBugs | Additional arguments for the `mvn` or `mvnw` executable. | | `MAVEN_PATH` | SpotBugs | Path to the `mvn` executable. | | `MAVEN_REPO_PATH` | SpotBugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | @@ -977,19 +982,19 @@ import the following default SAST analyzer images from `registry.gitlab.com` int [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/sast/bandit:2 -registry.gitlab.com/security-products/sast/brakeman:2 -registry.gitlab.com/security-products/sast/eslint:2 -registry.gitlab.com/security-products/sast/flawfinder:2 -registry.gitlab.com/security-products/sast/gosec:3 -registry.gitlab.com/security-products/sast/kubesec:2 -registry.gitlab.com/security-products/sast/nodejs-scan:2 -registry.gitlab.com/security-products/sast/phpcs-security-audit:2 -registry.gitlab.com/security-products/sast/pmd-apex:2 -registry.gitlab.com/security-products/sast/security-code-scan:2 -registry.gitlab.com/security-products/sast/semgrep:2 -registry.gitlab.com/security-products/sast/sobelow:2 -registry.gitlab.com/security-products/sast/spotbugs:2 +registry.gitlab.com/security-products/bandit:2 +registry.gitlab.com/security-products/brakeman:2 +registry.gitlab.com/security-products/eslint:2 +registry.gitlab.com/security-products/flawfinder:2 +registry.gitlab.com/security-products/gosec:3 +registry.gitlab.com/security-products/kubesec:2 +registry.gitlab.com/security-products/nodejs-scan:2 +registry.gitlab.com/security-products/phpcs-security-audit:2 +registry.gitlab.com/security-products/pmd-apex:2 +registry.gitlab.com/security-products/security-code-scan:2 +registry.gitlab.com/security-products/semgrep:2 +registry.gitlab.com/security-products/sobelow:2 +registry.gitlab.com/security-products/spotbugs:2 ``` The process for importing Docker images into a local offline Docker registry depends on diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 0a18e7d5f45..3937cbd77b6 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -195,13 +195,18 @@ Secret Detection can be customized by defining available CI/CD variables: | CI/CD variable | Default value | Description | |-----------------------------------|---------------|-------------| -| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | -| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | -| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. | | `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | | `SECRET_DETECTION_IMAGE_SUFFIX` | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355519) in GitLab 14.10. | +In previous GitLab versions, the following variables were also available: + +| CI/CD variable | Default value | Description | +|-----------------------------------|---------------|-------------| +| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | +| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | +| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352565) in GitLab 15.0. | + ### Custom rulesets **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md index 643da47d876..9771658da4e 100644 --- a/doc/user/application_security/secret_detection/post_processing.md +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -46,7 +46,7 @@ sequenceDiagram Cloud Vendor-->>+RevocationAPI: ACCEPTED ``` -## Integrate your cloud provider service with GitLab Saas +## Integrate your cloud provider service with GitLab SaaS Third party cloud and SaaS providers can [express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 488ec336646..577606885ca 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -17,6 +17,7 @@ To use the Security Dashboards, you must: - Configure jobs to use the [`reports` syntax](../../../ci/yaml/index.md#artifactsreports). - Use [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or later. If you use the shared runners on GitLab.com, you are using the correct version. +- Have the [correct role](../../permissions.md) for the project or group. ## When Security Dashboards are updated diff --git a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png b/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png Binary files differdeleted file mode 100644 index a11a7fafc4a..00000000000 --- a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md deleted file mode 100644 index 9b8dd2825ea..00000000000 --- a/doc/user/application_security/threat_monitoring/index.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -type: reference, howto -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Threat Monitoring **(ULTIMATE)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14707) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Threat Monitoring is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The **Threat Monitoring** page provides alerts and metrics -for the GitLab application runtime security features. You can access -these by navigating to your project's **Security & Compliance > Threat -Monitoring** page. - -GitLab supports statistics for the following security features: - -- [Container Network Policies](../../../topics/autodevops/stages.md#network-policy) - -## Container Network Policy Alert list - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3438) in GitLab 13.9. - -The policy alert list displays your policy's alert activity. You can sort the list by these columns: - -- Date and time -- Events -- Status - -You can filter the list with the **Policy Name** filter and the **Status** filter at the top. Use -the selector menu in the **Status** column to set the status for each alert: - -- Unreviewed -- In review -- Resolved -- Dismissed - -By default, the list doesn't display resolved or dismissed alerts. - - - -Clicking an alert's row opens the alert drawer, which shows more information about the alert. A user -can also create an incident from the alert and update the alert status in the alert drawer. - -Clicking an alert's name takes the user to the [alert details page](../../../operations/incident_management/alerts.md#alert-details-page). diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 18b99e06299..87344e4ff65 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -42,7 +42,7 @@ A vulnerability's status can be one of the following: | Dismissed | A user has seen this vulnerability and dismissed it because it is not accurate or otherwise not to be resolved. | | Resolved | The vulnerability has been fixed or is no longer present. | -Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an existing vulnerability is no longer detected in a project's `default` branch, you should change its status to Resolved. This ensures that if it is accidentally reintroduced in a future merge, it will be visible again as a new record. You can use the [Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are no longer detected, and [change their status](../vulnerability_report#change-status-of-multiple-vulnerabilities). +Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an existing vulnerability is no longer detected in a project's `default` branch, you should change its status to Resolved. This ensures that if it is accidentally reintroduced in a future merge, it will be visible again as a new record. You can use the [Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are no longer detected, and [change their status](../vulnerability_report#change-status-of-vulnerabilities). ## Change vulnerability status diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index 89464064ea3..967a6d9fa89 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -56,8 +56,6 @@ the following tables: | GitLab analyzer | Outputs severity levels? | Native severity level type | Native severity level example | |------------------------------------------------------------------------------------------|------------------------------|----------------------------|-------------------------------------| -| [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) | **{check-circle}** Yes | String | `low`, `medium`, `high`, `critical` | -| [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) | **{check-circle}** Yes | String | `low`, `medium`, `high`, `critical` | | [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | **{check-circle}** Yes | CVSS v2.0 Rating and CVSS v3.1 Qualitative Severity Rating | `(AV:N/AC:L/Au:S/C:P/I:P/A:N)`, `CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H` | ## Container Scanning diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index a9cef15e3e8..e499ddbbd6b 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -11,7 +11,7 @@ The Vulnerability Report provides information about vulnerabilities from scans o The scan results from a pipeline are only ingested after all the jobs in the pipeline complete. Partial results for a pipeline with jobs in progress can be seen in the pipeline security tab. -The report is available for projects, groups, and the Security Center. +The report is available for users with the [correct role](../../permissions.md) on projects, groups, and the Security Center. At all levels, the Vulnerability Report contains: @@ -34,13 +34,18 @@ in that row: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6165) in GitLab 11.1. -The project-level Vulnerability Report also contains: +At the project level, the Vulnerability Report also contains: - A time stamp showing when it was updated, including a link to the latest pipeline. - The number of failures that occurred in the most recent pipeline. Select the failure notification to view the **Failed jobs** tab of the pipeline's page. -To access the report, navigate to **Security & Compliance > Vulnerability Report**. +### View the project-level vulnerability report + +To view the project-level vulnerability report: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Vulnerability report**. ## Vulnerability Report actions @@ -56,17 +61,22 @@ From the Vulnerability Report you can: ## Vulnerability Report filters -You can filter the vulnerabilities table by: +You can filter the Vulnerability Report to narrow focus on only vulnerabilities matching specific +criteria. + +The available filters are: <!-- vale gitlab.SubstitutionWarning = NO --> -| Filter | Available options | -|:---------|:------------------| -| Status | Detected, Confirmed, Dismissed, Resolved. | -| Severity | Critical, High, Medium, Low, Info, Unknown. | -| Tool | For more details, see [Tool filter](#tool-filter). | -| Project | For more details, see [Project filter](#project-filter). | -| Activity | For more details, see [Activity filter](#activity-filter). | +- **Status**: Detected, Confirmed, Dismissed, Resolved. +- **Severity**: Critical, High, Medium, Low, Info, Unknown. +- **Tool**: For more details, see [Tool filter](#tool-filter). +- **Project**: For more details, see [Project filter](#project-filter). +- **Activity**: For more details, see [Activity filter](#activity-filter). + +The filters' criteria are combined to show only vulnerabilities matching all criteria. +An exception to this behavior is the Activity filter. For more details about how it works, see +[Activity filter](#activity-filter). <!-- vale gitlab.SubstitutionWarning = YES --> @@ -75,7 +85,7 @@ You can filter the vulnerabilities table by: To filter the list of vulnerabilities: 1. Select a filter. -1. Select values from the dropdown. +1. Select values from the dropdown list. 1. Repeat the above steps for each desired filter. After each filter is selected: @@ -83,11 +93,9 @@ After each filter is selected: - The list of matching vulnerabilities is updated. - The vulnerability severity totals are updated. -The filters' criteria are combined to show only vulnerabilities matching all criteria. -An exception to this behavior is the Activity filter. For more details about how it works, see -[Activity filter](#activity-filter). +### Tool filter -## Tool filter +> The third-party tool filter was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229661) in GitLab 13.12. The tool filter allows you to focus on vulnerabilities detected by selected tools. @@ -95,7 +103,7 @@ When using the tool filter, you can choose: - **All tools** (default). - Individual GitLab-provided tools. -- Any integrated 3rd-party tool. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229661) in GitLab 13.12. +- Any integrated third-party tool. For details of each of the available tools, see [Security scanning tools](../index.md#security-scanning-tools). @@ -103,11 +111,9 @@ For details of each of the available tools, see [Security scanning tools](../ind The content of the Project filter depends on the current level: -| Level | Content of the Project filter | -|:---------------|:------------------------------| -| Security Center | Only projects you've [added to your personal Security Center](../security_dashboard/index.md#add-projects-to-the-security-center). | -| Group level | All projects in the group. | -| Project level | Not applicable. | +- **Security Center**: Only projects you've [added to your personal Security Center](../security_dashboard/index.md#add-projects-to-the-security-center). +- **Group level**: All projects in the group. +- **Project level**: Not applicable. ### Activity filter @@ -119,13 +125,11 @@ all options can be selected in combination. Selection behavior when using the Activity filter: -| Activity selection | Results displayed | -|:------------------------------------|:------------------| -| All | Vulnerabilities with any Activity status (same as ignoring this filter). Selecting this deselects any other Activity filter options. | -| No activity | Only vulnerabilities without either an associated Issue or that are no longer detected. Selecting this deselects any other Activity filter options. | -| With issues | Only vulnerabilities with one or more associated issues. Does not include vulnerabilities that also are no longer detected. | -| No longer detected | Only vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. Does not include vulnerabilities with one or more associated issues. | -| With issues and No longer detected | Only vulnerabilities that have one or more associated issues and also are no longer detected in the latest pipeline scan of the `default` branch. | +- **All**: Vulnerabilities with any Activity status (same as ignoring this filter). Selecting this deselects any other Activity filter options. +- **No activity**: Only vulnerabilities without either an associated issue or that are no longer detected. Selecting this deselects any other Activity filter options. +- **With issues**: Only vulnerabilities with one or more associated issues. Does not include vulnerabilities that also are no longer detected. +- **No longer detected**: Only vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. Does not include vulnerabilities with one or more associated issues. +- **With issues** and **No longer detected**: Only vulnerabilities that have one or more associated issues and also are no longer detected in the latest pipeline scan of the `default` branch. ## View details of a vulnerability @@ -155,23 +159,32 @@ If Jira issue support is enabled, the issue link found in the Activity entry lin > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292636) in GitLab 13.10, all statuses became selectable. +From the Vulnerability Report you can change the status of one or more vulnerabilities. + To change the status of vulnerabilities in the table: -1. Select the checkbox for each vulnerability you want to update the status of. -1. In the dropdown that appears select the desired status, then select **Change status**. +1. Select the checkbox beside each vulnerability you want to update the status of. To select all, + select the checkbox in the table header. +1. In the **Set status** dropdown list, select the desired status. +1. Select **Change status**.  -### Change status of multiple vulnerabilities +## Dismissing a vulnerability -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35816) in GitLab 12.9. +When you evaluate a vulnerability and decide it requires no more action, you can mark it +as **Dismissed**. Dismissed vulnerabilities don't appear in the merge request security widget +when detected in future scans. -You can change the status of multiple vulnerabilities at once: +When a vulnerability is dismissed, a record is made of: -1. In the list of vulnerabilities, select the checkbox for each vulnerability you want to update. - To select all, select the checkbox in the table header. -1. Above the table, select a new status. -1. Click **Change status** to save. +- Who dismissed it. +- Date and time when it was dismissed. +- Optionally, a reason why it was dismissed. + +Vulnerability records cannot be deleted, so a permanent record always remains. + +If a vulnerability is dismissed in error, reverse the dismissal by changing its status. ## Export vulnerability details @@ -190,13 +203,19 @@ Fields included are: - Scanner name - Status - Vulnerability -- Details +- Basic details - Additional information - Severity - [CVE](https://cve.mitre.org/) (Common Vulnerabilities and Exposures) - [CWE](https://cwe.mitre.org/) (Common Weakness Enumeration) - Other identifiers +NOTE: +Full details are available through our +[Job Artifacts API](../../../api/job_artifacts.md#download-a-single-artifact-file-from-specific-tag-or-branch). +Use one of the `gl-*-report.json` report filenames in place of `*artifact_path` +to obtain, for example, the path of files in which vulnerabilities were detected. + ### Export details in CSV format To export details of all vulnerabilities listed in the Vulnerability Report, select **Export**. @@ -224,6 +243,7 @@ To undo this action, select a different status from the same menu. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301003) in GitLab 14.9. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/353796) in GitLab 14.10. +> - [Feature flag `new_vulnerability_form`](https://gitlab.com/gitlab-org/gitlab/-/issues/359049) removed in GitLab 15.0. To add a new vulnerability finding from your project level Vulnerability Report page: diff --git a/doc/user/clusters/agent/ci_cd_tunnel.md b/doc/user/clusters/agent/ci_cd_tunnel.md index c15041f6b0d..1b99fcf9739 100644 --- a/doc/user/clusters/agent/ci_cd_tunnel.md +++ b/doc/user/clusters/agent/ci_cd_tunnel.md @@ -1,261 +1,11 @@ --- -stage: Configure -group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'ci_cd_workflow.md' +remove_date: '2022-07-20' --- -# Using a GitLab CI/CD workflow for Kubernetes **(FREE)** +This document was moved to [another location](ci_cd_workflow.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. -> - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. -> - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. -> - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. -> - Support for Omnibus installations was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5686) in GitLab 14.5. -> - The ability to switch between certificate-based clusters and agents was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335089) in GitLab 14.9. The certificate-based cluster context is always called `gitlab-deploy`. - -You can use a GitLab CI/CD workflow to safely deploy to and update your Kubernetes clusters. - -To do so, you must first [install an agent in your cluster](install/index.md). When done, you have a Kubernetes context and can -run Kubernetes API commands in your GitLab CI/CD pipeline. - -To ensure access to your cluster is safe: - -- Each agent has a separate context (`kubecontext`). -- Only the project where the agent is configured, and any additional projects you authorize, can access the agent in your cluster. - -You do not need to have a runner in the cluster with the agent. - -## GitLab CI/CD workflow steps - -To update a Kubernetes cluster by using GitLab CI/CD, complete the following steps. - -1. Ensure you have a working Kubernetes cluster and the manifests are in a GitLab project. -1. In the same GitLab project, [register and install the GitLab agent](install/index.md). -1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to - select the agent's Kubernetes context and run the Kubernetes API commands. -1. Run your pipeline to deploy to or update the cluster. - -If you have multiple GitLab projects that contain Kubernetes manifests: - -1. [Install the GitLab agent](install/index.md) in its own project, or in one of the - GitLab projects where you keep Kubernetes manifests. -1. [Authorize the agent](#authorize-the-agent) to access your GitLab projects. -1. Optional. For added security, [use impersonation](#use-impersonation-to-restrict-project-and-group-access). -1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to - select the agent's Kubernetes context and run the Kubernetes API commands. -1. Run your pipeline to deploy to or update the cluster. - -## Authorize the agent - -You must authorize the agent to access the project where you keep your Kubernetes manifests. -You can authorize the agent to access individual projects, or authorize a group or subgroup, -so all projects within have access. For added security, you can also -[use impersonation](#use-impersonation-to-restrict-project-and-group-access). - -### Authorize the agent to access your projects - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. - -To authorize the agent to access the GitLab project where you keep Kubernetes manifests: - -1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). -1. Edit the file. Under the `ci_access` keyword, add the `projects` attribute. -1. For the `id`, add the path: - - ```yaml - ci_access: - projects: - - id: path/to/project - ``` - - - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. - - You can install additional agents into the same cluster to accommodate additional hierarchies. - - You can authorize up to 100 projects. - -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. -Choose the context to run `kubectl` commands from your CI/CD scripts. - -### Authorize the agent to access projects in your groups - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. - -To authorize the agent to access all of the GitLab projects in a group or subgroup: - -1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). -1. Edit the file. Under the `ci_access` keyword, add the `groups` attribute. -1. For the `id`, add the path: - - ```yaml - ci_access: - groups: - - id: path/to/group/subgroup - ``` - - - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. - - You can install additional agents into the same cluster to accommodate additional hierarchies. - - All of the subgroups of an authorized group also have access to the same agent (without being specified individually). - - You can authorize up to 100 groups. - -All the projects that belong to the group and its subgroups are now authorized to access the agent. -All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. -Choose the context to run `kubectl` commands from your CI/CD scripts. - -## Update your `.gitlab-ci.yml` file to run `kubectl` commands - -In the project where you want to run Kubernetes commands, edit your project's `.gitlab-ci.yml` file. - -In the first command under the `script` keyword, set your agent's context. -Use the format `path/to/agent/repository:agent-name`. For example: - -```yaml - deploy: - image: - name: bitnami/kubectl:latest - entrypoint: [""] - script: - - kubectl config get-contexts - - kubectl config use-context path/to/agent/repository:agent-name - - kubectl get pods -``` - -If you are not sure what your agent's context is, open a terminal and connect to your cluster. -Run `kubectl config get-contexts`. - -### Environments with both certificate-based and agent-based connections - -When you deploy to an environment that has both a [certificate-based -cluster](../../infrastructure/clusters/index.md) (deprecated) and an agent connection: - -- The certificate-based cluster's context is called `gitlab-deploy`. This context - is always selected by default. -- In GitLab 14.9 and later, agent contexts are included in the - `KUBECONFIG`. You can select them by using `kubectl config use-context - path/to/agent/repository:agent-name`. -- In GitLab 14.8 and earlier, you can still use agent connections, but for environments that - already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. - -To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` -configuration context. For example: - - ```yaml - deploy: - variables: - KUBE_CONTEXT: my-context # The name to use for the new context - AGENT_ID: 1234 # replace with your agent's numeric ID - K8S_PROXY_URL: wss://kas.gitlab.com/k8s-proxy/ # replace with your agent server (KAS) Kubernetes proxy URL - # ... any other variables you have configured - before_script: - - kubectl config set-credentials agent:$AGENT_ID --token="ci:${AGENT_ID}:${CI_JOB_TOKEN}" - - kubectl config set-cluster gitlab --server="${K8S_PROXY_URL}" - - kubectl config set-context "$KUBE_CONTEXT" --cluster=gitlab --user="agent:${AGENT_ID}" - - kubectl config use-context "$KUBE_CONTEXT" - # ... rest of your job configuration - ``` - -## Use impersonation to restrict project and group access **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. - -By default, your CI/CD job inherits all the permissions from the service account used to install the -agent in the cluster. -To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). - -To specify impersonations, use the `access_as` attribute in your agent configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. - -You can impersonate: - -- The agent itself (default). -- The CI/CD job that accesses the cluster. -- A specific user or system account defined within the cluster. - -### Impersonate the agent - -The agent is impersonated by default. You don't need to do anything to impersonate it. - -### Impersonate the CI/CD job that accesses the cluster - -To impersonate the CI/CD job that accesses the cluster, under the `access_as` key, add the `ci_job: {}` key-value. - -When the agent makes the request to the actual Kubernetes API, it sets the -impersonation credentials in the following way: - -- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. -- `Groups` is set to: - - `gitlab:ci_job` to identify all requests coming from CI jobs. - - The list of IDs of groups the project is in. - - The project ID. - - The slug of the environment this job belongs to. - - Example: for a CI job in `group1/group1-1/project1` where: - - - Group `group1` has ID 23. - - Group `group1/group1-1` has ID 25. - - Project `group1/group1-1/project1` has ID 150. - - Job running in a prod environment. - - Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. - -- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: - -| Property | Description | -| -------- | ----------- | -| `agent.gitlab.com/id` | Contains the agent ID. | -| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | -| `agent.gitlab.com/project_id` | Contains the CI project ID. | -| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | -| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | -| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | -| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | - -Example to restrict access by the CI/CD job's identity: - -```yaml -ci_access: - projects: - - id: path/to/project - access_as: - ci_job: {} -``` - -### Impersonate a static identity - -For a given connection, you can use a static identity for the impersonation. - -Under the `access_as` key, add the `impersonate` key to make the request using the provided identity. - -The identity can be specified with the following keys: - -- `username` (required) -- `uid` -- `groups` -- `extra` - -See the [official Kubernetes documentation for details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). - -## Troubleshooting - -### `kubectl` commands not supported - -The commands `kubectl exec`, `kubectl cp`, and `kubectl attach` are not supported. -Anything that uses these API endpoints does not work, because they use the deprecated -SPDY protocol. -[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) to add support for these commands. - -### Grant write permissions to `~/.kube/cache` - -Tools like `kubectl`, Helm, `kpt`, and `kustomize` cache information about -the cluster in `~/.kube/cache`. If this directory is not writable, the tool fetches information on each invocation, -making interactions slower and creating unnecessary load on the cluster. For the best experience, in the -image you use in your .`gitlab-ci.yml` file, ensure this directory is writable. - -### Enable TLS - -If you are on a self-managed GitLab instance, ensure your instance is configured with Transport Layer Security (TLS). - -If you attempt to use `kubectl` without TLS, you might get an error like: - -```shell -$ kubectl get pods -error: You must be logged in to the server (the server has asked for the client to provide credentials) -``` +<!-- This redirect file can be deleted after <2022-07-20>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/user/clusters/agent/ci_cd_workflow.md b/doc/user/clusters/agent/ci_cd_workflow.md new file mode 100644 index 00000000000..644a753e282 --- /dev/null +++ b/doc/user/clusters/agent/ci_cd_workflow.md @@ -0,0 +1,263 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Using GitLab CI/CD with a Kubernetes cluster **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. +> - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. +> - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. +> - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. +> - Support for Omnibus installations was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5686) in GitLab 14.5. +> - The ability to switch between certificate-based clusters and agents was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335089) in GitLab 14.9. The certificate-based cluster context is always called `gitlab-deploy`. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80508) from _CI/CD tunnel_ to _CI/CD workflow_ in GitLab 14.9. + +You can use a GitLab CI/CD workflow to safely deploy to and update your Kubernetes clusters. + +To do so, you must first [install an agent in your cluster](install/index.md). When done, you have a Kubernetes context and can +run Kubernetes API commands in your GitLab CI/CD pipeline. + +To ensure access to your cluster is safe: + +- Each agent has a separate context (`kubecontext`). +- Only the project where the agent is configured, and any additional projects you authorize, can access the agent in your cluster. + +You do not need to have a runner in the cluster with the agent. + +## GitLab CI/CD workflow steps + +To update a Kubernetes cluster by using GitLab CI/CD, complete the following steps. + +1. Ensure you have a working Kubernetes cluster and the manifests are in a GitLab project. +1. In the same GitLab project, [register and install the GitLab agent](install/index.md). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. + +If you have multiple GitLab projects that contain Kubernetes manifests: + +1. [Install the GitLab agent](install/index.md) in its own project, or in one of the + GitLab projects where you keep Kubernetes manifests. +1. [Authorize the agent](#authorize-the-agent) to access your GitLab projects. +1. Optional. For added security, [use impersonation](#use-impersonation-to-restrict-project-and-group-access). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. + +## Authorize the agent + +You must authorize the agent to access the project where you keep your Kubernetes manifests. +You can authorize the agent to access individual projects, or authorize a group or subgroup, +so all projects within have access. For added security, you can also +[use impersonation](#use-impersonation-to-restrict-project-and-group-access). + +### Authorize the agent to access your projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. + +To authorize the agent to access the GitLab project where you keep Kubernetes manifests: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the `config.yaml` file. Under the `ci_access` keyword, add the `projects` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + projects: + - id: path/to/project + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can install additional agents into the same cluster to accommodate additional hierarchies. + - You can authorize up to 100 projects. + +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +### Authorize the agent to access projects in your groups + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. + +To authorize the agent to access all of the GitLab projects in a group or subgroup: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the `config.yaml` file. Under the `ci_access` keyword, add the `groups` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + groups: + - id: path/to/group/subgroup + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can install additional agents into the same cluster to accommodate additional hierarchies. + - All of the subgroups of an authorized group also have access to the same agent (without being specified individually). + - You can authorize up to 100 groups. + +All the projects that belong to the group and its subgroups are now authorized to access the agent. +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +## Update your `.gitlab-ci.yml` file to run `kubectl` commands + +In the project where you want to run Kubernetes commands, edit your project's `.gitlab-ci.yml` file. + +In the first command under the `script` keyword, set your agent's context. +Use the format `path/to/agent/repository:agent-name`. For example: + +```yaml +deploy: + image: + name: bitnami/kubectl:latest + entrypoint: [''] + script: + - kubectl config get-contexts + - kubectl config use-context path/to/agent/repository:agent-name + - kubectl get pods +``` + +If you are not sure what your agent's context is, open a terminal and connect to your cluster. +Run `kubectl config get-contexts`. + +### Environments with both certificate-based and agent-based connections + +When you deploy to an environment that has both a [certificate-based +cluster](../../infrastructure/clusters/index.md) (deprecated) and an agent connection: + +- The certificate-based cluster's context is called `gitlab-deploy`. This context + is always selected by default. +- In GitLab 14.9 and later, agent contexts are included in the + `KUBECONFIG`. You can select them by using `kubectl config use-context path/to/agent/repository:agent-name`. +- In GitLab 14.8 and earlier, you can still use agent connections, but for environments that + already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. + +To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` +configuration context. For example: + +```yaml +deploy: + variables: + KUBE_CONTEXT: my-context # The name to use for the new context + AGENT_ID: 1234 # replace with your agent's numeric ID + K8S_PROXY_URL: https://<KAS_DOMAIN>/k8s-proxy/ # For agent server (KAS) deployed in Kubernetes cluster (for gitlab.com use kas.gitlab.com); replace with your URL + # K8S_PROXY_URL: https://<GITLAB_DOMAIN>/-/kubernetes-agent/k8s-proxy/ # For agent server (KAS) in Omnibus + # ... any other variables you have configured + before_script: + - kubectl config set-credentials agent:$AGENT_ID --token="ci:${AGENT_ID}:${CI_JOB_TOKEN}" + - kubectl config set-cluster gitlab --server="${K8S_PROXY_URL}" + - kubectl config set-context "$KUBE_CONTEXT" --cluster=gitlab --user="agent:${AGENT_ID}" + - kubectl config use-context "$KUBE_CONTEXT" + # ... rest of your job configuration +``` + +## Use impersonation to restrict project and group access **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. + +By default, your CI/CD job inherits all the permissions from the service account used to install the +agent in the cluster. +To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +To specify impersonations, use the `access_as` attribute in your agent configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. + +You can impersonate: + +- The agent itself (default). +- The CI/CD job that accesses the cluster. +- A specific user or system account defined within the cluster. + +### Impersonate the agent + +The agent is impersonated by default. You don't need to do anything to impersonate it. + +### Impersonate the CI/CD job that accesses the cluster + +To impersonate the CI/CD job that accesses the cluster, under the `access_as` key, add the `ci_job: {}` key-value. + +When the agent makes the request to the actual Kubernetes API, it sets the +impersonation credentials in the following way: + +- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. +- `Groups` is set to: + + - `gitlab:ci_job` to identify all requests coming from CI jobs. + - The list of IDs of groups the project is in. + - The project ID. + - The slug of the environment this job belongs to. + + Example: for a CI job in `group1/group1-1/project1` where: + + - Group `group1` has ID 23. + - Group `group1/group1-1` has ID 25. + - Project `group1/group1-1/project1` has ID 150. + - Job running in a prod environment. + + Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. + +- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: + +| Property | Description | +| ------------------------------------ | ---------------------------------------------------------------------------- | +| `agent.gitlab.com/id` | Contains the agent ID. | +| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | +| `agent.gitlab.com/project_id` | Contains the CI project ID. | +| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | +| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | +| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | +| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | + +Example `config.yaml` to restrict access by the CI/CD job's identity: + +```yaml +ci_access: + projects: + - id: path/to/project + access_as: + ci_job: {} +``` + +### Impersonate a static identity + +For a given connection, you can use a static identity for the impersonation. + +Under the `access_as` key, add the `impersonate` key to make the request using the provided identity. + +The identity can be specified with the following keys: + +- `username` (required) +- `uid` +- `groups` +- `extra` + +See the [official Kubernetes documentation for details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +## Troubleshooting + +### `kubectl` commands not supported + +The commands `kubectl exec`, `kubectl cp`, `kubectl attach`, `kubectl run --attach=true` and `kubectl port-forward` are not supported. +Anything that uses these API endpoints does not work, because they use the deprecated +SPDY protocol. +[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) to add support for these commands. + +### Grant write permissions to `~/.kube/cache` + +Tools like `kubectl`, Helm, `kpt`, and `kustomize` cache information about +the cluster in `~/.kube/cache`. If this directory is not writable, the tool fetches information on each invocation, +making interactions slower and creating unnecessary load on the cluster. For the best experience, in the +image you use in your `.gitlab-ci.yml` file, ensure this directory is writable. + +### Enable TLS + +If you are on a self-managed GitLab instance, ensure your instance is configured with Transport Layer Security (TLS). + +If you attempt to use `kubectl` without TLS, you might get an error like: + +```shell +$ kubectl get pods +error: You must be logged in to the server (the server has asked for the client to provide credentials) +``` diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md index e99e3b00ec7..6ca9d855b44 100644 --- a/doc/user/clusters/agent/gitops.md +++ b/doc/user/clusters/agent/gitops.md @@ -4,7 +4,7 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Using a GitOps workflow for Kubernetes **(PREMIUM)** +# Using GitOps with a Kubernetes cluster **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index bab3f3137fe..d54d432f0f5 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -35,7 +35,7 @@ In a [**GitOps** workflow](gitops.md), you keep your Kubernetes manifests in Git any time you update your manifests, the agent updates the cluster. This workflow is fully driven with Git and is considered pull-based, because the cluster is pulling updates from your GitLab repository. -In a [**CI/CD** workflow](ci_cd_tunnel.md), you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. +In a [**CI/CD** workflow](ci_cd_workflow.md), you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. This workflow is considered push-based, because GitLab is pushing requests from GitLab CI/CD to your cluster. ## Supported cluster versions @@ -43,6 +43,7 @@ This workflow is considered push-based, because GitLab is pushing requests from GitLab supports the following Kubernetes versions. You can upgrade your Kubernetes version to a supported version at any time: +- 1.22 (support ends on March 22, 2023) - 1.21 (support ends on November 22, 2022) - 1.20 (support ends on July 22, 2022) @@ -65,7 +66,7 @@ Read about how to [migrate to the agent for Kubernetes](../../infrastructure/clu ## Related topics - [GitOps workflow](gitops.md) -- [GitLab CI/CD workflow](ci_cd_tunnel.md) +- [GitLab CI/CD workflow](ci_cd_workflow.md) - [Install the agent](install/index.md) - [Work with the agent](repository.md) - [Troubleshooting](troubleshooting.md) diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index e76ef9e827d..f747c6c0e25 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -20,39 +20,51 @@ Before you can install the agent in your cluster, you need: - [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/quickstart) - [Amazon Elastic Kubernetes Service (EKS)](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) - [Digital Ocean](https://docs.digitalocean.com/products/kubernetes/quickstart/) -- On self-managed GitLab instances, a GitLab administrator must set up the [agent server](../../../../administration/clusters/kas.md). +- On self-managed GitLab instances, a GitLab administrator must set up the [agent server](../../../../administration/clusters/kas.md). Then it will be available by default at `wss://gitlab.example.com/-/kubernetes-agent/`. On GitLab.com, the agent server is available at `wss://kas.gitlab.com`. ## Installation steps To install the agent in your cluster: +1. [Choose a name for the agent](#agent-naming-convention). 1. [Register the agent with GitLab](#register-the-agent-with-gitlab). 1. [Install the agent in your cluster](#install-the-agent-in-the-cluster). <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a GitLab 14.2 [walk-through of this process](https://www.youtube.com/watch?v=XuBpKtsgGkE). +### Agent naming convention + +The agent name must follow the [DNS label standard from RFC 1123](https://tools.ietf.org/html/rfc1123). +The name must: + +- Be unique in the project. +- Contain at most 63 characters. +- Contain only lowercase alphanumeric characters or `-`. +- Start with an alphanumeric character. +- End with an alphanumeric character. + ### Register the agent with GitLab > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5786) in GitLab 14.1, you can create a new agent record directly from the GitLab UI. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347240) in GitLab 14.9, the agent can be registered without creating an agent configuration file. -You must register an agent with GitLab. - FLAG: In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `certificate_based_clusters` changed the **Actions** menu to focus on the agent rather than certificates. The flag is [enabled on GitLab.com and self-managed](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). Prerequisites: -- For a [GitLab CI/CD workflow](../ci_cd_tunnel.md), ensure that +- For a [GitLab CI/CD workflow](../ci_cd_workflow.md), ensure that [GitLab CI/CD is enabled](../../../../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). -To register an agent with GitLab: +You must register an agent before you can install the agent in your cluster. To register an agent: 1. On the top bar, select **Menu > Projects** and find your project. + If you have an [agent configuration file](#create-an-agent-configuration-file), + it must be in this project. Your cluster manifest files should also be in this project. 1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. Select **Connect a cluster (agent)**. - - If you want to create a configuration with CI/CD defaults, type a name for the agent. + - If you want to create a configuration with CI/CD defaults, type a name that meets [the naming convention](#agent-naming-convention). - If you already have an [agent configuration file](#create-an-agent-configuration-file), select it from the list. 1. Select **Register an agent**. 1. GitLab generates an access token for the agent. Securely store this token. You need it to install the agent in your cluster and to [update the agent](#update-the-agent-version) to another version. @@ -63,24 +75,23 @@ To register an agent with GitLab: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the agent configuration file can be added to multiple directories (or subdirectories) of the repository. > - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. -The agent is configured through a configuration file. This file is optional. Without a configuration file, you can still use the CI/CD workflow in the project where the agent is registered. - -You need a configuration file if: +The agent uses a YAML file for configuration settings. You need a configuration file if: - You want to use [a GitOps workflow](../gitops.md#gitops-configuration-reference). -- You want to authorize a different project to use the agent for a [GitLab CI/CD workflow](../ci_cd_tunnel.md#authorize-the-agent). +- You want to authorize a different project to use the agent for a [GitLab CI/CD workflow](../ci_cd_workflow.md#authorize-the-agent). -To create an agent configuration file, go to the GitLab project. In the repository, create a file called `config.yaml` at this path: +To create an agent configuration file: -```plaintext -.gitlab/agents/<agent-name>/config.yaml -``` +1. In the repository, create a directory in this location. The `<agent-name>` must meet [the naming convention](#agent-naming-convention). + + ```plaintext + .gitlab/agents/<agent-name> + ``` -- Ensure the agent name follows the [naming convention](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). -- Ensure the filename has the `.yaml` file extension (`config.yaml`). The `.yml` extension is not accepted. -- Add content to the `config.yaml` file: - - For a GitOps workflow, view [the configuration reference](../gitops.md#gitops-configuration-reference) for details. - - For a GitLab CI/CD workflow, you can leave the file blank for now. +1. In the directory, create a `config.yaml` file. Ensure the filename ends in `.yaml`, not `.yml`. +1. Add content to the `config.yaml` file: + - For a GitOps workflow, view [the configuration reference](../gitops.md#gitops-configuration-reference) for details. + - For a GitLab CI/CD workflow, view [the configuration reference](../ci_cd_workflow.md) for details. ### Install the agent in the cluster @@ -98,9 +109,9 @@ If you do not know which one to choose, we recommend starting with Helm. To install the agent on your cluster using Helm: -1. [Install Helm](https://helm.sh/docs/intro/install/) +1. [Install Helm](https://helm.sh/docs/intro/install/). 1. In your computer, open a terminal and [connect to your cluster](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/). -1. Run the command you copied when registering your agent with GitLab. +1. Run the command you copied when you [registered your agent with GitLab](#register-the-agent-with-gitlab). Optionally, you can [customize the Helm installation](#customize-the-helm-installation). diff --git a/doc/user/clusters/agent/repository.md b/doc/user/clusters/agent/repository.md index 2087c804e26..8f3a8830202 100644 --- a/doc/user/clusters/agent/repository.md +++ b/doc/user/clusters/agent/repository.md @@ -1,210 +1,11 @@ --- -stage: Configure -group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'work_with_agent.md' +remove_date: '2022-07-19' --- -# Working with the agent for Kubernetes **(FREE)** +This document was moved to [another location](work_with_agent.md). -Use the following tasks when working with the agent for Kubernetes. - -## View your agents - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, the installed `agentk` version is displayed on the **Agent** tab. - -Prerequisite: - -- You must have at least the Developer role. - -To view the list of agents: - -1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Agent** tab to view clusters connected to GitLab through the agent. - -On this page, you can view: - -- All the registered agents for the current project. -- The connection status. -- The version of `agentk` installed on your cluster. -- The path to each agent configuration file. - -## View an agent's activity information - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. - -The activity logs help you to identify problems and get the information -you need for troubleshooting. You can see events from a week before the -current date. To view an agent's activity: - -1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select the agent you want to see activity for. - -The activity list includes: - -- Agent registration events: When a new token is **created**. -- Connection events: When an agent is successfully **connected** to a cluster. - -The connection status is logged when you connect an agent for -the first time or after more than an hour of inactivity. - -View and provide feedback about the UI in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). - -## Debug the agent - -To debug the cluster-side component (`agentk`) of the agent, set the log -level according to the available options: - -- `off` -- `warning` -- `error` -- `info` -- `debug` - -The log level defaults to `info`. You can change it by using a top-level `observability` -section in the configuration file, for example: - -```yaml -observability: - logging: - level: debug -``` - -## Reset the agent token - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327152) in GitLab 14.9. - -To reset the agent token without downtime: - -1. Create a new token: - 1. On the top bar, select **Menu > Projects** and find your project. - 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. - 1. Select the agent you want to create a token for. - 1. On the **Tokens** tab, select **Create token**. - 1. Enter token's name and description (optional) and select **Create token**. -1. Securely store the generated token. -1. Use the token to [install the agent in your cluster](install/index.md#install-the-agent-in-the-cluster) and to [update the agent](install/index.md#update-the-agent-version) to another version. -1. Delete the token you're no longer using. - -## Remove an agent - -You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the -[GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). The agent and any associated tokens -are removed from GitLab, but no changes are made in your Kubernetes cluster. You must -clean up those resources manually. - -### Remove an agent through the GitLab UI - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. - -To remove an agent from the UI: - -1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. -1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). -1. Select **Delete agent**. - -### Remove an agent with the GitLab GraphQL API - -1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. - - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. - - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. - - ```graphql - query{ - project(fullPath: "<full-path-to-agent-configuration-project>") { - clusterAgent(name: "<agent-name>") { - id - tokens { - edges { - node { - id - } - } - } - } - } - } - ``` - -1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. - - ```graphql - mutation deleteAgent { - clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { - errors - } - } - - mutation deleteToken { - clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { - errors - } - } - ``` - -1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: - - ```json - { - "level": "warn", - "time": "2021-04-29T23:44:07.598Z", - "msg": "GetConfiguration.Recv failed", - "error": "rpc error: code = Unauthenticated desc = unauthenticated" - } - ``` - -1. Delete the agent in your cluster: - - ```shell - kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml - ``` - -## Surface network security alerts from cluster to GitLab **(ULTIMATE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Cilium integration is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The agent for Kubernetes also provides an integration with Cilium. This integration provides a simple way to -generate network policy-related alerts and to surface those alerts in GitLab. - -Several components work in concert for the agent to generate the alerts: - -- A working Kubernetes cluster. -- Cilium integration through either of these options: - - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). - - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an - existing installation. -- One or more network policies through any of these options: - - Use the [Container Network Policy editor](../../application_security/policies/index.md#container-network-policy-editor) to create and manage policies. - - Use an [AutoDevOps](../../application_security/policies/index.md#container-network-policy) configuration. - - Add the required labels and annotations to existing network policies. -- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) - -The setup process follows the same [agent's installation steps](install/index.md), -with the following differences: - -- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). -- You do not need to specify the `gitops` configuration section. - -To integrate, add a top-level `cilium` section to your `config.yml` file. Currently, the -only configuration option is the Hubble relay address: - -```yaml -cilium: - hubble_relay_address: "<hubble-relay-host>:<hubble-relay-port>" -``` - -If your Cilium integration was performed through [GitLab Managed Apps](../applications.md#install-cilium-using-gitlab-cicd) or the -[cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium), -you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the address: - -```yaml -cilium: - hubble_relay_address: "hubble-relay.gitlab-managed-apps.svc.cluster.local:80" -``` +<!-- This redirect file can be deleted after <2022-07-19>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index c5c7e46c078..0932e9179f9 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -11,7 +11,7 @@ When you are using the GitLab agent for Kubernetes, you might experience issues You can start by viewing the service logs: ```shell -kubectl logs -f -l=app=gitlab-agent -n gitlab-kubernetes-agent +kubectl logs -f -l=app=gitlab-agent -n gitlab-agent ``` If you are a GitLab administrator, you can also view the [GitLab agent server logs](../../../administration/clusters/kas.md#troubleshooting). @@ -113,14 +113,14 @@ will be picked up automatically. For example, if your internal CA certificate is `myCA.pem`: ```plaintext -kubectl -n gitlab-kubernetes-agent create configmap ca-pemstore --from-file=myCA.pem +kubectl -n gitlab-agent create configmap ca-pemstore --from-file=myCA.pem ``` Then in `resources.yml`: ```yaml spec: - serviceAccountName: gitlab-kubernetes-agent + serviceAccountName: gitlab-agent containers: - name: agent image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:<version>" @@ -140,7 +140,7 @@ Then in `resources.yml`: volumes: - name: token-volume secret: - secretName: gitlab-kubernetes-agent-token + secretName: gitlab-agent-token - name: ca-pemstore-volume configMap: name: ca-pemstore diff --git a/doc/user/clusters/agent/vulnerabilities.md b/doc/user/clusters/agent/vulnerabilities.md index 480b09ff2ab..69f5b1d9063 100644 --- a/doc/user/clusters/agent/vulnerabilities.md +++ b/doc/user/clusters/agent/vulnerabilities.md @@ -34,80 +34,34 @@ You can use [cluster image scanning](../../application_security/cluster_image_sc to scan container images in your cluster for security vulnerabilities. To begin scanning all resources in your cluster, add a `starboard` -configuration block to your agent configuration file with no `filters`: +configuration block to your agent configuration with a `cadence` field +containing a CRON expression for when the scans will be run. ```yaml starboard: vulnerability_report: - filters: [] + cadence: '0 0 * * *' # Daily at 00:00 (Kubernetes cluster time) ``` -The namespaces that are able to be scanned depend on the [Starboard Operator install mode](https://aquasecurity.github.io/starboard/latest/operator/configuration/#install-modes). -By default, the Starboard Operator only scans resources in the `default` namespace. To change this -behavior, edit the `STARBOARD_OPERATOR` environment variable in the `starboard-operator` deployment -definition. +The `cadence` field is required. GitLab supports the following types of CRON syntax for the cadence field: -By adding filters, you can limit scans by: +- A daily cadence of once per hour at a specified hour, for example: `0 18 * * *` +- A weekly cadence of once per week on a specified day and at a specified hour, for example: `0 13 * * 0` -- Resource name -- Kind -- Container name -- Namespace +It is possible that other elements of the CRON syntax will work in the cadence field, however, GitLab does not officially test or support them. -```yaml -starboard: - vulnerability_report: - filters: - - namespaces: - - staging - - production - kinds: - - Deployment - - DaemonSet - containers: - - ruby - - postgres - - nginx - resources: - - my-app-name - - postgres - - ingress-nginx -``` - -A resource is scanned if the resource matches any of the given names and all of the given filter -types (`namespaces`, `kinds`, `containers`, `resources`). If a filter type is omitted, then all -names are scanned. In this example, a resource isn't scanned unless it has a container named `ruby`, -`postgres`, or `nginx`, and it's a `Deployment`: - -```yaml -starboard: - vulnerability_report: - filters: - - kinds: - - Deployment - containers: - - ruby - - postgres - - nginx -``` - -There is also a global `namespaces` field that applies to all filters: +By default, cluster image scanning will attempt to scan the workloads in all +namespaces for vulnerabilities. The `vulnerability_report` block has a `namespaces` +field which can be used to restrict which namespaces are scanned. For example, +if you would like to scan only the `development`, `staging`, and `production` +namespaces, you can use this configuration: ```yaml starboard: vulnerability_report: + cadence: '0 0 * * *' namespaces: - - production - filters: - - kinds: - - Deployment - - kinds: - - DaemonSet - resources: - - log-collector + - development + - staging + - production ``` - -In this example, the following resources are scanned: - -- All deployments (`Deployment`) in the `production` namespace. -- All daemon sets (`DaemonSet`) named `log-collector` in the `production` namespace. diff --git a/doc/user/clusters/agent/work_with_agent.md b/doc/user/clusters/agent/work_with_agent.md new file mode 100644 index 00000000000..8872ecf7ce5 --- /dev/null +++ b/doc/user/clusters/agent/work_with_agent.md @@ -0,0 +1,163 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Working with the agent for Kubernetes **(FREE)** + +Use the following tasks when working with the agent for Kubernetes. + +## View your agents + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, the installed `agentk` version is displayed on the **Agent** tab. + +Prerequisite: + +- You must have at least the Developer role. + +To view the list of agents: + +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Agent** tab to view clusters connected to GitLab through the agent. + +On this page, you can view: + +- All the registered agents for the current project. +- The connection status. +- The version of `agentk` installed on your cluster. +- The path to each agent configuration file. + +## View an agent's activity information + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. + +The activity logs help you to identify problems and get the information +you need for troubleshooting. You can see events from a week before the +current date. To view an agent's activity: + +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select the agent you want to see activity for. + +The activity list includes: + +- Agent registration events: When a new token is **created**. +- Connection events: When an agent is successfully **connected** to a cluster. + +The connection status is logged when you connect an agent for +the first time or after more than an hour of inactivity. + +View and provide feedback about the UI in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). + +## Debug the agent + +To debug the cluster-side component (`agentk`) of the agent, set the log +level according to the available options: + +- `off` +- `warning` +- `error` +- `info` +- `debug` + +The log level defaults to `info`. You can change it by using a top-level `observability` +section in the configuration file, for example: + +```yaml +observability: + logging: + level: debug +``` + +## Reset the agent token + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327152) in GitLab 14.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336641) in GitLab 14.10, the agent token can be revoked from the UI. + +To reset the agent token without downtime: + +1. Create a new token: + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. + 1. Select the agent you want to create a token for. + 1. On the **Access tokens** tab, select **Create token**. + 1. Enter token's name and description (optional) and select **Create token**. +1. Securely store the generated token. +1. Use the token to [install the agent in your cluster](install/index.md#install-the-agent-in-the-cluster) and to [update the agent](install/index.md#update-the-agent-version) to another version. +1. To delete the token you're no longer using, return to the token list and select **Revoke** (**{remove}**). + +## Remove an agent + +You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the +[GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). The agent and any associated tokens +are removed from GitLab, but no changes are made in your Kubernetes cluster. You must +clean up those resources manually. + +### Remove an agent through the GitLab UI + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. + +To remove an agent from the UI: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). +1. Select **Delete agent**. + +### Remove an agent with the GitLab GraphQL API + +1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. + - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. + - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. + + ```graphql + query{ + project(fullPath: "<full-path-to-agent-configuration-project>") { + clusterAgent(name: "<agent-name>") { + id + tokens { + edges { + node { + id + } + } + } + } + } + } + ``` + +1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. + + ```graphql + mutation deleteAgent { + clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { + errors + } + } + + mutation deleteToken { + clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { + errors + } + } + ``` + +1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: + + ```json + { + "level": "warn", + "time": "2021-04-29T23:44:07.598Z", + "msg": "GetConfiguration.Recv failed", + "error": "rpc error: code = Unauthenticated desc = unauthenticated" + } + ``` + +1. Delete the agent in your cluster: + + ```shell + kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml + ``` diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 2bcfea50ee3..7b5ea7d12fd 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -2,1134 +2,12 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../update/removals.md#managed-cluster-applicationsgitlab-ciyml' --- -# GitLab Managed Apps (DEPRECATED) **(FREE)** +# GitLab Managed Apps (removed) **(FREE)** -NOTE: -The new recommended way to manage cluster applications is to use the [cluster management project template](management_project_template.md). -If you want to migrate your GitLab managed apps management to this template, reference to [migrating from GitLab managed apps to project template](migrating_from_gma_to_project_template.md). - -**GitLab Managed Apps** was created to help you configure applications in your -cluster directly from GitLab. You could use this feature through two different -methods: "one-click install" and "CI/CD template". Both methods are **deprecated**: - -- The **one-click install** method was [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63348) in GitLab 14.0. -- The **CI/CD template method** was deprecated in GitLab 13.12 and is scheduled - to be removed in GitLab 15.0. - -Both methods were limiting as you couldn't fully customize your third-party apps installed -through GitLab Managed Apps. Therefore, we decided to deprecate this feature and provide -better [GitOps-driven alternatives](https://about.gitlab.com/direction/configure/kubernetes_management/#gitlab-managed-applications) to our users, such as [cluster integrations](integrations.md) and [cluster management project](management_project.md). - -## Install using GitLab CI/CD (DEPRECATED) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20822) in GitLab 12.6. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -WARNING: -The GitLab Managed Apps CI/CD installation method was [deprecated in 13.12](https://gitlab.com/gitlab-org/gitlab/-/issues/327908). -Your applications continue to work. However, we no longer support and maintain the GitLab CI/CD template for -Managed Apps (`Managed-Cluster-Applications.gitlab-ci.yml`). -The new recommended way to manage cluster applications is to use the [cluster management project template](management_project_template.md). -If you want to migrate your GitLab managed apps management to this template, reference to [migrating from GitLab managed apps to project template](migrating_from_gma_to_project_template.md). - -The CI/CD template was the primary method for installing applications to clusters via GitLab Managed Apps -and customize them through Helm. - -Supported applications: - -- [Ingress](#install-ingress-using-gitlab-cicd) -- [cert-manager](#install-cert-manager-using-gitlab-cicd) -- [Sentry](#install-sentry-using-gitlab-cicd) -- [GitLab Runner](#install-gitlab-runner-using-gitlab-cicd) -- [Cilium](#install-cilium-using-gitlab-cicd) -- [Falco](#install-falco-using-gitlab-cicd) -- [Vault](#install-vault-using-gitlab-cicd) -- [JupyterHub](#install-jupyterhub-using-gitlab-cicd) -- [Elastic Stack](#install-elastic-stack-using-gitlab-cicd) -- [Crossplane](#install-crossplane-using-gitlab-cicd) -- [Fluentd](#install-fluentd-using-gitlab-cicd) -- [Knative](#install-knative-using-gitlab-cicd) -- [PostHog](#install-posthog-using-gitlab-cicd) -- [Prometheus](#install-prometheus-using-gitlab-cicd) - -### Prerequisites - -You can find and import all the files referenced below -in the [example cluster applications -project](https://gitlab.com/gitlab-org/cluster-integration/example-cluster-applications/). - -To install applications using GitLab CI/CD: - -1. Connect the cluster to a [cluster management project](management_project.md). -1. In that project, add a `.gitlab-ci.yml` file with the following content: - - ```yaml - include: - - template: Managed-Cluster-Applications.gitlab-ci.yml - ``` - - The job provided by this template connects to the `*` (default) cluster using tools provided - in a custom Docker image. It requires that you have a runner registered with the Docker, - Kubernetes, or Docker Machine executor. - - To install to a specific cluster, read - [Use the template with a custom environment](#use-the-template-with-a-custom-environment). - -1. Add a `.gitlab/managed-apps/config.yaml` file to define which - applications you would like to install. Define the `installed` key as - `true` to install the application and `false` to uninstall the - application. For example, to install Ingress: - - ```yaml - ingress: - installed: true - ``` - -1. Optionally, define `.gitlab/managed-apps/<application>/values.yaml` file to - customize values for the installed application. - -A GitLab CI/CD pipeline runs on the default branch to install the -applications you have configured. In case of pipeline failure, the -output of the [Helm Tiller](https://v2.helm.sh/docs/install/#running-tiller-locally) binary -is saved as a [CI job artifact](../../ci/pipelines/job_artifacts.md). - -#### Prerequisites in GitLab versions earlier than 13.5 - -For GitLab versions 13.5 and earlier, the Ingress, Fluentd, Prometheus, and Sentry -apps were fetched from the central Helm stable repository (`https://kubernetes-charts.storage.googleapis.com/`). -This repository [was deleted](https://github.com/helm/charts#deprecation-timeline) -on November 13, 2020. This causes the installation CI/CD pipeline to -fail. Upgrade to GitLab 13.6, or alternatively, you can -use the following `.gitlab-ci.yml`, which has been tested in GitLab 13.5: - -```yaml -include: - - template: Managed-Cluster-Applications.gitlab-ci.yml - -apply: - image: "registry.gitlab.com/gitlab-org/cluster-integration/cluster-applications:v0.37.0" -``` - -### Use the template with a custom environment - -If you only want apps to be installed on a specific cluster, or if your cluster's -scope does not match `production`, you can override the environment name in your `.gitlab-ci.yml` -file: - -```yaml -include: - - template: Managed-Cluster-Applications.gitlab-ci.yml - -apply: - except: - variables: - - '$CI_JOB_NAME == "apply"' - -.managed-apps: - extends: apply - -example-install: - extends: .managed-apps - environment: - name: example/production -``` - -### Important notes - -Note the following: - -- We recommend using the cluster management project exclusively for managing deployments to a cluster. - Do not add your application's source code to such projects. -- When you set the value for `installed` key back to `false`, the application is - unprovisioned from the cluster. -- If you update `.gitlab/managed-apps/<application>/values.yaml` with new values, the - application is redeployed. - -### Install Ingress using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -To install Ingress, define the `.gitlab/managed-apps/config.yaml` file -with: - -```yaml -ingress: - installed: true -``` - -Ingress is installed into the `gitlab-managed-apps` namespace -of your cluster. - -You can customize the installation of Ingress by defining a -`.gitlab/managed-apps/ingress/values.yaml` file in your cluster -management project. Refer to the -[chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) -for the available configuration options. - -Support for installing the Ingress managed application is provided by the GitLab Configure group. -If you run into unknown issues, [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), -and ping at least 2 people from the -[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). - -### Install cert-manager using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -cert-manager is installed using GitLab CI/CD by defining configuration in -`.gitlab/managed-apps/config.yaml`. - -cert-manager: - -- Is installed into the `gitlab-managed-apps` namespace of your cluster. -- Can be installed with or without a default - [Let's Encrypt `ClusterIssuer`](https://cert-manager.io/docs/configuration/acme/), which requires an - email address to be specified. The email address is used by Let's Encrypt to - contact you about expiring certificates and issues related to your account. - -The following configuration is required to install cert-manager using GitLab CI/CD: - -```yaml -certManager: - installed: true - letsEncryptClusterIssuer: - installed: true - email: "user@example.com" -``` - -The following installs cert-manager using GitLab CI/CD without the default `ClusterIssuer`: - -```yaml -certManager: - installed: true - letsEncryptClusterIssuer: - installed: false -``` - -You can customize the installation of cert-manager by defining a -`.gitlab/managed-apps/cert-manager/values.yaml` file in your cluster -management project. Refer to the -[chart](https://github.com/jetstack/cert-manager) for the -available configuration options. - -Support for installing the Cert Manager managed application is provided by the -GitLab Configure group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). - -### Install Sentry using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) -at least 3 GB of available RAM for database migrations. - -To install Sentry, define the `.gitlab/managed-apps/config.yaml` file -with: - -```yaml -sentry: - installed: true -``` - -Sentry is installed into the `gitlab-managed-apps` namespace -of your cluster. - -You can customize the installation of Sentry by defining -`.gitlab/managed-apps/sentry/values.yaml` file in your cluster -management project. Refer to the -[chart](https://github.com/helm/charts/tree/master/stable/sentry) -for the available configuration options. - -We recommend you pay close attention to the following configuration options: - -- `email`. Needed to invite users to your Sentry instance and to send error emails. -- `user`. Where you can set the login credentials for the default administrator user. -- `postgresql`. For a PostgreSQL password that can be used when running future updates. - -When upgrading, it's important to provide the existing PostgreSQL password (given -using the `postgresql.postgresqlPassword` key) to avoid authentication errors. -Read the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) -for more information. - -Here is an example configuration for Sentry: - -```yaml -# Admin user to create -user: - # Indicated to create the admin user or not, - # Default is true as the initial installation. - create: true - email: "<your email>" - password: "<your password>" - -email: - from_address: "<your from email>" - host: smtp - port: 25 - use_tls: false - user: "<your email username>" - password: "<your email password>" - enable_replies: false - -ingress: - enabled: true - hostname: "<sentry.example.com>" - -# Needs to be here between runs. -# See https://github.com/helm/charts/tree/master/stable/postgresql#upgrade for more info -postgresql: - postgresqlPassword: example-postgresql-password -``` - -Support for installing the Sentry managed application is provided by the -GitLab Monitor group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Monitor group](https://about.gitlab.com/handbook/product/categories/#monitor-group). - -### Install PostHog using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -[PostHog](https://posthog.com) 🦔 is a developer-friendly, open-source product analytics platform. - -To install PostHog into the `gitlab-managed-apps` namespace of your cluster, -define the `.gitlab/managed-apps/config.yaml` file with: - -```yaml -posthog: - installed: true -``` - -You can customize the installation of PostHog by defining `.gitlab/managed-apps/posthog/values.yaml` -in your cluster management project. Refer to the -[Configuration section](https://github.com/PostHog/charts/tree/master/charts/posthog) -of the PostHog chart's README for the available configuration options. - -You must provide a PostgreSQL password in `postgresql.postgresqlPassword` -to avoid authentication errors. Read the -[PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) -for more information. - -Redis pods are restarted between upgrades. To prevent downtime, provide a Redis -password using the `redis.password` key. This prevents a new password from -being generated on each restart. - -Here is an example configuration for PostHog: - -```yaml -ingress: - enabled: true - hostname: "<posthog.example.com>" - -# This will be autogenerated if you skip it. Include if you have 2 or more web replicas -posthogSecret: 'long-secret-key-used-to-sign-cookies' - -# Needs to be here between runs. -# See https://github.com/helm/charts/tree/master/stable/postgresql#upgrade for more info -postgresql: - postgresqlPassword: example-postgresql-password - -# Recommended to set this to a value to redis prevent downtime between upgrades -redis: - password: example-redis-password -``` - -Support for the PostHog managed application is provided by the PostHog team. -If you run into issues, -[open a support ticket](https://github.com/PostHog/posthog/issues/new/choose) directly. - -### Install Prometheus using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25138) in GitLab 12.8. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -[Prometheus](https://prometheus.io/docs/introduction/overview/) is an -open-source monitoring and alerting system for supervising your -deployed applications. - -To install Prometheus into the `gitlab-managed-apps` namespace of your cluster, -define the `.gitlab/managed-apps/config.yaml` file with: - -```yaml -prometheus: - installed: true -``` - -You can customize the installation of Prometheus by defining -`.gitlab/managed-apps/prometheus/values.yaml` in your cluster management -project. Refer to the -[Configuration section](https://github.com/helm/charts/tree/master/stable/prometheus#configuration) -of the Prometheus chart's README for the available configuration options. - -Support for installing the Prometheus managed application is provided by the -GitLab Monitor group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the [Monitor group](https://about.gitlab.com/handbook/product/categories/#monitor-group). - -### Install GitLab Runner using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -GitLab Runner is installed using GitLab CI/CD by defining configuration in -`.gitlab/managed-apps/config.yaml`. - -The following configuration is required to install GitLab Runner using GitLab CI/CD: - -```yaml -gitlabRunner: - installed: true -``` - -GitLab Runner is installed into the `gitlab-managed-apps` namespace of your cluster. - -For GitLab Runner to function, you _must_ specify the following: - -- `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) - to register the Runner against. -- `runnerRegistrationToken`: The registration token for adding new runners to GitLab. - This must be [retrieved from your GitLab instance](../../ci/runners/index.md). - -These values can be specified using [CI/CD variables](../../ci/variables/index.md): - -- `GITLAB_RUNNER_GITLAB_URL` is used for `gitlabUrl`. -- `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` - -The methods of specifying these values are mutually exclusive. Either specify variables `GITLAB_RUNNER_REGISTRATION_TOKEN` and `GITLAB_RUNNER_TOKEN` as CI variables (recommended) or provide values for `runnerRegistrationToken:` and `runnerToken:` in `.gitlab/managed-apps/gitlab-runner/values.yaml`. If you choose to use CI variables, comment out or remove `runnerRegistrationToken:` and `runnerToken:` from `.gitlab/managed-apps/gitlab-runner/values`. - -The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../ci/variables/index.md#protect-a-cicd-variable) and [masked variable](../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml` file. - -You can customize the installation of GitLab Runner by defining -`.gitlab/managed-apps/gitlab-runner/values.yaml` file in your cluster -management project. Refer to the -[chart](https://gitlab.com/gitlab-org/charts/gitlab-runner) for the -available configuration options. - -Support for installing the GitLab Runner managed application is provided by the -GitLab Runner group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Runner group](https://about.gitlab.com/handbook/product/categories/#runner-group). - -### Install Cilium using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/22) in GitLab 12.8. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -[Cilium](https://cilium.io/) is a networking plugin for Kubernetes that you can use to implement -support for [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information, see [Network Policies](../../topics/autodevops/stages.md#network-policy). - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see the -[Container Network Security Demo for GitLab 12.8](https://www.youtube.com/watch?v=pgUEdhdhoUI). - -Enable Cilium in the `.gitlab/managed-apps/config.yaml` file to install it: - -```yaml -# possible values are gke or eks -clusterType: gke - -cilium: - installed: true -``` - -The `clusterType` variable enables the recommended Helm variables for a corresponding cluster type. -You can check the recommended variables for each cluster type in the official documentation: - -- [Google GKE](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-gke/#deploy-cilium) -- [AWS EKS](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-eks/#deploy-cilium) - -Do not use `clusterType` for sandbox environments like [minikube](https://minikube.sigs.k8s.io/docs/). - -You can customize Cilium's Helm variables by defining the -`.gitlab/managed-apps/cilium/values.yaml` file in your cluster -management project. Refer to the -[Cilium chart](https://github.com/cilium/cilium/tree/master/install/kubernetes/cilium) -for the available configuration options. - -You can check Cilium's installation status on the cluster management page: - -- [Project-level cluster](../project/clusters/index.md): Navigate to your project's - **Infrastructure > Kubernetes clusters** page. -- [Group-level cluster](../group/clusters/index.md): Navigate to your group's - **Kubernetes** page. - -WARNING: -Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) -of all affected pods in all namespaces to ensure that they are -[managed](https://docs.cilium.io/en/v1.8/operations/troubleshooting/#ensure-managed-pod) -by the correct networking plugin. Whenever Hubble is enabled, its related pod might require a -restart depending on whether it started prior to Cilium. For more information, see -[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) -in the Kubernetes docs. - -NOTE: -Major upgrades might require additional setup steps. For more information, see -the official [upgrade guide](https://docs.cilium.io/en/v1.8/operations/upgrade/). - -By default, Cilium's -[audit mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/#enable-policy-audit-mode) -is enabled. In audit mode, Cilium doesn't drop disallowed packets. You -can use `policy-verdict` log to observe policy-related decisions. You -can disable audit mode by adding the following to -`.gitlab/managed-apps/cilium/values.yaml`: - -```yaml -config: - policyAuditMode: false - -agent: - monitor: - eventTypes: ["drop"] # Note: possible values are documented at https://docs.cilium.io/en/stable/cmdref/cilium_monitor/ -``` - -The Cilium monitor log for traffic is logged out by the -`cilium-monitor` sidecar container. You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -You can disable the monitor log in `.gitlab/managed-apps/cilium/values.yaml`: - -```yaml -agent: - monitor: - enabled: false -``` - -The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default -and it's set to collect per namespace flow metrics. This metrics are accessible on the -[Threat Monitoring](../application_security/threat_monitoring/index.md) -dashboard. You can disable Hubble by adding the following to -`.gitlab/managed-apps/cilium/values.yaml`: - -```yaml -global: - hubble: - enabled: false -``` - -You can also adjust Helm values for Hubble by using -`.gitlab/managed-apps/cilium/values.yaml`: - -```yaml -global: - hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -Support for installing the Cilium managed application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Install Falco using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/91) in GitLab 13.1. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -GitLab Container Host Security Monitoring uses [Falco](https://falco.org/) -as a runtime security tool that listens to the Linux kernel using eBPF. Falco parses system calls -and asserts the stream against a configurable rules engine in real-time. For more information, see -[Falco's Documentation](https://falco.org/docs/). - -You can enable Falco in the -`.gitlab/managed-apps/config.yaml` file: - -```yaml -falco: - installed: true -``` - -You can customize Falco's Helm variables by defining the -`.gitlab/managed-apps/falco/values.yaml` file in your cluster -management project. Refer to the -[Falco chart](https://github.com/falcosecurity/charts/tree/master/falco) -for the available configuration options. - -WARNING: -By default eBPF support is enabled and Falco uses an -[eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) -to pass system calls to user space. If your cluster doesn't support this, you can -configure it to use Falco kernel module instead by adding the following to -`.gitlab/managed-apps/falco/values.yaml`: - -```yaml -ebpf: - enabled: false -``` - -In rare cases where probe installation on your cluster isn't possible and the kernel/probe -isn't pre-compiled, you may need to manually prepare the kernel module or eBPF probe with -[`driverkit`](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) -and install it on each cluster node. - -By default, Falco is deployed with a limited set of rules. To add more rules, add -the following to `.gitlab/managed-apps/falco/values.yaml` (you can get examples from -[Cloud Native Security Hub](https://securityhub.dev/)): - -```yaml -customRules: - file-integrity.yaml: |- - - rule: Detect New File - desc: detect new file created - condition: > - evt.type = chmod or evt.type = fchmod - output: > - File below a known directory opened for writing (user=%user.name - command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) - priority: ERROR - tags: [filesystem] - - rule: Detect New Directory - desc: detect new directory created - condition: > - mkdir - output: > - File below a known directory opened for writing (user=%user.name - command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) - priority: ERROR - tags: [filesystem] -``` - -By default, Falco only outputs security events to logs as JSON objects. To set it to output to an -[external API](https://falco.org/docs/alerts/#https-output-send-alerts-to-an-https-end-point) -or [application](https://falco.org/docs/alerts/#program-output), -add the following to `.gitlab/managed-apps/falco/values.yaml`: - -```yaml -falco: - programOutput: - enabled: true - keepAlive: false - program: mail -s "Falco Notification" someone@example.com - - httpOutput: - enabled: true - url: http://some.url -``` - -You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l app=falco -``` - -Support for installing the Falco managed application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Install Vault using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9982) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -[HashiCorp Vault](https://www.vaultproject.io/) is a secrets management solution which -can be used to safely manage and store passwords, credentials, certificates, and more. A Vault -installation could be leveraged to provide a single secure data store for credentials -used in your applications, GitLab CI/CD jobs, and more. It could also serve as a way of -providing SSL/TLS certificates to systems and deployments in your infrastructure. Leveraging -Vault as a single source for all these credentials allows greater security by having -a single source of access, control, and auditability around all your sensitive -credentials and certificates. This feature requires giving GitLab the highest level of access and -control. Therefore, if GitLab is compromised, the security of this Vault instance is as well. To -avoid this security risk, GitLab recommends using your own HashiCorp Vault to leverage -[external secrets with CI](../../ci/secrets/index.md). - -To install Vault, enable it in the `.gitlab/managed-apps/config.yaml` file: - -```yaml -vault: - installed: true -``` - -By default you receive a basic Vault setup with no scalable storage backend. This -is enough for simple testing and small-scale deployments, though has limits -to how much it can scale, and as it's a single instance deployment, upgrading the -Vault application causes downtime. - -To optimally use Vault in a production environment, it's ideal to have a good understanding -of the internals of Vault and how to configure it. This can be done by reading -the [Vault Configuration guide](../../ci/secrets/#configure-your-vault-server), -the [Vault documentation](https://www.vaultproject.io/docs/internals) and -the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). - -At a minimum, most users set up: - -- A [seal](https://www.vaultproject.io/docs/configuration/seal) for extra encryption - of the main key. -- A [storage backend](https://www.vaultproject.io/docs/configuration/storage) that's - suitable for environment and storage security requirements. -- [HA Mode](https://www.vaultproject.io/docs/concepts/ha). -- The [Vault UI](https://www.vaultproject.io/docs/configuration/ui). - -The following is an example values file (`.gitlab/managed-apps/vault/values.yaml`) -that configures Google Key Management Service for auto-unseal, using a Google Cloud Storage backend, enabling -the Vault UI, and enabling HA with 3 pod replicas. The `storage` and `seal` stanzas -below are examples and should be replaced with settings specific to your environment. - -```yaml -# Enable the Vault WebUI -ui: - enabled: true -server: - # Disable the built in data storage volume as it's not safe for High Availability mode - dataStorage: - enabled: false - # Enable High Availability Mode - ha: - enabled: true - # Configure Vault to listen on port 8200 for normal traffic and port 8201 for inter-cluster traffic - config: | - listener "tcp" { - tls_disable = 1 - address = "[::]:8200" - cluster_address = "[::]:8201" - } - # Configure Vault to store its data in a GCS Bucket backend - storage "gcs" { - path = "gcs://my-vault-storage/vault-bucket" - ha_enabled = "true" - } - # Configure Vault to unseal storage using a GKMS key - seal "gcpckms" { - project = "vault-helm-dev-246514" - region = "global" - key_ring = "vault-helm-unseal-kr" - crypto_key = "vault-helm-unseal-key" - } -``` - -After you have successfully installed Vault, you must -[initialize the Vault](https://learn.hashicorp.com/tutorials/vault/getting-started-deploy#initializing-the-vault) -and obtain the initial root token. You need access to your Kubernetes cluster that -Vault has been deployed into in order to do this. To initialize the Vault, get a -shell to one of the Vault pods running inside Kubernetes (typically this is done -by using the `kubectl` command line tool). After you have a shell into the pod, -run the `vault operator init` command: - -```shell -kubectl -n gitlab-managed-apps exec -it vault-0 sh -/ $ vault operator init -``` - -This should give you your unseal keys and initial root token. Make sure to note these down -and keep these safe, as they're required to unseal the Vault throughout its lifecycle. - -Support for installing the Vault managed application is provided by the -GitLab Release Management group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Release Management group](https://about.gitlab.com/handbook/product/categories/#release-management-group). - -### Install JupyterHub using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/40) in GitLab 12.8. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -JupyterHub is installed using GitLab CI/CD by defining configuration in -`.gitlab/managed-apps/config.yaml` as follows: - -```yaml -jupyterhub: - installed: true - gitlabProjectIdWhitelist: [] - gitlabGroupWhitelist: [] -``` - -In the configuration: - -- `gitlabProjectIdWhitelist` restricts GitLab authentication to only members of the specified projects. -- `gitlabGroupWhitelist` restricts GitLab authentication to only members of the specified groups. -- Specifying an empty array for both allows any user on the GitLab instance to sign in. - -JupyterHub is installed into the `gitlab-managed-apps` namespace of your cluster. - -For JupyterHub to function, you must set up an [OAuth Application](../../integration/oauth_provider.md). -Set: - -- "Redirect URI" to `http://<JupyterHub Host>/hub/oauth_callback`. -- "Scope" to `api read_repository write_repository`. - -In addition, the following variables must be specified using [CI/CD variables](../../ci/variables/index.md): - -- `JUPYTERHUB_PROXY_SECRET_TOKEN` - Secure string used for signing communications - from the hub. Read [`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#proxy-secrettoken). -- `JUPYTERHUB_COOKIE_SECRET` - Secure string used for signing secure cookies. Read - [`hub.cookieSecret`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#hub-cookiesecret). -- `JUPYTERHUB_HOST` - Hostname used for the installation. For example, `jupyter.gitlab.example.com`. -- `JUPYTERHUB_GITLAB_HOST` - Hostname of the GitLab instance used for authentication. - For example, `gitlab.example.com`. -- `JUPYTERHUB_AUTH_CRYPTO_KEY` - A 32-byte encryption key used to set - [`auth.state.cryptoKey`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#auth-state-cryptokey). -- `JUPYTERHUB_AUTH_GITLAB_CLIENT_ID` - "Application ID" for the OAuth Application. -- `JUPYTERHUB_AUTH_GITLAB_CLIENT_SECRET` - "Secret" for the OAuth Application. - -By default, JupyterHub is installed using a -[default values file](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/blob/master/src/default-data/jupyterhub/values.yaml.gotmpl). -You can customize the installation of JupyterHub by defining a -`.gitlab/managed-apps/jupyterhub/values.yaml` file in your cluster management project. - -Refer to the -[chart reference](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html) for the -available configuration options. - -Support for installing the JupyterHub managed application is provided by the GitLab Configure group. -If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). - -### Install Elastic Stack using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25138) in GitLab 12.8. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -Elastic Stack is installed using GitLab CI/CD by defining configuration in -`.gitlab/managed-apps/config.yaml`. - -The following configuration is required to install Elastic Stack using GitLab CI/CD: - -```yaml -elasticStack: - installed: true -``` - -Elastic Stack is installed into the `gitlab-managed-apps` namespace of your cluster. - -You can check the default -[`values.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) -we set for this chart. - -You can customize the installation of Elastic Stack by defining -`.gitlab/managed-apps/elastic-stack/values.yaml` file in your cluster -management project. Refer to the -[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for all -available configuration options. - -Support for installing the Elastic Stack managed application is provided by the -GitLab Monitor group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the [Monitor group](https://about.gitlab.com/handbook/product/categories/#monitor-group). - -### Install Crossplane using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35675) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -Crossplane is installed using GitLab CI/CD by defining configuration in -`.gitlab/managed-apps/config.yaml`. - -The following configuration is required to install Crossplane using GitLab CI/CD: - -```yaml -Crossplane: - installed: true -``` - -Crossplane is installed into the `gitlab-managed-apps` namespace of your cluster. - -You can check the default -[`values.yaml`](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) -we set for this chart. - -You can customize the installation of Crossplane by defining -`.gitlab/managed-apps/crossplane/values.yaml` file in your cluster -management project. Refer to the -[chart](https://github.com/crossplane/crossplane/tree/master/cluster/charts/crossplane#configuration) -for the available configuration options. Note that this link points to the documentation -for the current development release, which may differ from the version you have installed. - -Support for the Crossplane managed application is provided by the Crossplane team. -If you run into issues, -[open a support ticket](https://github.com/crossplane/crossplane/issues/new/choose) directly. - -### Install Fluentd using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/76) in GitLab 12.10. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -To install Fluentd into the `gitlab-managed-apps` namespace of your cluster using -GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: - -```yaml -Fluentd: - installed: true -``` - -You can also review the default values set for this chart in the -[`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. - -You can customize the installation of Fluentd by defining -`.gitlab/managed-apps/fluentd/values.yaml` file in your cluster management -project. Refer to the -[configuration chart](https://github.com/helm/charts/tree/master/stable/fluentd#configuration) -for the current development release of Fluentd for all available configuration options. - -The configuration chart link points to the current development release, which -may differ from the version you have installed. To ensure compatibility, switch -to the specific branch or tag you are using. - -Support for installing the Fluentd managed application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Install Knative using GitLab CI/CD - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -To install Knative, define the `.gitlab/managed-apps/config.yaml` file -with: - -```yaml -knative: - installed: true -``` - -You can customize the installation of Knative by defining `.gitlab/managed-apps/knative/values.yaml` -file in your cluster management project. Refer to the [chart](https://gitlab.com/gitlab-org/charts/knative) -for all available configuration options. - -Here is an example configuration for Knative: - -```yaml -domain: 'my.wildcard.A.record.dns' -``` - -If you plan to use GitLab Serverless capabilities, be sure to set an `A record` -wildcard domain on your custom configuration. - -Support for installing the Knative managed application is provided by the -GitLab Configure group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Configure group](https://about.gitlab.com/handbook/product/categories/#configure-group). - -#### Knative Metrics - -GitLab provides [Invocation Metrics](../project/clusters/serverless/index.md#invocation-metrics) -for your functions. To collect these metrics, you must have: - -1. Knative and Prometheus managed applications installed on your cluster. -1. Manually applied the custom metrics on your cluster by running the following command: - - ```shell - kubectl apply -f https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/raw/02c8231e30ef5b6725e6ba368bc63863ceb3c07d/src/default-data/knative/istio-metrics.yaml - ``` - -#### Uninstall Knative - -To uninstall Knative, you must first manually remove any custom metrics you have added -by running the following command: - -```shell -kubectl delete -f https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/raw/02c8231e30ef5b6725e6ba368bc63863ceb3c07d/src/default-data/knative/istio-metrics.yaml -``` - -### Install AppArmor using GitLab CI/CD - -> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/100) in GitLab 13.1. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. - -To install AppArmor into the `gitlab-managed-apps` namespace of your cluster using -GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: - -```yaml -apparmor: - installed: true -``` - -You can define one or more AppArmor profiles by adding them into -`.gitlab/managed-apps/apparmor/values.yaml` as the following: - -```yaml -profiles: - profile-one: |- - profile profile-one { - file, - } -``` - -Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for more information on this chart. - -#### Using AppArmor profiles in your deployments - -After installing AppAmor, you can use profiles by adding Pod Annotations. If you're using -Auto DevOps, you can [customize `auto-deploy-values.yaml`](../../topics/autodevops/customize.md#customize-values-for-helm-chart) -to annotate your pods. Although it's helpful to be aware of the -[list of custom attributes](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app#gitlabs-auto-deploy-helm-chart), -you're only required to set `podAnnotations` as follows: - -```yaml -podAnnotations: - container.apparmor.security.beta.kubernetes.io/auto-deploy-app: localhost/profile-one -``` - -The only information to be changed here is the profile name which is `profile-one` -in this example. Refer to the -[AppArmor tutorial](https://kubernetes.io/docs/tutorials/security/apparmor/#securing-a-pod) -for more information on how AppArmor is integrated in Kubernetes. - -#### Using PodSecurityPolicy in your deployments - -To enable AppArmor annotations on a Pod Security Policy you must first -load the corresponding AppArmor profile. - -[Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) are -resources at the cluster level that control security-related -properties of deployed pods. You can use such a policy to enable -loaded AppArmor profiles and apply necessary pod restrictions across a -cluster. You can deploy a new policy by adding the following -to`.gitlab/managed-apps/apparmor/values.yaml`: - -```yaml -securityPolicies: - example: - defaultProfile: profile-one - allowedProfiles: - - profile-one - - profile-two - spec: - privileged: false - seLinux: - rule: RunAsAny - supplementalGroups: - rule: RunAsAny - runAsUser: - rule: RunAsAny - fsGroup: - rule: RunAsAny - volumes: - - '*' -``` - -This example creates a single policy named `example` with the provided specification, -and enables [AppArmor annotations](https://kubernetes.io/docs/tutorials/security/apparmor/#podsecuritypolicy-annotations) on it. - -Support for installing the AppArmor managed application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping -at least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -## Install with one click (REMOVED) - -> [Removed](https://gitlab.com/groups/gitlab-org/-/epics/4280) in GitLab 14.0. - -The one-click installation method was deprecated in GitLab 13.9 and removed in [GitLab 14.0](https://gitlab.com/groups/gitlab-org/-/epics/4280). -The removal does not break nor uninstall any apps you have installed, it only -removes the "Applications" tab from the cluster page. -The new recommended way to manage cluster applications is to use the [cluster management project template](management_project_template.md). - -- If you want to migrate your GitLab managed apps management to this template, read - [migrating from GitLab managed apps to project template](migrating_from_gma_to_project_template.md). -- If you don't want to use the template, you can also manually manage your applications. - For that, follow the process to - [take ownership of your GitLab Managed Apps](#take-ownership-of-your-gitlab-managed-apps). - -If you are not yet on GitLab 14.0 or later, you can refer to [an older version of this document](https://docs.gitlab.com/13.12/ee/user/clusters/applications.html#install-with-one-click-deprecated). - -## Browse applications logs - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36769) in GitLab 13.2. - -Logs produced by pods running **GitLab Managed Apps** can be browsed using -[**Log Explorer**](../project/clusters/kubernetes_pod_logs.md). - -## Take ownership of your GitLab Managed Apps - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327803) in GitLab 13.12. - -With the removal of the One-click install method in GitLab 14.0, -the **Applications** tab (under your project's **Infrastructure > Kubernetes clusters**) -is no longer displayed: - - - -This tab was dedicated to installing and maintaining GitLab Managed Apps. -To continue managing your installed applications, one of the possible ways is to -install [Helm](https://helm.sh/) locally, as described below. - -### View installed applications - -To view the applications you have installed in your cluster through GitLab Managed Apps, -you need to verify the resources you have in the `gitlab-managed-apps` namespace. -On your computer, [configure `kubectl`](https://kubernetes.io/docs/reference/kubectl/overview/) -to connect to your cluster, open the terminal and run: - -```shell -kubectl get all -n gitlab-managed-apps -``` - -If there is no output or the namespace does not exist, you do not have any applications -installed through GitLab Managed Apps. If this is the case, you have nothing else to do. - -### Identify the Helm version - -Next, verify which Helm version GitLab used to install your applications. - -#### For apps installed with Helm v3 - -To list your apps installed with Helm v3, run: - -```shell -kubectl get secrets -n gitlab-managed-apps | grep 'helm.sh/release' -``` - -You can manage these applications with Helm v3 and you don't need any further steps. - -All applications not listed with the command above were installed with Helm v2. - -#### For apps installed with Helm v2 - -If you have apps installed with Helm v2, you can either: - -- A. Install Helm v3 and [upgrade your apps to Helm v3](https://helm.sh/docs/topics/v2_v3_migration/). -- B. Install Helm v2 and keep using this Helm version, which is not recommended as Helm v2 was deprecated in favor of -Helm v3. - -If you choose to keep using Helm v2 (B), follow the steps below to manage your apps: - -1. Install [Helm v2](https://v2.helm.sh/docs/install/) in your computer. -1. Start a local Tiller server: - - ```shell - export TILLER_NAMESPACE=gitlab-managed-apps - tiller -listen localhost:44134 - ``` - -1. In another tab, initialize your Helm client: - - ```shell - export HELM_HOST="localhost:44134" - helm init --client-only - ``` - -1. Now your environment is ready to manage your apps with Helm v2. For example, to list your releases: - - ```shell - helm ls - ``` - -### Cluster integrations - -Some applications were not only installed in your cluster by GitLab through -Managed Apps but were also directly integrated with GitLab. If you had one of -these applications installed before GitLab 14.0, then a corresponding [cluster -integration](integrations.md) has been automatically enabled: - -- [Prometheus cluster integration](integrations.md#prometheus-cluster-integration) -- [Elastic Stack cluster integration](integrations.md#elastic-stack-cluster-integration) +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 13.12. +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333610) in GitLab 15.0. +Use the [cluster management project template](management_project_template.md) instead. diff --git a/doc/user/clusters/cost_management.md b/doc/user/clusters/cost_management.md index c99eef385ce..47dc9c42797 100644 --- a/doc/user/clusters/cost_management.md +++ b/doc/user/clusters/cost_management.md @@ -8,10 +8,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216737) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + Cluster cost management provides insights into cluster resource usage. GitLab provides an example [`kubecost-cost-model`](https://gitlab.com/gitlab-examples/kubecost-cost-model/) project that uses the GitLab Prometheus integration and diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index 9e4c672ac45..3f38a473128 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -2,289 +2,12 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../update/removals.md#managed-cluster-applicationsgitlab-ciyml' --- -# Crossplane configuration (DEPRECATED) **(FREE)** +# Crossplane configuration (removed) **(FREE)** -> [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -After [installing](applications.md#install-crossplane-using-gitlab-cicd) Crossplane, you must configure it for use. -The process of configuring Crossplane includes: - -1. [Configure RBAC permissions](#configure-rbac-permissions). -1. [Configure Crossplane with a cloud provider](#configure-crossplane-with-a-cloud-provider). -1. [Configure managed service access](#configure-managed-service-access). -1. [Set up Resource classes](#setting-up-resource-classes). -1. Use [Auto DevOps configuration options](#auto-devops-configuration-options). -1. [Connect to the PostgreSQL instance](#connect-to-the-postgresql-instance). - -To allow Crossplane to provision cloud services such as PostgreSQL, the cloud provider -stack must be configured with a user account. For example: - -- A service account for GCP. -- An IAM user for AWS. - -Some important notes: - -- This guide uses GCP as an example, but the processes for AWS and Azure are similar. -- Crossplane requires the Kubernetes cluster to be VPC native with Alias IPs enabled, - so the IP addresses of the pods can be routed within the GCP network. - -First, declare some environment variables with configuration for use in this guide: - -```shell -export PROJECT_ID=crossplane-playground # the GCP project where all resources reside. -export NETWORK_NAME=default # the GCP network where your GKE is provisioned. -export REGION=us-central1 # the GCP region where the GKE cluster is provisioned. -``` - -## Configure RBAC permissions - -For GitLab-managed clusters, role-based access control (RBAC) is configured automatically. - -For non-GitLab managed clusters, ensure that the service account for the token -provided can manage resources in the `database.crossplane.io` API group: - -1. Save the following YAML as `crossplane-database-role.yaml`: - - ```yaml - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRole - metadata: - name: crossplane-database-role - labels: - rbac.authorization.k8s.io/aggregate-to-edit: "true" - rules: - - apiGroups: - - database.crossplane.io - resources: - - postgresqlinstances - verbs: - - get - - list - - create - - update - - delete - - patch - - watch - ``` - -1. Apply the cluster role to the cluster: - - ```shell - kubectl apply -f crossplane-database-role.yaml - ``` - -## Configure Crossplane with a cloud provider - -See [Configure Your Cloud Provider Account](https://crossplane.github.io/docs/v1.6/) -to configure the installed cloud provider stack with a user account. - -The Secret, and the Provider resource referencing the Secret, must be -applied to the `gitlab-managed-apps` namespace in the guide. Make sure you change that -while following the process. - -## Configure Managed Service Access - -Next, configure connectivity between the PostgreSQL database and the GKE cluster -by either: - -- Using Crossplane as demonstrated below. -- Directly in the GCP console by - [configuring private services access](https://cloud.google.com/vpc/docs/configure-private-services-access). - -1. Run the following command, which creates a `network.yaml` file, and configures - `GlobalAddress` and connection resources: - - ```plaintext - cat > network.yaml <<EOF - --- - # gitlab-ad-globaladdress defines the IP range that will be allocated - # for cloud services connecting to the instances in the given Network. - - apiVersion: compute.gcp.crossplane.io/v1alpha3 - kind: GlobalAddress - metadata: - name: gitlab-ad-globaladdress - spec: - providerRef: - name: gcp-provider - reclaimPolicy: Delete - name: gitlab-ad-globaladdress - purpose: VPC_PEERING - addressType: INTERNAL - prefixLength: 16 - network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - --- - # gitlab-ad-connection is what allows cloud services to use the allocated - # GlobalAddress for communication. Behind the scenes, it creates a VPC peering - # to the network that those service instances actually live. - - apiVersion: servicenetworking.gcp.crossplane.io/v1alpha3 - kind: Connection - metadata: - name: gitlab-ad-connection - spec: - providerRef: - name: gcp-provider - reclaimPolicy: Delete - parent: services/servicenetworking.googleapis.com - network: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - reservedPeeringRangeRefs: - - name: gitlab-ad-globaladdress - EOF - ``` - -1. Apply the settings specified in the file with the following command: - - ```shell - kubectl apply -f network.yaml - ``` - -1. Verify the creation of the network resources, and that both resources are ready and synced. - - ```shell - kubectl describe connection.servicenetworking.gcp.crossplane.io gitlab-ad-connection - kubectl describe globaladdress.compute.gcp.crossplane.io gitlab-ad-globaladdress - ``` - -## Setting up Resource classes - -Use resource classes to define a configuration for the required managed service. -This example defines the PostgreSQL Resource class: - -1. Run the following command, which define a `gcp-postgres-standard.yaml` resource - class containing a default `CloudSQLInstanceClass` with labels: - - ```plaintext - cat > gcp-postgres-standard.yaml <<EOF - apiVersion: database.gcp.crossplane.io/v1beta1 - kind: CloudSQLInstanceClass - metadata: - name: cloudsqlinstancepostgresql-standard - labels: - gitlab-ad-demo: "true" - specTemplate: - writeConnectionSecretsToNamespace: gitlab-managed-apps - forProvider: - databaseVersion: POSTGRES_11_7 - region: $REGION - settings: - tier: db-custom-1-3840 - dataDiskType: PD_SSD - dataDiskSizeGb: 10 - ipConfiguration: - privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - # this should match the name of the provider created in the above step - providerRef: - name: gcp-provider - reclaimPolicy: Delete - --- - apiVersion: database.gcp.crossplane.io/v1beta1 - kind: CloudSQLInstanceClass - metadata: - name: cloudsqlinstancepostgresql-standard-default - annotations: - resourceclass.crossplane.io/is-default-class: "true" - specTemplate: - writeConnectionSecretsToNamespace: gitlab-managed-apps - forProvider: - databaseVersion: POSTGRES_11_7 - region: $REGION - settings: - tier: db-custom-1-3840 - dataDiskType: PD_SSD - dataDiskSizeGb: 10 - ipConfiguration: - privateNetwork: projects/$PROJECT_ID/global/networks/$NETWORK_NAME - # this should match the name of the provider created in the above step - providerRef: - name: gcp-provider - reclaimPolicy: Delete - EOF - ``` - -1. Apply the resource class configuration with the following command: - - ```shell - kubectl apply -f gcp-postgres-standard.yaml - ``` - -1. Verify creation of the Resource class with the following command: - - ```shell - kubectl get cloudsqlinstanceclasses - ``` - -The Resource Classes allow you to define classes of service for a managed service. -We could create another `CloudSQLInstanceClass` which requests for a larger or a -faster disk. It could also request for a specific version of the database. - -## Auto DevOps Configuration Options - -You can run the Auto DevOps pipeline with either of the following options: - -- Setting the Environment variables `AUTO_DEVOPS_POSTGRES_MANAGED` and - `AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR` to provision PostgreSQL using Crossplane. -- Overriding values for the Helm chart: - - Set `postgres.managed` to `true`, which selects a default resource class. - Mark the resource class with the annotation - `resourceclass.crossplane.io/is-default-class: "true"`. The CloudSQLInstanceClass - `cloudsqlinstancepostgresql-standard-default` is used to satisfy the claim. - - Set `postgres.managed` to `true` with `postgres.managedClassSelector` - providing the resource class to choose, based on labels. In this case, the - value of `postgres.managedClassSelector.matchLabels.gitlab-ad-demo="true"` - selects the CloudSQLInstance class `cloudsqlinstancepostgresql-standard` - to satisfy the claim request. - -The Auto DevOps pipeline should provision a PostgresqlInstance when it runs successfully. - -To verify the PostgreSQL instance was created, run this command. When the `STATUS` -field of the PostgresqlInstance changes to `BOUND`, it's successfully provisioned: - -```shell -$ kubectl get postgresqlinstance - -NAME STATUS CLASS-KIND CLASS-NAME RESOURCE-KIND RESOURCE-NAME AGE -staging-test8 Bound CloudSQLInstanceClass cloudsqlinstancepostgresql-standard CloudSQLInstance xp-ad-demo-24-staging-staging-test8-jj55c 9m -``` - -The endpoint of the PostgreSQL instance, and the user credentials, are present in -a secret called `app-postgres` within the same project namespace. You can verify the -secret with the following command: - -```shell -$ kubectl describe secret app-postgres - -Name: app-postgres -Namespace: xp-ad-demo-24-staging -Labels: <none> -Annotations: crossplane.io/propagate-from-name: 108e460e-06c7-11ea-b907-42010a8000bd - crossplane.io/propagate-from-namespace: gitlab-managed-apps - crossplane.io/propagate-from-uid: 10c79605-06c7-11ea-b907-42010a8000bd - -Type: Opaque - -Data -==== -privateIP: 8 bytes -publicIP: 13 bytes -serverCACertificateCert: 1272 bytes -serverCACertificateCertSerialNumber: 1 bytes -serverCACertificateCreateTime: 24 bytes -serverCACertificateExpirationTime: 24 bytes -username: 8 bytes -endpoint: 8 bytes -password: 27 bytes -serverCACertificateCommonName: 98 bytes -serverCACertificateInstance: 41 bytes -serverCACertificateSha1Fingerprint: 40 bytes -``` - -## Connect to the PostgreSQL instance - -Follow this [GCP guide](https://cloud.google.com/sql/docs/postgres/connect-kubernetes-engine) if you -would like to connect to the newly provisioned PostgreSQL database instance on CloudSQL. +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) +in GitLab 14.5. and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/333610) +in GitLab 15.0. Use [crossplane](http://crossplane.io/) directly instead. diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index 71eb08ee640..4ba0de3bf55 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -9,10 +9,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13392) in GitLab 12.3 for group-level clusters. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14809) in GitLab 12.4 for instance-level clusters. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + Cluster environments provide a consolidated view of which CI [environments](../../ci/environments/index.md) are deployed to the Kubernetes cluster and it: diff --git a/doc/user/clusters/img/applications_tab_v13_12.png b/doc/user/clusters/img/applications_tab_v13_12.png Binary files differdeleted file mode 100644 index c4f1a8862e8..00000000000 --- a/doc/user/clusters/img/applications_tab_v13_12.png +++ /dev/null diff --git a/doc/user/clusters/integrations.md b/doc/user/clusters/integrations.md index 74f6ec283ea..a6dbb5fe0d7 100644 --- a/doc/user/clusters/integrations.md +++ b/doc/user/clusters/integrations.md @@ -6,16 +6,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Cluster integrations (DEPRECATED) **(FREE)** -> [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + GitLab provides several ways to integrate applications to your Kubernetes cluster. To enable cluster integrations, first add a Kubernetes cluster to a GitLab -[project](../project/clusters/add_remove_clusters.md) or +[project](../project/clusters/index.md) or [group](../group/clusters/index.md) or [instance](../instance/clusters/index.md). @@ -97,9 +101,21 @@ To enable the Prometheus integration for your cluster: 1. Click **Save changes**. 1. Go to the **Health** tab to see your cluster's metrics. -## Elastic Stack cluster integration +## Elastic Stack cluster integration **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61077) in GitLab 13.12. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 14.7. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/360182) behind a [feature flag](../../administration/feature_flags.md) named `monitor_logging` in GitLab 15.0. Disabled by default. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61077) in GitLab 13.12. +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) +in GitLab 14.7. +It will be removed completely in GitLab 15.2. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `monitor_logging`. +On GitLab.com, this feature is not available. +This feature is not recommended for production use. You can integrate your cluster with [Elastic Stack](https://www.elastic.co/elastic-stack/) to index and [query your pod diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 7658bc41dd9..d59edb1e2c2 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -8,12 +8,16 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32810) in GitLab 12.5. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: The cluster management project was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To manage cluster applications, use the [GitLab agent](agent/index.md) with the [Cluster Management Project Template](management_project_template.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + A project can be designated as the management project for a cluster. A management project can be used to run deployment jobs with Kubernetes diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index ab17e462c6a..8ca1bf5d57f 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -32,7 +32,7 @@ If you have already configured the agent and connected a cluster with GitLab: 1. [Create a project from the cluster management project template](#create-a-project-based-on-the-cluster-management-project-template). 1. In the project where you configured your agent, - [grant the agent access to the new project](agent/ci_cd_tunnel.md#authorize-the-agent). + [grant the agent access to the new project](agent/ci_cd_workflow.md#authorize-the-agent). 1. In the new project, create an [environment variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project) named `$KUBE_CONTEXT` and set the value to `path/to/agent-configuration-project:your-agent-name`. @@ -78,7 +78,7 @@ This image contains a set of Bash utility scripts to support [Helm v3 releases]( The template contains a [Helmfile](https://github.com/roboll/helmfile) you can use to manage cluster applications with [Helm v3](https://helm.sh/). -This file has a list of paths to other Helmfiles for each app. They're all commented out by default, so you must uncomment +This file has a list of paths to other Helm files for each app. They're all commented out by default, so you must uncomment the paths for the apps that you would like to use in your cluster. By default, each `helmfile.yaml` in these sub-paths has the attribute `installed: true`. This means that every time @@ -93,12 +93,7 @@ application in the template. The [built-in supported applications](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/tree/master/applications) are: -- [Apparmor](../infrastructure/clusters/manage/management_project_applications/apparmor.md) - [Cert-manager](../infrastructure/clusters/manage/management_project_applications/certmanager.md) -- [Cilium](../infrastructure/clusters/manage/management_project_applications/cilium.md) -- [Elastic Stack](../infrastructure/clusters/manage/management_project_applications/elasticstack.md) -- [Falco](../infrastructure/clusters/manage/management_project_applications/falco.md) -- [Fluentd](../infrastructure/clusters/manage/management_project_applications/fluentd.md) - [GitLab Runner](../infrastructure/clusters/manage/management_project_applications/runner.md) - [Ingress](../infrastructure/clusters/manage/management_project_applications/ingress.md) - [Prometheus](../infrastructure/clusters/manage/management_project_applications/prometheus.md) diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index 09453262fbb..9a59d135fa0 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -6,8 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Migrate from GitLab Managed Apps to Cluster Management Projects **(FREE)** -The [GitLab Managed Apps](applications.md) were deprecated in GitLab 14.0 -in favor of [Cluster Management Projects](management_project.md). +The GitLab Managed Apps were deprecated in GitLab 14.0 +in favor of user-controlled Cluster Management projects. Managing your cluster applications through a project enables you a lot more flexibility to manage your cluster than through the late GitLab Managed Apps. To migrate to the cluster management project you need @@ -21,8 +21,10 @@ follow the steps below. See also [video walk-throughs](#video-walk-throughs) with examples. 1. Create a new project based on the [Cluster Management Project template](management_project_template.md#create-a-project-based-on-the-cluster-management-project-template). -1. [Associate your new Cluster Management Project with your cluster](management_project.md#associate-the-cluster-management-project-with-the-cluster). +1. [Install an agent](agent/install/index.md) for this project in your cluster. +1. Set the `KUBE_CONTEXT` CI/CD variable to the newly installed agent's context, as instructed in the `.gitlab-ci.yml` from the Project Template. 1. Detect apps deployed through Helm v2 releases by using the pre-configured [`.gitlab-ci.yml`](management_project_template.md#the-gitlab-ciyml-file) file: + - In case you had overwritten the default GitLab Managed Apps namespace, edit `.gitlab-ci.yml`, and make sure the script is receiving the correct namespace as an argument: @@ -92,6 +94,7 @@ See also [video walk-throughs](#video-walk-throughs) with examples. chart version proposed in `applications/vault/values.yaml`. - Cert-manager: + - For users on Kubernetes version 1.20 or above, the deprecated cert-manager v0.10 is no longer valid and the upgrade includes a breaking change. So we suggest that you [backup and uninstall cert-manager v0.10](#backup-and-uninstall-cert-manager-v010), and install the latest cert-manager instead. To install this version, uncomment `applications/cert-manager/helmfile.yaml` diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index a2172b72572..659c0326728 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -134,7 +134,7 @@ License Compliance can be configured using CI/CD variables. | `ASDF_PYTHON_VERSION` | no | Version of Python to use for the scan. [Configuration](#selecting-the-version-of-python) | | `ASDF_RUBY_VERSION` | no | Version of Ruby to use for the scan. | | `GRADLE_CLI_OPTS` | no | Additional arguments for the Gradle executable. If not supplied, defaults to `--exclude-task=test`. | -| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if you have multiple projects in nested directories, you can update your `.gitlab-ci-yml` template to specify a recursive scan, like `LICENSE_FINDER_CLI_OPTS: '--recursive'`. | +| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if you have multiple projects in nested directories, you can update your `.gitlab-ci.yml` template to specify a recursive scan, like `LICENSE_FINDER_CLI_OPTS: '--recursive'`. | | `LM_JAVA_VERSION` | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. [Configuration](#selecting-the-version-of-java) | | `LM_PYTHON_VERSION` | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. [Configuration](#selecting-the-version-of-python) | | `MAVEN_CLI_OPTS` | no | Additional arguments for the `mvn` executable. If not supplied, defaults to `-DskipTests`. | @@ -506,7 +506,7 @@ example: } ``` -If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/index.md#protect-a-cicd-variable) +If credentials are required to authenticate then you can configure a [protected CI/CD variable](../../../ci/variables/index.md#protected-cicd-variables) following the naming convention described in the [`CONAN_LOGIN_USERNAME` documentation](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name). #### Custom root certificates for Conan @@ -853,7 +853,7 @@ A full list of variables can be found in [CI/CD variables](#available-cicd-varia To find out what tools are pre-installed in the `license_scanning` Docker image use the following command: ```shell -$ docker run --entrypoint='' registry.gitlab.com/security-products/license-finder:3 /bin/bash -lc 'asdf list' +$ docker run --entrypoint='' registry.gitlab.com/security-products/license-finder:4 /bin/bash -lc 'asdf list' golang 1.14 gradle @@ -880,7 +880,7 @@ sbt To interact with the `license_scanning` runtime environment use the following command: ```shell -$ docker run -it --entrypoint='' registry.gitlab.com/security-products/license-finder:3 /bin/bash -l +$ docker run -it --entrypoint='' registry.gitlab.com/security-products/license-finder:4 /bin/bash -l root@6abb70e9f193:~# ``` diff --git a/doc/user/crm/index.md b/doc/user/crm/index.md index 7fc11add2cd..7a400205e30 100644 --- a/doc/user/crm/index.md +++ b/doc/user/crm/index.md @@ -6,12 +6,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Customer relations management (CRM) **(FREE)** -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `customer_relations`. -On GitLab.com, this feature is not available. - > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2256) in GitLab 14.6 [with a flag](../../administration/feature_flags.md) named `customer_relations`. Disabled by default. > - In GitLab 14.8 and later, you can [create contacts and organizations only in root groups](https://gitlab.com/gitlab-org/gitlab/-/issues/350634). +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/346082) in GitLab 15.0. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `customer_relations`. +On GitLab.com, this feature is available. With customer relations management (CRM) you can create a record of contacts (individuals) and organizations (companies) and relate them to issues. @@ -118,7 +119,7 @@ organizations using the GraphQL API. ### View issues linked to a contact -To view a contact's issues: +To view a contact's issues, select a contact from the issue sidebar, or: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Customer relations > Contacts**. @@ -166,11 +167,12 @@ API. ## Autocomplete contacts **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2256) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `contacts_autocomplete`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2256) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `contacts_autocomplete`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/352123) in GitLab 15.0. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `contacts_autocomplete`. -On GitLab.com, this feature is not available. +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `contacts_autocomplete`. +On GitLab.com, this feature is available. This feature is not ready for production use. When you use the `/add_contacts` or `/remove_contacts` quick actions, follow them with `[contact:` and an autocomplete list appears: diff --git a/doc/user/discussions/img/add_internal_note_v15_0.png b/doc/user/discussions/img/add_internal_note_v15_0.png Binary files differnew file mode 100644 index 00000000000..cf052edd5e7 --- /dev/null +++ b/doc/user/discussions/img/add_internal_note_v15_0.png diff --git a/doc/user/discussions/img/confidential_comments_v13_9.png b/doc/user/discussions/img/confidential_comments_v13_9.png Binary files differdeleted file mode 100644 index b5be5a622a9..00000000000 --- a/doc/user/discussions/img/confidential_comments_v13_9.png +++ /dev/null diff --git a/doc/user/discussions/img/create-new-issue_v14_3.png b/doc/user/discussions/img/create-new-issue_v14_3.png Binary files differdeleted file mode 100644 index d76fed6cb42..00000000000 --- a/doc/user/discussions/img/create-new-issue_v14_3.png +++ /dev/null diff --git a/doc/user/discussions/img/create-new-issue_v15.png b/doc/user/discussions/img/create-new-issue_v15.png Binary files differnew file mode 100644 index 00000000000..779196b6ba4 --- /dev/null +++ b/doc/user/discussions/img/create-new-issue_v15.png diff --git a/doc/user/discussions/img/unresolved_threads_v14_1.png b/doc/user/discussions/img/unresolved_threads_v14_1.png Binary files differdeleted file mode 100644 index 806dfdc995c..00000000000 --- a/doc/user/discussions/img/unresolved_threads_v14_1.png +++ /dev/null diff --git a/doc/user/discussions/img/unresolved_threads_v15.png b/doc/user/discussions/img/unresolved_threads_v15.png Binary files differnew file mode 100644 index 00000000000..113af20effc --- /dev/null +++ b/doc/user/discussions/img/unresolved_threads_v15.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 8537856ef25..97c3d5f1058 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -150,42 +150,45 @@ Notes are added to the page details. If an issue or merge request is locked and closed, you cannot reopen it. -## Mark a comment as confidential **(FREE SELF)** +## Add an internal note > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207473) in GitLab 13.9 [with a flag](../../administration/feature_flags.md) named `confidential_notes`. Disabled by default. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/351143) in GitLab 14.10: you can only mark comments in issues and epics as confidential. Previously, it was also possible for comments in merge requests and snippets. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87403) from "confidential comments" to "internal notes" in GitLab 15.0. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87383) in GitLab 15.0. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `confidential_notes`. -On GitLab.com, this feature is not available. -You should not use this feature for production environments. +On self-managed GitLab, by default this feature is available. To hide the feature, +ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `confidential_notes`. +On GitLab.com, this feature is available. -You can make a comment **in an issue or an epic** confidential, so that it is visible only to you (the commenting user) and -the project members who have at least the Reporter role. +You can add an internal note **to an issue or an epic**. It's then visible only to the following people: + +- Project members who have at least the Reporter role +- Issue or epic author +- Users assigned to the issue or epic Keep in mind: -- You can only mark comments as confidential when you create them. -- You can't change the confidentiality of existing comments. -- Replies to comments use same confidentiality as the original comment. +- Replies to internal notes are also internal. +- You can not turn an internal note into a regular comment. Prerequisites: - You must either: - Have at least the Reporter role for the project. - - Be the issue assignee. - - Be the issue author. + - Be the issue or epic assignee. + - Be the issue or epic author. -To mark a comment as confidential: +To add an internal note: 1. Start adding a new comment. -1. Below the comment, select the **Make this comment confidential** checkbox. -1. Select **Comment**. +1. Below the comment, select the **Make this an internal note** checkbox. +1. Select **Add internal note**. - + -You can also make an [entire issue confidential](../project/issues/confidential_issues.md). +You can also mark an [issue as confidential](../project/issues/confidential_issues.md). ## Show only comments @@ -300,7 +303,7 @@ To resolve a thread: At the top of the page, the number of unresolved threads is updated: - + ### Move all unresolved threads in a merge request to an issue @@ -308,7 +311,7 @@ If you have multiple unresolved threads in a merge request, you can create an issue to resolve them separately. In the merge request, at the top of the page, select **Create issue to resolve all threads** (**{issue-new}**): - + All threads are marked as resolved, and a link is added from the merge request to the newly created issue. @@ -327,12 +330,13 @@ the newly created issue. ### Prevent merge unless all threads are resolved You can prevent merge requests from being merged until all threads are -resolved. +resolved. When this setting is enabled, the **Unresolved threads** counter in a merge request +is shown in orange when at least one thread remains unresolved. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Merge requests**. -1. Under **Merge checks**, select the **All discussions must be resolved** checkbox. +1. Under **Merge checks**, select the **All threads must be resolved** checkbox. 1. Select **Save changes**. ### Automatically resolve threads in a merge request when they become outdated @@ -344,7 +348,7 @@ with a new push. 1. On the left sidebar, select **Settings > General**. 1. Expand **Merge requests**. 1. Under **Merge options**, select the - **Automatically resolve merge request diff discussions when they become outdated** checkbox. + **Automatically resolve merge request diff threads when they become outdated** checkbox. 1. Select **Save changes**. Threads are now resolved if a push makes a diff section outdated. diff --git a/doc/user/free_user_limit.md b/doc/user/free_user_limit.md new file mode 100644 index 00000000000..2340ef254f6 --- /dev/null +++ b/doc/user/free_user_limit.md @@ -0,0 +1,60 @@ +--- +stage: Growth +group: Conversion +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Free user limit **(FREE SAAS)** + +In GitLab 15.1 (June 22, 2022) and later, namespaces in GitLab.com on the Free tier +will be limited to five (5) members per [namespace](group/index.md#namespaces). +This limit applies to top-level groups and personal namespaces. + +In a personal namespace, the limit applies across all projects in your personal +namespace. + +On the transition date, if your namespace has six or more unique members: + +- Five members will keep a status of `Active`. +- Remaining members will get a status of `Over limit` and lose access to the + group. +- Members invited through a group or project invitation outside of the namespace + will be removed. You can add these members back by inviting them through their + username or email address on the **Members** page for your group or project. + +## How active members are determined + +On the transition date, we'll automatically select the members who keep their `Active` status +in the following order, until we reach a total of five: + +1. Members with the Owner or Maintainer role. +1. The most recently active members. + +## Manage members in your namespace + +To help manage your free user limit, +you can view and manage the total number of members across all projects and groups +in your namespace. + +Prerequisite: + +- You must have the Owner role for the group. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Usage Quotas**. +1. To view all members, select the **Seats** tab. +1. To remove a member, select **Remove user**. + +NOTE: +The **Usage Quotas** page is not available for personal namespaces. You can +view and [remove members](project/members/index.md#remove-a-member-from-a-project) +in each project instead. The five user limit includes all +unique members across all projects in your personal namespace. + +If you need more time to manage your members, or to try GitLab features +with a team of more than five members, you can [start a trial](https://about.gitlab.com/free-trial/). +A trial lasts for 30 days and includes an unlimited number of members. + +## Related topics + +- [GitLab SaaS Free tier frequently asked questions](https://about.gitlab.com/pricing/faq-efficient-free-tier/) diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 6bb45575fc3..e88777a7223 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -9,6 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w This page contains information about the settings that are used on GitLab.com, available to [GitLab SaaS](https://about.gitlab.com/pricing/) customers. +See some of these settings on the [instance configuration page](https://gitlab.com/help/instance_configuration) of GitLab.com. + ## Password requirements GitLab.com has the following requirements for passwords on new accounts and password changes: @@ -97,6 +99,10 @@ If you are on: - Premium tier and above, you can disable this by changing the [group setting](../group/index.md#enable-delayed-project-deletion). - Free tier, you cannot disable this setting or restore projects. +## Inactive project deletion + +[Inactive project deletion](../../administration/inactive_project_deletion.md) is disabled on GitLab.com. + ## Alternative SSH port GitLab.com can be reached by using a [different SSH port](https://about.gitlab.com/blog/2016/02/18/gitlab-dot-com-now-supports-an-alternate-git-plus-ssh-port/) for `git+ssh`. @@ -127,9 +133,9 @@ Below are the settings for [GitLab Pages](https://about.gitlab.com/stages-devops | IP address | `35.185.44.232` | - | | Custom domains support | **{check-circle}** Yes | **{dotted-circle}** No | | TLS certificates support | **{check-circle}** Yes | **{dotted-circle}** No | -| Maximum size (compressed) | 1 GB | 100 MB | +| [Maximum size](../../administration/pages/index.md#set-global-maximum-pages-size-per-project) (compressed) | 1 GB | 100 MB | -The maximum size of your Pages site is regulated by the artifacts maximum size, +The maximum size of your Pages site is also regulated by the artifacts maximum size, which is part of [GitLab CI/CD](#gitlab-cicd). There are also [rate limits set for GitLab Pages](#gitlabcom-specific-rate-limits). @@ -330,7 +336,7 @@ after the limits change in January, 2021: | **Authenticated** API traffic (for a given **user**) | **2,000** requests per minute | **2,000** requests per minute | | **Authenticated** non-API HTTP traffic (for a given **user**) | **1,000** requests per minute | **1,000** requests per minute | | **All** traffic (from a given **IP address**) | **2,000** requests per minute | **2,000** requests per minute | -| **Issue creation** | **300** requests per minute | **300** requests per minute | +| **Issue creation** | **300** requests per minute | **200** requests per minute | | **Note creation** (on issues and merge requests) | **60** requests per minute | **60** requests per minute | | **Advanced, project, and group search** API (for a given **IP address**) | **10** requests per minute | **10** requests per minute | | **GitLab Pages** requests (for a given **IP address**) | | **1000** requests per **50 seconds** | diff --git a/doc/user/group/epics/linked_epics.md b/doc/user/group/epics/linked_epics.md index b695bae39e4..89699661efa 100644 --- a/doc/user/group/epics/linked_epics.md +++ b/doc/user/group/epics/linked_epics.md @@ -6,12 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Linked epics **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353473) in GitLab 14.9 [with a flag](../../../administration/feature_flags.md) named `related_epics_widget`. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) -named `related_epics_widget`. On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353473) in GitLab 14.9 [with a flag](../../../administration/feature_flags.md) named `related_epics_widget`. Enabled by default. +> - [Feature flag `related_epics_widget`](https://gitlab.com/gitlab-org/gitlab/-/issues/357089) removed in GitLab 15.0. Linked epics are a bi-directional relationship between any two epics and appear in a block below the epic description. You can link epics in different groups. @@ -38,7 +34,11 @@ To link one epic to another: - **relates to** - **[blocks](#blocking-epics)** - **[is blocked by](#blocking-epics)** -1. Enter the epic number or paste in the full URL of the epic. +1. To enter the linked epic, either: + + - Enter `&`, followed by the epic's number. For example, `&123`. + - Enter `&`, followed by a word from the epic's title. For example, `&Deliver`. + - Paste in the epic's full URL.  diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index 0185cb9cfdf..6967815bf22 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -132,6 +132,8 @@ migrated: - image URL - Boards - Board Lists +- Releases + - Milestones ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) in GitLab 15.0). ## Troubleshooting Group Migration diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 085cd054c14..87146329031 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -53,7 +53,7 @@ For example, consider a user named Alex: | GitLab URL | Namespace | | ---------- | --------- | | Alex creates an account with the username `alex`: `https://gitlab.example.com/alex`. | The namespace in this case is `alex`. | -| Alex creates a group for their team with the group name `alex-team`. The group and its projects are available at: `https://gitlab.example.com/alex-team`. | The namespace in this cases is `alex-team`. | +| Alex creates a group for their team with the group name `alex-team`. The group and its projects are available at: `https://gitlab.example.com/alex-team`. | The namespace in this case is `alex-team`. | | Alex creates a subgroup of `alex-team` with the subgroup name `marketing`. The subgroup and its projects are available at: `https://gitlab.example.com/alex-team/marketing`. | The namespace in this case is `alex-team/marketing`. | ## Create a group @@ -279,7 +279,7 @@ To view the activity feed in Atom format, select the Similar to how you [share a project with a group](../project/members/share_project_with_groups.md), you can share a group with another group. To invite a group, you must be a member of it. Members get direct access -to the shared group. This includes members who inherited group membership from a parent group. +to the shared group. This includes members who inherited group membership from a parent group. To share a given group, for example, `Frontend` with another group, for example, `Engineering`: @@ -456,25 +456,28 @@ To restore a group that is marked for deletion: ## Prevent group sharing outside the group hierarchy -This setting is only available on top-level groups. It affects all subgroups. +You can configure a top-level group so its subgroups and projects +cannot invite other groups outside of the top-level group's hierarchy. +This option is only available for top-level groups. -When checked, any group in the top-level group hierarchy can be shared only with other groups in the hierarchy. +For example, in the following group and project hierarchy: -For example, with these groups: - -- **Animals > Dogs** +- **Animals > Dogs > Dog Project** - **Animals > Cats** - **Plants > Trees** -If you select this setting in the **Animals** group: +If you prevent group sharing outside the hierarchy for the **Animals** group: -- **Dogs** can be shared with **Cats**. -- **Dogs** cannot be shared with **Trees**. +- **Dogs** can invite the group **Cats**. +- **Dogs** cannot invite the group **Trees**. +- **Dog Project** can invite the group **Cats**. +- **Dog Project** cannot invite the group **Trees**. To prevent sharing outside of the group's hierarchy: -1. Go to the group's **Settings > General** page. -1. Expand the **Permissions and group features** section. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand **Permissions and group features**. 1. Select **Prevent members from sending invitations to groups outside of `<group_name>` and its subgroups**. 1. Select **Save changes**. @@ -610,15 +613,16 @@ applies to: You should consider these security implications before configuring IP address restrictions: -- **SSH requests**: While you can restrict HTTP traffic on GitLab.com with IP address restrictions, +- **SSH requests, including `git` operations will fail from all IP addresses**: While you can restrict HTTP traffic on GitLab.com with IP address restrictions, they cause SSH requests, including Git operations over SSH, to fail. For more information, read [issue 271673](https://gitlab.com/gitlab-org/gitlab/-/issues/271673). -- **Administrators and group owners**: Users with these permission levels can always +- **Administrators and group owners can access group settings from any IP address**: Users with these permission levels can always access the group settings, regardless of IP restriction, but they cannot access projects belonging to the group when accessing from a disallowed IP address. -- **GitLab API and runner activities**: Only the [group](../../api/groups.md) (including all +- **Some GitLab API endpoints will remain accessible from any IP**: Only the [group](../../api/groups.md) (including all [group resources](../../api/api_resources.md#group-resources)) APIs and [project](../../api/api_resources.md#project-resources) (including all [project resources](../../api/api_resources.md#project-resources)) APIs are protected by IP address restrictions. +- **Activities performed by GitLab Runners are not bound by IP restrictions**: When you register a runner, it is not bound by the IP restrictions. When the runner requests a new job or an update to a job's state, it is also not bound by the IP restrictions. But when the running CI/CD job sends Git requests from a diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 1c316f2157d..858b13b87b8 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -41,7 +41,8 @@ From there you can create a new iteration or select an iteration to get a more d ## Create an iteration -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) in GitLab 14.10. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) in GitLab 14.10. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. WARNING: Manual iteration management is in its end-of-life process. Creating an iteration is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) @@ -49,7 +50,7 @@ in GitLab 14.10, and is planned for removal in GitLab 16.0. Prerequisites: -- You must have at least the Developer role for a group. +- You must have at least the Reporter role for a group. To create an iteration: @@ -63,6 +64,7 @@ To create an iteration: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218277) in GitLab 13.2. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) in GitLab 14.10. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. WARNING: Editing all attributes, with the exception of `description` is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) @@ -71,7 +73,7 @@ In the future only editing an iteration's `description` will be allowed. Prerequisites: -- You must have at least the Developer role for a group. +- You must have at least the Reporter role for a group. To edit an iteration, select the three-dot menu (**{ellipsis_v}**) > **Edit**. @@ -79,6 +81,7 @@ To edit an iteration, select the three-dot menu (**{ellipsis_v}**) > **Edit**. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292268) in GitLab 14.3. > - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) in GitLab 14.10. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. WARNING: Manual iteration management is in its end-of-life process. Deleting an iteration is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/356069) @@ -86,7 +89,7 @@ in GitLab 14.10, and is planned for removal in GitLab 16.0. Prerequisites: -- You must have at least the Developer role for a group. +- You must have at least the Reporter role for a group. To delete an iteration, select the three-dot menu (**{ellipsis_v}**) > **Delete**. @@ -169,37 +172,65 @@ To group issues by label: ## Iteration cadences -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5077) in GitLab 14.1. -> - Deployed behind a [feature flag](../../feature_flags.md), named `iteration_cadences`, disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an -administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`iteration_cadences` for a root group. -On GitLab.com, this feature is not available. This feature is not ready for production use. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5077) in GitLab 14.1 [with a flag](../../../administration/feature_flags.md), named `iteration_cadences`. Disabled by default. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/354977) in GitLab 15.0: All scheduled iterations must start on the same day of the week as the cadence start day. Start date of cadence cannot be edited after the first iteration starts. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/354878) in GitLab 15.0. Iteration cadences automate iteration scheduling. You can use them to -automate creating iterations every 1, 2, 3, 4, or 6 weeks. You can also +automate creating iterations every 1, 2, 3, or 4 weeks. You can also configure iteration cadences to automatically roll over incomplete issues to the next iteration. ### Create an iteration cadence +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. + Prerequisites: -- You must have at least the Developer role for a group. +- You must have at least the Reporter role for a group. To create an iteration cadence: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. Select **New iteration cadence**. -1. Fill out required fields, and select **Create iteration cadence**. The cadence list page opens. +1. Complete the fields. + - Enter the title and description of the iteration cadence. + - Enter the first iteration start date of the iteration cadence. Iterations will be scheduled to + begin on the same day of the week as the day of the week of the start date. + - From the **Duration** dropdown list, select how many weeks each iteration should last. + - From the **Upcoming iterations** dropdown list, select how many upcoming iterations should be + created and maintained by GitLab. + - Optional. To move incomplete issues to the next iteration, select **Roll over issues**. +1. Select **Create cadence**. The cadence list page opens. + +### Edit an iteration cadence + +Prerequisites: + +- You must have at least the Developer role for a group. + +To edit an iteration cadence: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Issues > Iterations**. +1. Select **Edit iteration cadence**. + +When you edit the **Duration**, **Upcoming iterations**, or **First iteration start date** fields, +only upcoming iterations are affected. + +You can edit the first iteration start date of a cadence if the cadence has not started yet. + +Editing **Upcoming iterations** is a non-destructive action. +If ten upcoming iterations already exist, changing the number under **Upcoming iterations** to `2` +doesn't delete the eight existing upcoming iterations. ### Delete an iteration cadence +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. + Prerequisites: -- You must have at least the Developer role for a group. +- You must have at least the Reporter role for a group. Deleting an iteration cadence also deletes all iterations within that cadence. @@ -210,18 +241,67 @@ To delete an iteration cadence: 1. Select the three-dot menu (**{ellipsis_v}**) > **Delete cadence** for the cadence you want to delete. 1. Select **Delete cadence** in the confirmation modal. -### Convert manual cadence to use automatic scheduling +### Manual iteration cadences + +When you **enable** the iteration cadences feature, all previously +created iterations are added to a default iteration cadence. +You can continue to add, edit, and remove iterations in +this default cadence. + +#### Convert a manual cadence to use automatic scheduling WARNING: -The upgrade is irreversible. After it's done, manual iteration cadences cannot be created. +The upgrade is irreversible. After it's done, a new manual iteration cadence cannot be created. -When you **enable** the iteration cadences feature, all iterations are added -to a default iteration cadence. -In this default iteration cadence, you can continue to add, edit, and remove iterations. +Prerequisites: +- You must have created [iterations](#iterations) without cadences before enabling iteration cadences for your group. To upgrade the iteration cadence to use the automation features: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Issues > Iterations**. 1. Select the three-dot menu (**{ellipsis_v}**) > **Edit cadence** for the cadence you want to upgrade. -1. Fill out required fields, and select **Save changes**. +1. Complete the required fields **Duration** and **Upcoming iterations**. +1. Select **Save changes**. + +#### Start dates of converted cadences + +The first iteration start date of your converted cadence is set to the start date of its +**first** existing iteration. + +If you attempt to set a new start date, the conversion fails with an error message. +If your manual cadence is empty, converting it to use automatic scheduling is effectively +the same as creating a new automated cadence. + +GitLab will start scheduling new iterations on the same day of the week as the start date, +starting from the nearest such day from the current date. + +During the conversion process GitLab does not delete or modify existing **ongoing** or +**closed** iterations. If you have iterations with start dates in the future, +they are updated to fit your cadence settings. + +#### Converted cadences example + +For example, suppose it's Friday, April 15, and you have three iterations in a manual cadence: + +- Monday, April 4 - Friday, April 8 (closed) +- Tuesday, April 12 - Friday, April 15 (ongoing) +- Tuesday, May 3 - Friday, May 6 (upcoming) + +On Friday, April 15, you convert the manual cadence +to automate scheduling iterations every week, up to two upcoming iterations. + +The first iteration is closed, and the second iteration is ongoing, +so they aren't deleted or modified in the conversion process. + +To observe the weekly duration, the third iteration is updated so that it: + +- Starts on Monday, April 18 - which is the nearest date that is Monday. +- Ends on Sunday, April 24. + +Finally, to always have two upcoming iterations, an additional iteration is scheduled: + +- Monday, April 4 - Friday, April 8 (closed) +- Tuesday, April 12 - Friday, April 15 (ongoing) +- Monday, April 18 - Sunday, April 24 (upcoming) +- Monday, April 25 - Sunday, May 1 (upcoming) diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index 89c5c6ed466..10ca6d139ad 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -1,5 +1,4 @@ --- -type: reference stage: Plan group: Product Planning info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments @@ -7,7 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Roadmap **(PREMIUM)** -> - Introduced in GitLab 10.5. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/198062) from GitLab Ultimate to GitLab Premium in 12.9. > - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/5164) and later, the epic bars show epics' title, progress, and completed weight percentage. > - Milestones appear in roadmaps in [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/6802), and later. @@ -27,10 +25,10 @@ You can expand epics that contain child epics to show their child epics in the r You can select the chevron (**{chevron-down}**) next to the epic title to expand and collapse the child epics. -On top of the milestone bars, you can see their title. -When you hover over a milestone bar or title, a popover appears with its title, start date, and due -date. You can also select the chevron (**{chevron-down}**) next to the **Milestones** heading to -toggle the list of the milestone bars. +On top of the milestone bars, you can see their title. When you point to a +milestone bar or title, a popover appears with its title, start date, and due +date. You can also select the chevron (**{chevron-down}**) next to the **Milestones** +heading to toggle the list of the milestone bars.  @@ -41,8 +39,8 @@ toggle the list of the milestone bars. > - Filtering by epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218623) in GitLab 13.11. > - Filtering by milestone [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323917) in GitLab 14.5. -WARNING: -Filtering roadmaps by milestone might not be available to you. Check the **version history** note above for details. +NOTE: +Filtering roadmaps by milestone might not be available to you. Be sure to review this section's version history for details. When you want to explore a roadmap, there are several ways to make it easier by sorting epics or filtering them by what's important for you. @@ -52,8 +50,9 @@ You can sort epics in the Roadmap view by: - Start date - Due date -Each option contains a button that toggles the sort order between **ascending** and **descending**. -The sort option and order persist when browsing Epics, including the [epics list view](../epics/index.md). +Each option contains a button that toggles the sort order between **ascending** +and **descending**. The sort option and order persist when browsing Epics, including +the [epics list view](../epics/index.md). You can also filter epics in the Roadmap view by the epics': @@ -66,7 +65,7 @@ You can also filter epics in the Roadmap view by the epics':  -Roadmaps can also be [visualized inside an epic](../epics/index.md#roadmap-in-epics). +You can also [visualize roadmaps inside of an epic](../epics/index.md#roadmap-in-epics). ### Roadmap settings @@ -78,38 +77,40 @@ When you enable the roadmap settings sidebar, you can use it to refine epics sho You can configure the following: - Select date range. -- Turn milestones on or off and select whether to show all, group, subgroup, or project milestones. +- Turn milestones on or off, and select whether to show all, group, subgroup, or + project milestones. - Show all, open, or closed epics. - Turn progress tracking for child issues on or off and select whether to use issue weights or counts. - The progress tracking setting is not saved in user preferences but is saved or shared using URL parameters. +The progress tracking setting isn't saved in user preferences, but is saved or +shared using URL parameters. ## Timeline duration -> - Introduced in GitLab 11.0. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/198062) from GitLab Ultimate to GitLab Premium in 12.9. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/198062) from GitLab Ultimate to GitLab Premium in 12.9. ### Date range presets -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/204994) in GitLab 14.3. [Deployed behind the `roadmap_daterange_filter` flag](../../../administration/feature_flags.md), disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/204994) in GitLab 14.3 [with a flag](../../../administration/feature_flags.md) named `roadmap_daterange_filter`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/323917) in GitLab 14.3. -> - [Feature flag `roadmap_daterange_filter` removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72419) in GitLab 14.5. +> - Generally available in GitLab 14.5. [Feature flag `roadmap_daterange_filter`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72419) removed. -Roadmap provides three date range options, each with predetermined timeline duration: +Roadmap provides these date range options, each with a predetermined timeline duration: -- **This quarter**: includes weeks present in current quarter. -- **This year**: includes weeks or months present in current year. -- **Within 3 years**: includes weeks, months, or quarters present in the previous 18 months and - upcoming 18 months (that is, three years in total). +- **This quarter**: Includes the weeks present in the current quarter. +- **This year**: Includes the weeks or months present in the current year. +- **Within 3 years**: Includes the weeks, months, or quarters present both in + the previous 18 months and the upcoming 18 months (three years in total). ### Layout presets -Depending on selected [date range preset](#date-range-presets), Roadmap supports the following layout presets: +Depending on selected [date range preset](#date-range-presets), Roadmap supports +these layout presets: -- **Quarters**: only available when the "Within 3 years" date range is selected. -- **Months**: available when either "This year" or "Within 3 years" date range is selected. -- **Weeks** (default): available for all the date range presets. +- **Quarters**: Available only when the **Within 3 years** date range is selected. +- **Months**: Available when either **This year** or **Within 3 years** date range is selected. +- **Weeks** (default): Available for all the date range presets. ### Quarters @@ -125,12 +126,11 @@ the timeline header represent the month of the quarter.  -In the **Months** preset, roadmap shows epics and milestones which have start or due dates -**falling within** or -**going through** currently selected date range preset, where **today** -is shown by the vertical red line in the timeline. The sub-headers underneath the month name on -the timeline header represent the date on starting day (Sunday) of the week. This preset is -selected by default. +In the **Months** preset, roadmap shows epics and milestones which have start or +due dates **falling within** or **going through** currently selected date range +preset, where **today** is shown by the vertical red line in the timeline. The +sub-headers underneath the month name on the timeline header represent the date +on the start day (Sunday) of the week. This preset is selected by default. ### Weeks diff --git a/doc/user/group/saml_sso/group_managed_accounts.md b/doc/user/group/saml_sso/group_managed_accounts.md index bffaef40800..6771ff8739a 100644 --- a/doc/user/group/saml_sso/group_managed_accounts.md +++ b/doc/user/group/saml_sso/group_managed_accounts.md @@ -107,14 +107,14 @@ Since personal access tokens are the only token needed for programmatic access t ### Set a limit Only a GitLab administrator or an owner of a group-managed account can set a limit. When this field -is left empty, the [instance-level restriction](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) +is left empty, the [instance-level restriction](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) on the lifetime of personal access tokens apply. To set a limit on how long personal access tokens are valid for users in a group managed account: 1. Navigate to the **Settings > General** page in your group's sidebar. 1. Expand the **Permissions and group features** section. -1. Fill in the **Maximum allowable lifetime for personal access tokens (days)** field. +1. Fill in the **Maximum allowable lifetime for access tokens (days)** field. 1. Click **Save changes**. Once a lifetime for personal access tokens is set: diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 4d122e337db..1b480a52920 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -129,7 +129,7 @@ SSO has the following effects when enabled: - For Git activity over SSH and HTTPS, users must have at least one active session signed-in through SSO before they can push to or pull from a GitLab repository. - Git activity originating from CI/CD jobs do not have the SSO check enforced. -- Credentials that are not tied to regular users (for example, access tokens and deploy keys) do not have the SSO check enforced. +- Credentials that are not tied to regular users (for example, project and group access tokens, and deploy keys) do not have the SSO check enforced. - Users must be signed-in through SSO before they can pull images using the [Dependency Proxy](../../packages/dependency_proxy/index.md). When SSO is enforced, users are not immediately revoked. If the user: @@ -589,7 +589,7 @@ Here are possible causes and solutions: | Cause | Solution | | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | You've tried to link multiple SAML identities to the same user, for a given identity provider. | Change the identity that you sign in with. To do so, [unlink the previous SAML identity](#unlinking-accounts) from this GitLab account before attempting to sign in again. | -| The NameID changes everytime the user requests SSO identification | Check the NameID is not set with `Transient` format, or the NameID is not changing on subsequent requests.| +| The NameID changes every time the user requests SSO identification | Check the NameID is not set with `Transient` format, or the NameID is not changing on subsequent requests.| ### Message: "SAML authentication failed: Email has already been taken" diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 3960c97142e..bc1799c2e54 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -110,7 +110,7 @@ Before you start this section: After the above steps are complete: 1. Sign in to Okta. -1. Ensure you are in the Admin section by selecting the **Admin** button located in the top right. The admin button is not visible from the admin page. +1. Ensure you are in the Admin Area by selecting the **Admin** button located in the top right. The button is not visible from the Admin Area. 1. In the **Application** tab, select **Browse App Catalog**. 1. Search for **GitLab**, find and select on the 'GitLab' application. 1. On the GitLab application overview page, select **Add**. diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index 0666303bcf8..4b791d5a221 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -25,7 +25,7 @@ Group access tokens are similar to [project access tokens](../../project/setting and [personal access tokens](../../profile/personal_access_tokens.md), except they are associated with a group rather than a project or user. -In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) as personal access tokens if the limit is set. +In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. You can use group access tokens: @@ -50,7 +50,7 @@ To create a group access token: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the group. -1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. +1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-group-access-token). 1. Select **Create group access token**. diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md index 7b63466656d..23a638fb98c 100644 --- a/doc/user/group/settings/import_export.md +++ b/doc/user/group/settings/import_export.md @@ -86,7 +86,7 @@ To export the contents of a group: NOTE: The maximum import file size can be set by the Administrator, default is `0` (unlimited). -As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](../../../api/settings.md#change-application-settings) or the [Admin UI](../../admin_area/settings/account_and_limit_settings.md). Default [modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to 0 in GitLab 13.8. +As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](../../../api/settings.md#change-application-settings) or the [Admin Area](../../admin_area/settings/account_and_limit_settings.md). Default [modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50MB to 0 in GitLab 13.8. You can also use the [group import/export API](../../../api/group_import_export.md). diff --git a/doc/user/group/value_stream_analytics/img/new_value_stream_v13_12.png b/doc/user/group/value_stream_analytics/img/new_value_stream_v13_12.png Binary files differdeleted file mode 100644 index d64ec31aabf..00000000000 --- a/doc/user/group/value_stream_analytics/img/new_value_stream_v13_12.png +++ /dev/null diff --git a/doc/user/group/value_stream_analytics/img/vsa_aggregated_data_toggle_v14_9.png b/doc/user/group/value_stream_analytics/img/vsa_aggregated_data_toggle_v14_9.png Binary files differdeleted file mode 100644 index 5ad8026b8fd..00000000000 --- a/doc/user/group/value_stream_analytics/img/vsa_aggregated_data_toggle_v14_9.png +++ /dev/null diff --git a/doc/user/group/value_stream_analytics/img/vsa_custom_stage_v13_10.png b/doc/user/group/value_stream_analytics/img/vsa_custom_stage_v13_10.png Binary files differdeleted file mode 100644 index 9345c4023de..00000000000 --- a/doc/user/group/value_stream_analytics/img/vsa_custom_stage_v13_10.png +++ /dev/null diff --git a/doc/user/group/value_stream_analytics/img/vsa_default_stage_v13_10.png b/doc/user/group/value_stream_analytics/img/vsa_default_stage_v13_10.png Binary files differdeleted file mode 100644 index a29689a2c18..00000000000 --- a/doc/user/group/value_stream_analytics/img/vsa_default_stage_v13_10.png +++ /dev/null diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index 3fce9baa9ce..4be97779603 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -11,6 +11,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w Value stream analytics provides metrics about each stage of your software development process. +A **value stream** is the entire work process that delivers value to customers. For example, +the [DevOps lifecycle](https://about.gitlab.com/stages-devops-lifecycle/) is a value stream that starts +with the "manage" stage and ends with the "protect" stage. + Use value stream analytics to identify: - The amount of time it takes to go from an idea to production. @@ -30,7 +34,7 @@ Value stream analytics is also available for [projects](../../analytics/value_st Prerequisite: - You must have at least the Reporter role to view value stream analytics for groups. -- You must create a [custom value stream](#custom-value-streams). Value stream analytics only shows custom value streams created for your group. +- You must create a [custom value stream](#create-a-value-stream-with-gitlab-default-stages). Value stream analytics only shows custom value streams created for your group. To view value stream analytics for your group: @@ -41,9 +45,6 @@ To view value stream analytics for your group: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. Select whether to view metrics for items with a start or stop event: - - To view items with a stop event in the date range, turn on the **Filter by stop date** toggle. Enabled by default. - - To view items with a start event in the date range, turn off the **Filter by stop date** toggle. 1. To adjust the date range: - In the **From** field, select a start date. - In the **To** field, select an end date. The charts and list show workflow items created @@ -78,9 +79,6 @@ To view the median time spent in each stage by a group: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. Select whether to view metrics for items with a start or stop event: - - To view items with a stop event in the date range, turn on the **Filter by stop date** toggle. Enabled by default. - - To view items with a start event in the date range, turn off the **Filter by stop date** toggle. 1. To adjust the date range: - In the **From** field, select a start date. - In the **To** field, select an end date. @@ -91,7 +89,10 @@ To view the median time spent in each stage by a group: Value stream analytics shows the lead time and cycle time for issues in your groups: - Lead time: Median time from when the issue was created to when it was closed. -- Cycle time: Median time from first commit to issue closed. Commits are associated with issues when users [cross-link them in the commit message](../../project/issues/crosslinking_issues.md#from-commit-messages). +- Cycle time: Median time from first commit to issue closed. GitLab measures cycle time from the earliest +commit of a [linked issue's merge request](../../project/issues/crosslinking_issues.md#from-commit-messages) +to when that issue is closed. The cycle time approach underestimates the lead time because merge request creation +is always later than commit time. To view the lead time and cycle time for issues: @@ -101,9 +102,6 @@ To view the lead time and cycle time for issues: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. Select whether to view metrics for items with a start or stop event: - - To view items with a stop event in the date range, turn on the **Filter by stop date** toggle. Enabled by default. - - To view items with a start event in the date range, turn off the **Filter by stop date** toggle. 1. To adjust the date range: - In the **From** field, select a start date. - In the **To** field, select an end date. @@ -124,9 +122,6 @@ To view the lead time for changes for merge requests in your group: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. Select whether to view metrics for items with a start or stop event: - - To view items with a stop event in the date range, turn on the **Filter by stop date** toggle. Enabled by default. - - To view items with a start event in the date range, turn off the **Filter by stop date** toggle. 1. To adjust the date range: - In the **From** field, select a start date. - In the **To** field, select an end date. @@ -140,7 +135,7 @@ The **Lead Time for Changes** metrics display below the **Filter results** text To view deployment metrics, you must have a [production environment configured](../../../ci/environments/index.md#deployment-tier-of-environments). -Value stream analytics shows the following deployment metrics for your group: +Value stream analytics shows the following deployment metrics for your group: - Deploys: The number of successful deployments in the date range. - Deployment Frequency: The average number of successful deployments per day in the date range. @@ -153,13 +148,14 @@ To view deployment metrics for your group: 1. Select the **Filter results** text box. 1. Select a parameter. 1. Select a value or enter text to refine the results. - 1. Select whether to view metrics for items with a start or stop event: - - To view items with a stop event in the date range, turn on the **Filter by stop date** toggle. Enabled by default. - - To view items with a start event in the date range, turn off the **Filter by stop date** toggle. 1. To adjust the date range: - In the **From** field, select a start date. - In the **To** field, select an end date. +NOTE: +The date range selector filters items by the event time. This is the time when the currently +selected stage finished for the given item. + The **Deploys** and **Deployment Frequency** metrics display below the **Filter results** text box. Deployment metrics are calculated based on data from the @@ -174,19 +170,22 @@ In GitLab 13.8 and earlier, metrics are calculated based on when the deployment > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335391) in GitLab 14.5 [with a flag](../../../administration/feature_flags.md) named `use_vsa_aggregated_tables`. Disabled by default. > - Filter by stop date toggle [added](https://gitlab.com/gitlab-org/gitlab/-/issues/352428) in GitLab 14.9 > - Data refresh badge [added](https://gitlab.com/gitlab-org/gitlab/-/issues/341739) in GitLab 14.9 +> - Filter by stop date toggle [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84356) in GitLab 14.9 +> - Enable filtering by stop date [added](https://gitlab.com/gitlab-org/gitlab/-/issues/355000) in GitLab 15.0 -Plans for value stream analytics to filter items by stop event instead of start event are tracked in this [epic](https://gitlab.com/groups/gitlab-org/-/epics/6046). With the completion of this work, value stream analytics will only display items with a stop event in the date range. - -To preview this functionality, you can use the **Filter by stop date** toggle to enable or disable this filter until the [default filtering mode is introduced](../../../update/deprecations.md#value-stream-analytics-filtering-calculation-change) and the toggle is removed. +Value stream analytics uses a backend process to collect and aggregate stage-level data, which +ensures it can scale for large groups with a high number of issues and merge requests. Due to this process, +there may be a slight delay between when an action is taken (for example, closing an issue) and when the data +displays on the value stream analytics page. -If you turn on the **Filter by stop date** toggle, the results show items with a stop event within the date range. When this function is enabled, it may take up to 10 minutes for results to show due to data aggregation. There are occasions when it may take longer than 10 minutes for results to display: +It may take up to 10 minutes to process the data and display results. Data collection may take +longer than 10 minutes in the following cases: -- If this is the first time you are viewing value stream analytics and have not yet [created a value stream](#create-a-value-stream). +- If this is the first time you are viewing value stream analytics and have not yet [created a value stream](#create-a-value-stream-with-gitlab-default-stages). - If the group hierarchy has been re-arranged. - If there have been bulk updates on issues and merge requests. -To view when the data was most recently updated, in the right corner next to **Edit**, hover over the **Last updated** badge. This badge is only available if you have turned on the **Filter by start date** toggle. - +To view when the data was most recently updated, in the right corner next to **Edit**, hover over the **Last updated** badge. ## How value stream analytics measures stages @@ -261,80 +260,58 @@ These patterns are not case-sensitive. You can change the name of a project environment in your GitLab CI/CD configuration. -## Custom value streams - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in GitLab 12.9. - -Use custom value streams to create custom stages that align with your own development processes, -and hide default stages. The dashboard depicts stages as a horizontal process -flow. - -### Create a value stream +## Create a value stream with GitLab default stages > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221202) in GitLab 13.3 -A default value stream is readily available for each group. You can create additional value streams -based on the different areas of work that you would like to measure. - -Once created, a new value stream includes the stages that follow -[GitLab workflow](../../../topics/gitlab_flow.md) -best practices. You can customize this flow by adding, hiding or re-ordering stages. - -To create a value stream: +When you create a value stream, you can use GitLab default stages and hide or re-order them to customize. You can also +create custom stages in addition to those provided in the default template. 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. -1. If this is the first time you are creating a value stream, select **Create custom value stream**. Otherwise, in the top right, select the dropdown list and then **Create new Value Stream**. -1. Enter a name for the new Value Stream. - - You can [customize the stages](#create-a-value-stream-with-stages). -1. Select **Create Value Stream**. - - +1. Select **Create new Value Stream**. +1. Enter a name for the value stream. +1. Select **Create from default template**. +1. Customize the default stages: + - To re-order stages, select the up or down arrows. + - To hide a stage, select **Hide** (**{eye-slash}**). +1. To add a custom stage, select **Add another stage**. + - Enter a name for the stage. + - Select a **Start event** and a **Stop event**. +1. Select **Create value stream**. NOTE: If you have recently upgraded to GitLab Premium, it can take up to 30 minutes for data to collect and display. -### Create a value stream with stages +## Create a value stream with custom stages > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50229) in GitLab 13.7. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55572) in GitLab 13.10. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294190) in GitLab 13.11. -You can create value streams with stages, starting with a default or a blank template. You can -add stages as desired. - -To create a value stream with stages: +When you create a value stream, you can create and add custom stages that align with your own development workflows. 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. -1. If this is the first time you are creating a value stream, select **Create custom value stream**. Otherwise, in the top right, select the dropdown list and then **Create new Value Stream**. -1. Select either **Create from default template** or **Create from no template**. - - You can hide or re-order default stages in the value stream. - -  - - - You can add new stages by selecting **Add another stage**. - - You can select the name and start and end events for the stage. - -  -1. Select **Create Value Stream**. - -### Label-based stages - -The pre-defined start and end events can cover many use cases involving both issues and merge requests. +1. Select **Create value stream**. +1. For each stage: + - Enter a name for the stage. + - Select a **Start event** and a **Stop event**. +1. To add another stage, select **Add another stage**. +1. To re-order the stages, select the up or down arrows. +1. Select **Create value stream**. -In more complex workflows, use stages based on group labels. These events are based on -added or removed labels. In particular, [scoped labels](../../project/labels.md#scoped-labels) -are useful for complex workflows. +### Label-based stages for custom value streams -In this example, we'd like to measure times for deployment from a staging environment to production. The workflow is the following: +To measure complex workflows, you can use [scoped labels](../../project/labels.md#scoped-labels). For example, to measure deployment +time from a staging environment to production, you could use the following labels: - When the code is deployed to staging, the `workflow::staging` label is added to the merge request. - When the code is deployed to production, the `workflow::production` label is added to the merge request.  -### Edit a value stream +## Edit a value stream > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267537) in GitLab 13.10. @@ -342,19 +319,18 @@ After you create a value stream, you can customize it to suit your purposes. To 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. -1. In the top right, select the dropdown list and then select the relevant value stream. +1. In the top right, select the dropdown list, and select a value stream. 1. Next to the value stream dropdown list, select **Edit**. - The edit form is populated with the value stream details. 1. Optional: - Rename the value stream. - Hide or re-order default stages. - Remove existing custom stages. - - Add new stages by selecting the 'Add another stage' button + - To add new stages, select **Add another stage**. - Select the start and end events for the stage. 1. Optional. To undo any modifications, select **Restore value stream defaults**. 1. Select **Save Value Stream**. -### Delete a value stream +## Delete a value stream > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221205) in GitLab 13.4. diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index ab04187284d..be58b4565a1 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -97,6 +97,8 @@ Use CI/CD environment variables to configure your project. 1. Expand **Variables**. 1. Set the variable `BASE64_GOOGLE_CREDENTIALS` to the `base64` encoded JSON file you just created. 1. Set the variable `TF_VAR_gcp_project` to your GCP's `project` name. +1. Set the variable `TF_VAR_agent_token` to the agent token displayed in the previous task. +1. Set the variable `TF_VAR_kas_address` to the agent server address displayed in the previous task. **Optional configuration:** diff --git a/doc/user/infrastructure/clusters/index.md b/doc/user/infrastructure/clusters/index.md index cd7c5feb284..933b310ff3f 100644 --- a/doc/user/infrastructure/clusters/index.md +++ b/doc/user/infrastructure/clusters/index.md @@ -18,7 +18,7 @@ as well as its related [features](#deprecated-features). The certificate-based Kubernetes integration with GitLab is deprecated. It had the following issues: -- There were security issues as it required direct access to the Kube API by GitLab. +- There were security issues as it required direct access to the Kubernetes API by GitLab. - The configuration options weren't flexible. - The integration was flaky. - Users were constantly reporting issues with features based on this model. @@ -42,22 +42,18 @@ the GitLab agent model on the [agent's blueprint documentation](../../../archite ## Deprecated features -- [Create a new cluster through cluster certificates](../../project/clusters/add_remove_clusters.md) - [Connect an existing cluster through cluster certificates](../../project/clusters/add_existing_cluster.md) - [Access controls](../../project/clusters/cluster_access.md) - [GitLab-managed clusters](../../project/clusters/gitlab_managed_clusters.md) -- [GitLab Managed Apps](../../clusters/applications.md) - [Deploy applications through certificate-based connection](../../project/clusters/deploy_to_cluster.md) - [Cluster Management Project](../../clusters/management_project.md) - [Cluster integrations](../../clusters/integrations.md) - [Cluster cost management](../../clusters/cost_management.md) - [Cluster environments](../../clusters/environments.md) -- [Advanced traffic control with Canary Ingress](../../project/canary_deployments.md#advanced-traffic-control-with-canary-ingress-deprecated) -- [Serverless](../../project/clusters/serverless/index.md) +- [Show Canary Ingress deployments on deploy boards](../../project/canary_deployments.md#show-canary-ingress-deployments-on-deploy-boards-deprecated) - [Deploy Boards](../../project/deploy_boards.md) - [Pod logs](../../project/clusters/kubernetes_pod_logs.md) - [Clusters health](manage/clusters_health.md) -- [Crossplane integration](../../clusters/crossplane.md) - [Web terminals](../../../administration/integration/terminal.md) ### Cluster levels @@ -67,5 +63,5 @@ The concept of [project-level](../../project/clusters/index.md), [instance-level](../../instance/clusters/index.md) clusters becomes extinct in the new model, although the functionality remains to some extent. -The agent is always configured in a single GitLab project and you can expose the cluster connection to other projects and groups to [access it from GitLab CI/CD](../../clusters/agent/ci_cd_tunnel.md). +The agent is always configured in a single GitLab project and you can expose the cluster connection to other projects and groups to [access it from GitLab CI/CD](../../clusters/agent/ci_cd_workflow.md). By doing so, you are granting these projects and groups access to the same cluster, which is similar to group-level clusters' use case. diff --git a/doc/user/infrastructure/clusters/manage/clusters_health.md b/doc/user/infrastructure/clusters/manage/clusters_health.md index eeb931f392f..0eefae3f550 100644 --- a/doc/user/infrastructure/clusters/manage/clusters_health.md +++ b/doc/user/infrastructure/clusters/manage/clusters_health.md @@ -4,10 +4,15 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Clusters health **(FREE)** +# Clusters health (DEPRECATED) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/4701) in GitLab 10.6. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/208224) from GitLab Ultimate to GitLab Free in 13.2. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. However, you can **still use** Prometheus +for Kubernetes clusters connected to GitLab by [enabling Prometheus manually](../../../project/integrations/prometheus.md#manual-configuration-of-prometheus). When [the Prometheus cluster integration is enabled](../../../clusters/integrations.md#prometheus-cluster-integration), GitLab monitors the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md b/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md deleted file mode 100644 index ae335a180e8..00000000000 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/apparmor.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Install AppArmor with a cluster management project **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. - -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a -[management project template](../../../../../user/clusters/management_project_template.md), to install AppArmor you should -uncomment this line from your `helmfile.yaml`: - -```yaml - - path: applications/apparmor/helmfile.yaml -``` - -You can define one or more AppArmor profiles by adding them into -`applications/apparmor/values.yaml` as the following: - -```yaml -profiles: - profile-one: |- - profile profile-one { - file, - } -``` - -Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for more information on this chart. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md index 58de5f5e368..5ad1fb81a39 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/certmanager.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Support for cert-manager v1.4 was [introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/69405) in GitLab 14.3. > - [Upgraded](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/23) to cert-manager 1.7 in GitLab 14.8. -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install cert-manager you should uncomment this line from your `helmfile.yaml`: diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md b/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md deleted file mode 100644 index 5d704a2c6df..00000000000 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/cilium.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Install Cilium with a cluster management project **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. - -[Cilium](https://cilium.io/) is a networking plugin for Kubernetes that you can use to implement -support for [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, see the -[Container Network Security Demo for GitLab 12.8](https://www.youtube.com/watch?v=pgUEdhdhoUI). - -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a -[management project template](../../../../../user/clusters/management_project_template.md), to install cilium you should -uncomment this line from your `helmfile.yaml`: - -```yaml - - path: applications/cilium/helmfile.yaml -``` - -and update the `applications/cilium/values.yaml` to set the `clusterType`: - -```yaml -# possible values are gke or eks -clusterType: gke -``` - -The `clusterType` variable enables the recommended Helm variables for a corresponding cluster type. -You can check the recommended variables for each cluster type in the official documentation: - -- [Google GKE](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-gke/#deploy-cilium) -- [AWS EKS](https://docs.cilium.io/en/v1.8/gettingstarted/k8s-install-eks/#deploy-cilium) - -Do not use `clusterType` for sandbox environments like [minikube](https://minikube.sigs.k8s.io/docs/). - -You can customize Cilium's Helm variables by defining the -`applications/cilium/values.yaml` file in your cluster -management project. Refer to the -[Cilium chart](https://github.com/cilium/cilium/tree/master/install/kubernetes/cilium) -for the available configuration options. - -You can check Cilium's installation status on the cluster management page: - -- [Project-level cluster](../../../../project/clusters/index.md): Navigate to your project's - **Infrastructure > Kubernetes clusters** page. -- [Group-level cluster](../../../../group/clusters/index.md): Navigate to your group's - **Kubernetes** page. - -WARNING: -Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) -of all affected pods in all namespaces to ensure that they are -[managed](https://docs.cilium.io/en/v1.8/operations/troubleshooting/#ensure-managed-pod) -by the correct networking plugin. Whenever Hubble is enabled, its related pod might require a -restart depending on whether it started prior to Cilium. For more information, see -[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) -in the Kubernetes docs. - -NOTE: -Major upgrades might require additional setup steps. For more information, see -the official [upgrade guide](https://docs.cilium.io/en/v1.8/operations/upgrade/). - -By default, Cilium's -[audit mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/#enable-policy-audit-mode) -is enabled. In audit mode, Cilium doesn't drop disallowed packets. You -can use `policy-verdict` log to observe policy-related decisions. You -can disable audit mode by adding the following to -`applications/cilium/values.yaml`: - -```yaml -config: - policyAuditMode: false - -agent: - monitor: - eventTypes: ["drop"] -``` - -The Cilium monitor log for traffic is logged out by the -`cilium-monitor` sidecar container. You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -You can disable the monitor log in `.gitlab/managed-apps/cilium/values.yaml`: - -```yaml -agent: - monitor: - enabled: false -``` - -The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default -and it's set to collect per namespace flow metrics. This metrics are accessible on the -[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) -dashboard. You can disable Hubble by adding the following to -`applications/cilium/values.yaml`: - -```yaml -global: - hubble: - enabled: false -``` - -You can also adjust Helm values for Hubble by using -`applications/cilium/values.yaml`: - -```yaml -global: - hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md b/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md index f9d0948a2bb..7ab99ab3875 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/elasticstack.md @@ -2,28 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../index.md' --- -# Install Elastic Stack with a cluster management project **(FREE)** +# Install Elastic Stack with a cluster management project (removed) **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. - -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a -[management project template](../../../../../user/clusters/management_project_template.md), to install Elastic Stack you should -uncomment this line from your `helmfile.yaml`: - -```yaml - - path: applications/elastic-stack/helmfile.yaml -``` - -Elastic Stack is installed by default into the `gitlab-managed-apps` namespace of your cluster. - -You can check the default -[`values.yaml`](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/blob/master/applications/elastic-stack/values.yaml) -we set for this chart. - -You can customize the installation of Elastic Stack by updating the -`applications/elastic-stack/values.yaml` file in your cluster -management project. Refer to the -[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for all -available configuration options. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 14.8 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/360182) in 15.0. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md b/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md deleted file mode 100644 index 50401e9a391..00000000000 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/falco.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Install Falco with a cluster management project **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. - -GitLab Container Host Security Monitoring uses [Falco](https://falco.org/) -as a runtime security tool that listens to the Linux kernel using eBPF. Falco parses system calls -and asserts the stream against a configurable rules engine in real-time. For more information, see -[Falco's Documentation](https://falco.org/docs/). - -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a -[management project template](../../../../../user/clusters/management_project_template.md), to install Falco you should -uncomment this line from your `helmfile.yaml`: - -```yaml - - path: applications/falco/helmfile.yaml -``` - -You can customize Falco's Helm variables by defining the -`applications/falco/values.yaml` file in your cluster -management project. Refer to the -[Falco chart](https://github.com/falcosecurity/charts/tree/master/falco) -for the available configuration options. - -WARNING: -By default eBPF support is enabled and Falco uses an -[eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) -to pass system calls to user space. If your cluster doesn't support this, you can -configure it to use Falco kernel module instead by adding the following to -`applications/falco/values.yaml`: - -```yaml -ebpf: - enabled: false -``` - -In rare cases where probe installation on your cluster isn't possible and the kernel/probe -isn't pre-compiled, you may need to manually prepare the kernel module or eBPF probe with -[`driverkit`](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) -and install it on each cluster node. - -By default, Falco is deployed with a limited set of rules. To add more rules, add -the following to `applications/falco/values.yaml` (you can get examples from -[Cloud Native Security Hub](https://securityhub.dev/)): - -```yaml -customRules: - file-integrity.yaml: |- - - rule: Detect New File - desc: detect new file created - condition: > - evt.type = chmod or evt.type = fchmod - output: > - File below a known directory opened for writing (user=%user.name - command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) - priority: ERROR - tags: [filesystem] - - rule: Detect New Directory - desc: detect new directory created - condition: > - mkdir - output: > - File below a known directory opened for writing (user=%user.name - command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2]) - priority: ERROR - tags: [filesystem] -``` - -By default, Falco only outputs security events to logs as JSON objects. To set it to output to an -[external API](https://falco.org/docs/alerts/#https-output-send-alerts-to-an-https-end-point) -or [application](https://falco.org/docs/alerts/#program-output), -add the following to `applications/falco/values.yaml`: - -```yaml -falco: - programOutput: - enabled: true - keepAlive: false - program: mail -s "Falco Notification" someone@example.com - - httpOutput: - enabled: true - url: http://some.url -``` - -You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l app=falco -``` diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md b/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md deleted file mode 100644 index ea3a3503f9b..00000000000 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/fluentd.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Install Fluentd with a cluster management project **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. - -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a -[management project template](../../../../../user/clusters/management_project_template.md), to install Fluentd you should -uncomment this line from your `helmfile.yaml`: - -```yaml - - path: applications/fluentd/helmfile.yaml -``` - -You can also review the default values set for this chart in the -[`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. - -You can customize the installation of Fluentd by defining -`applications/fluentd/values.yaml` file in your cluster management -project. Refer to the -[configuration chart](https://github.com/helm/charts/tree/master/stable/fluentd#configuration) -for the current development release of Fluentd for all available configuration options. - -The configuration chart link points to the current development release, which -may differ from the version you have installed. To ensure compatibility, switch -to the specific branch or tag you are using. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md index 503f077df14..7983a640577 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/ingress.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install Ingress you should uncomment this line from your `helmfile.yaml`: diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md b/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md index f76c7363a83..383e857bb20 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/prometheus.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w open-source monitoring and alerting system for supervising your deployed applications. -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install Prometheus you should uncomment this line from your `helmfile.yaml`: diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md index 4faf5f46418..ef7c4637607 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install GitLab Runner you should uncomment this line from your `helmfile.yaml`: @@ -35,7 +35,7 @@ These values can be specified using [CI/CD variables](../../../../../ci/variable The methods of specifying these values are mutually exclusive. Either specify variables `GITLAB_RUNNER_REGISTRATION_TOKEN` and `CI_SERVER_URL` as CI variables (recommended) or provide values for `runnerRegistrationToken:` and `gitlabUrl:` in `applications/gitlab-runner/values.yaml.gotmpl`. -The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../../../../ci/variables/index.md#protect-a-cicd-variable) and [masked variable](../../../../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml.gotmpl` file. +The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../../../../ci/variables/index.md#protected-cicd-variables) and [masked variable](../../../../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml.gotmpl` file. You can customize the installation of GitLab Runner by defining `applications/gitlab-runner/values.yaml.gotmpl` file in your cluster diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md b/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md index b968e63d632..d2d314b649e 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/sentry.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) at least 3 GB of available RAM for database migrations. -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install Sentry you should uncomment this line from your `helmfile.yaml`: diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md index 4618a95f986..06e67b78c91 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/vault.md @@ -20,7 +20,7 @@ control. Therefore, if GitLab is compromised, the security of this Vault instanc avoid this security risk, GitLab recommends using your own HashiCorp Vault to leverage [external secrets with CI](../../../../../ci/secrets/index.md). -Assuming you already have a [Cluster management project](../../../../../user/clusters/management_project.md) created from a +Assuming you already have a project created from a [management project template](../../../../../user/clusters/management_project_template.md), to install Vault you should uncomment this line from your `helmfile.yaml`: diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index 61ec0a559f0..7b2b5b4afd4 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -9,19 +9,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w To connect your Kubernetes cluster with GitLab, you can use: - [A GitOps workflow](../../clusters/agent/gitops.md). -- [A GitLab CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +- [A GitLab CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). - [A certificate-based integration](index.md). The certificate-based integration is [**deprecated**](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/) -in GitLab 14.5. It is expected to be -[turned off by default in 15.0](../../../update/deprecations.md#certificate-based-integration-with-kubernetes) -and removed in GitLab 15.6. +in GitLab 14.5. The sunsetting plans are described: + +- for [GitLab.com customers](../../../update/deprecations.md#saas-certificate-based-integration-with-kubernetes). +- for [Self-managed customers](../../../update/deprecations.md#self-managed-certificate-based-integration-with-kubernetes). If you are using the certificate-based integration, you should move to another workflow as soon as possible. As a general rule, to migrate clusters that rely on GitLab CI/CD, -you can use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +you can use the [CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). This workflow uses an agent to connect to your cluster. The agent: - Is not exposed to the internet. @@ -39,7 +40,7 @@ Some features are currently available only when using certificate-based integrat With GitLab-managed clusters, GitLab creates separate service accounts and namespaces for every branch and deploys by using these resources. -The GitLab agent uses [impersonation](../../clusters/agent/ci_cd_tunnel.md#use-impersonation-to-restrict-project-and-group-access) +The GitLab agent uses [impersonation](../../clusters/agent/ci_cd_workflow.md#use-impersonation-to-restrict-project-and-group-access) strategies to deploy to your cluster with restricted account access. To do so: 1. Choose the impersonation strategy that suits your needs. @@ -48,30 +49,55 @@ strategies to deploy to your cluster with restricted account access. To do so: ### Migrate from Auto DevOps -To configure your Auto DevOps project to use the GitLab agent: - -1. Follow the steps to [install an agent](../../clusters/agent/install/index.md) in your cluster. -1. Go to the project where you use Auto DevOps. -1. On the left sidebar, select **Settings > CI/CD** and expand **Variables**. -1. Select **Add new variable**. -1. Add `KUBE_CONTEXT` as the key, `path/to/agent/project:agent-name` as the value, and select the environment scope of your choice. +In your Auto DevOps project, you can use the GitLab agent to connect with your Kubernetes cluster. + +1. [Install an agent](../../clusters/agent/install/index.md) in your cluster. +1. In GitLab, go to the project where you use Auto DevOps. +1. Add three variables. On the left sidebar, select **Settings > CI/CD** and expand **Variables**. + - Add a key called `KUBE_INGRESS_BASE_DOMAIN` with the application deployment domain as the value. + - Add a key called `KUBE_CONTEXT` with a value like `path/to/agent/project:agent-name`. + Select the environment scope of your choice. + If you are not sure what your agent’s context is, edit your `.gitlab-ci.yml` file and add a job to see the available contexts: + + ```yaml + deploy: + image: + name: bitnami/kubectl:latest + entrypoint: [""] + script: + - kubectl config get-contexts + ``` + + - Add a key called `KUBE_NAMESPACE` with a value of the Kubernetes namespace for your deployments to target. Set the same environment scope. 1. Select **Add variable**. -1. Repeat the process to add another variable, `KUBE_NAMESPACE`, setting the value for the Kubernetes namespace you want your deployments to target, and set the same environment scope from the previous step. 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. From the certificate-based clusters section, open the cluster that serves the same environment scope. 1. Select the **Details** tab and disable the cluster. -1. To activate the changes, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. +1. Edit your `.gitlab-ci.yml` file and ensure it's using the Auto DevOps template. For example: + + ```yaml + include: + template: Auto-DevOps.gitlab-ci.yml + + variables: + KUBE_INGRESS_BASE_DOMAIN: 74.220.23.215.nip.io + KUBE_CONTEXT: "gitlab-examples/ops/gitops-demo/k8s-agents:demo-agent" + KUBE_NAMESPACE: "demo-agent" + ``` + +1. To test your pipeline, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. For an example, [view this project](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service). ### Migrate generic deployments -Follow the process for the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +Follow the process for the [CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). ## Migrate from GitLab Managed applications -[GitLab Managed Apps (GMA)](../../clusters/applications.md#gitlab-managed-apps-deprecated) were deprecated in GitLab 14.0, and -the agent for Kubernetes does not support them. To migrate from GMA to the agent, go through the following steps: +GitLab Managed Apps (GMA) were deprecated in GitLab 14.0, and removed in GitLab 15.0. +The agent for Kubernetes does not support them. To migrate from GMA to the +agent, go through the following steps: 1. [Migrate from GitLab Managed Apps to a cluster management project](../../clusters/migrating_from_gma_to_project_template.md). 1. [Migrate the cluster management project to use the agent](../../clusters/management_project_template.md). diff --git a/doc/user/infrastructure/iac/img/terraform_list_view_v13_8.png b/doc/user/infrastructure/iac/img/terraform_list_view_v13_8.png Binary files differdeleted file mode 100644 index 6eb85285e81..00000000000 --- a/doc/user/infrastructure/iac/img/terraform_list_view_v13_8.png +++ /dev/null diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index bc7a3c0d069..0f7965a3527 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -86,7 +86,7 @@ To use a Terraform template: # TF_ROOT: terraform/production ``` -1. (Optional) Override in your `.gitlab-ci.yaml` file the attributes present +1. (Optional) Override in your `.gitlab-ci.yml` file the attributes present in the template you fetched to customize your configuration. ## GitLab-managed Terraform state @@ -107,13 +107,9 @@ workflows. ## The GitLab Terraform provider -NOTE: -The GitLab Terraform provider is released separately from GitLab. -We are working on migrating the GitLab Terraform provider to GitLab.com. - -The [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) is a plugin for Terraform to facilitate -managing of GitLab resources such as users, groups, and projects. -Its documentation is available on [Terraform](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs). +The [GitLab Terraform provider](https://github.com/gitlabhq/terraform-provider-gitlab) is a Terraform plugin to facilitate +managing of GitLab resources such as users, groups, and projects. It is released separately from GitLab +and its documentation is available on [Terraform](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs). ## Create a new cluster through IaC diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 60f97f522cf..f56fe92ec01 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -8,58 +8,43 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. -[Terraform remote backends](https://www.terraform.io/language/settings/backends) -enable you to store the state file in a remote, shared store. GitLab uses the -[Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) -to securely store the state files in local storage (the default) or -[the remote store of your choice](../../../administration/terraform_state.md). +Terraform uses state files to store details about your infrastructure configuration. +With Terraform remote [backends](https://www.terraform.io/language/settings/backends), +you can store the state file in a remote and shared store. -WARNING: -Using local storage (the default) on clustered deployments of GitLab will result in -a split state across nodes, making subsequent executions of Terraform inconsistent. -You are highly advised to use a remote storage resource in that case. - -The GitLab-managed Terraform state backend can store your Terraform state easily and -securely, and spares you from setting up additional remote resources like -Amazon S3 or Google Cloud Storage. Its features include: - -- Versioning of Terraform state files. -- Supporting encryption of the state file both in transit and at rest. -- Locking and unlocking state. -- Remote Terraform plan and apply execution. - -A GitLab **administrator** must [set up the Terraform state storage configuration](../../../administration/terraform_state.md) -before using this feature. +GitLab provides a [Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) +to securely store your state files with minimal configuration. -## Permissions for using Terraform +In GitLab, you can: -In GitLab version 13.1, at least the Maintainer role was required to use a -GitLab managed Terraform state backend. +- Version your Terraform state files. +- Encrypt the state file both in transit and at rest. +- Lock and unlock states. +- Remotely execute `terraform plan` and `terraform apply` commands. -In GitLab versions 13.2 and later, at least: +For self-managed instances, before you can use GitLab for your Terraform state files, +an administrator must [set up Terraform state storage](../../../administration/terraform_state.md). -- The Maintainer role is required to lock, unlock, and write to the state (using `terraform apply`). -- The Developer role is required to read the state (using `terraform plan -lock=false`). +## Initialize a Terraform state as a backend by using GitLab CI/CD -## Set up GitLab-managed Terraform state +After you execute the `terraform init` command, you can use GitLab CI/CD +to run `terraform` commands. -To get started with a GitLab-managed Terraform state, there are two different options: +Prerequisites: -- [Use a local machine](#get-started-using-local-development). -- [Use GitLab CI](#get-started-using-gitlab-ci). +- To lock, unlock, and write to the state by using `terraform apply`, you must have at least the Maintainer role. +- To read the state by using `terraform plan -lock=false`, you must have at least the Developer role. -Terraform States can be found by navigating to a Project's -**{cloud-gear}** **Infrastructure > Terraform** page. - -### Get started using local development +WARNING: +Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. +Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan +includes sensitive data, like passwords, access tokens, or certificates, you should +encrypt plan output or modify the project visibility settings. -If you plan to only run `terraform plan` and `terraform apply` commands from your -local machine, this is a simple way to get started: +To configure GitLab CI/CD as a backend: -1. Create your project on your GitLab instance. -1. Navigate to **Settings > General** and note your **Project name** - and **Project ID**. -1. Define the Terraform backend in your Terraform project to be: +1. In your Terraform project, in a `.tf` file like `backend.tf`, + define the [HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html): ```hcl terraform { @@ -68,172 +53,51 @@ local machine, this is a simple way to get started: } ``` -1. Create a [Personal Access Token](../../profile/personal_access_tokens.md) with - the `api` scope. - -1. On your local machine, run `terraform init`, passing in the following options, - replacing `<YOUR-STATE-NAME>`, `<YOUR-PROJECT-ID>`, `<YOUR-USERNAME>` and - `<YOUR-ACCESS-TOKEN>` with the relevant values. This command initializes your - Terraform state, and stores that state in your GitLab project. This example - uses `gitlab.com`: - - ```shell - terraform init \ - -backend-config="address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>" \ - -backend-config="lock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>/lock" \ - -backend-config="unlock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>/lock" \ - -backend-config="username=<YOUR-USERNAME>" \ - -backend-config="password=<YOUR-ACCESS-TOKEN>" \ - -backend-config="lock_method=POST" \ - -backend-config="unlock_method=DELETE" \ - -backend-config="retry_wait_min=5" - ``` - - WARNING: - The name of your state can contain only uppercase and lowercase letters, decimal digits, - hyphens, and underscores. - -If you already have a GitLab-managed Terraform state, you can use the `terraform init` command -with the pre-populated parameters values: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Infrastructure > Terraform**. -1. Next to the environment you want to use, select the [Actions menu](#managing-state-files) - **{ellipsis_v}** and select **Copy Terraform init command**. - -You can now run `terraform plan` and `terraform apply` as you normally would. - -### Get started using GitLab CI - -If you don't want to start with local development, you can also use GitLab CI to -run your `terraform plan` and `terraform apply` commands. - -Next, [configure the backend](#configure-the-backend). - -#### Configure the backend +1. In the root directory of your project repository, create a `.gitlab-ci.yml` file. Use + [this file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) + to populate it. + +1. Push your project to GitLab. This action triggers a pipeline, which + runs the `gitlab-terraform init`, `gitlab-terraform validate`, and + `gitlab-terraform plan` commands. +1. Trigger the manual `terraform apply` job from the previous pipeline to provision the defined infrastructure. -After executing the `terraform init` command, you must configure the Terraform backend -and the CI YAML file: +The output from the above `terraform` commands should be viewable in the job logs. -1. In your Terraform project, define the [HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html) - by adding the following code block in a `.tf` file (such as `backend.tf`) to - define the remote backend: +## Access the state from your local machine - ```hcl - terraform { - backend "http" { - } - } - ``` +You can access the GitLab-managed Terraform state from your local machine. -1. In the root directory of your project repository, configure a - `.gitlab-ci.yml` file. This example uses a pre-built image which includes a - `gitlab-terraform` helper. For supported Terraform versions, see the [GitLab - Terraform Images project](https://gitlab.com/gitlab-org/terraform-images). +WARNING: +On clustered deployments of GitLab, you should not use local storage. +A split state can occur across nodes, making subsequent Terraform executions +inconsistent. Instead, use a remote storage resource. - ```yaml - image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest - ``` +1. Ensure the Terraform state has been + [initialized for CI/CD](#initialize-a-terraform-state-as-a-backend-by-using-gitlab-cicd). +1. Copy a pre-populated Terraform `init` command: -1. In the `.gitlab-ci.yml` file, define some CI/CD variables to ease - development. In this example, `TF_ROOT` is the directory where the Terraform - commands must be executed, `TF_ADDRESS` is the URL to the state on the GitLab - instance where this pipeline runs, and the final path segment in `TF_ADDRESS` - is the name of the Terraform state. Projects may have multiple states, and - this name is arbitrary, so in this example we set it to `example-production` - which corresponds with the directory we're using as our `TF_ROOT`, and we - ensure that the `.terraform` directory is cached between jobs in the pipeline - using a cache key based on the state name (`example-production`): - - ```yaml - variables: - TF_ROOT: ${CI_PROJECT_DIR}/environments/example/production - TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/example-production - - cache: - key: example-production - paths: - - ${TF_ROOT}/.terraform - ``` + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Infrastructure > Terraform**. + 1. Next to the environment you want to use, select **Actions** + (**{ellipsis_v}**) and select **Copy Terraform init command**. -1. In a `before_script`, change to your `TF_ROOT`: - - ```yaml - before_script: - - cd ${TF_ROOT} - - stages: - - prepare - - validate - - build - - deploy - - init: - stage: prepare - script: - - gitlab-terraform init - - validate: - stage: validate - script: - - gitlab-terraform validate - - plan: - stage: build - script: - - gitlab-terraform plan - - gitlab-terraform plan-json - artifacts: - name: plan - paths: - - ${TF_ROOT}/plan.cache - reports: - terraform: ${TF_ROOT}/plan.json - - apply: - stage: deploy - environment: - name: production - script: - - gitlab-terraform apply - dependencies: - - plan - when: manual - only: - - master - ``` - -1. Push your project to GitLab, which triggers a CI job pipeline. This pipeline - runs the `gitlab-terraform init`, `gitlab-terraform validate`, and - `gitlab-terraform plan` commands. +1. Open a terminal and run this command on your local machine. -The output from the above `terraform` commands should be viewable in the job logs. +## Migrate to a GitLab-managed Terraform state -WARNING: -Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. -Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan -includes sensitive data such as passwords, access tokens, or certificates, GitLab strongly -recommends encrypting plan output or modifying the project visibility settings. +Terraform supports copying the state when the backend changes or is +reconfigured. Use these actions to migrate from another backend to +GitLab-managed Terraform state. -### Example project +You should use a local terminal to run the commands needed for migrating to GitLab-managed Terraform state. -See [this reference project](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws) using GitLab and Terraform to deploy a basic AWS EC2 in a custom VPC. +The following example demonstrates how to change the state name. The same workflow is needed to migrate to GitLab-managed Terraform state from a different state storage backend. -## Using a GitLab-managed Terraform state backend as a remote data source +## Use your GitLab backend as a remote data source -You can use a GitLab-managed Terraform state as a +You can use a GitLab-managed Terraform state backend as a [Terraform data source](https://www.terraform.io/language/state/remote-state-data). -To use your existing Terraform state backend as a data source, provide the following details -as [Terraform input variables](https://www.terraform.io/language/values/variables): - -- **address**: The URL of the remote state backend you want to use as a data source. - For example, `https://gitlab.com/api/v4/projects/<TARGET-PROJECT-ID>/terraform/state/<TARGET-STATE-NAME>`. -- **username**: The username to authenticate with the data source. If you are using a [Personal Access Token](../../profile/personal_access_tokens.md) for - authentication, this is your GitLab username. If you are using GitLab CI, this is `'gitlab-ci-token'`. -- **password**: The password to authenticate with the data source. If you are using a Personal Access Token for - authentication, this is the token value. If you are using GitLab CI, it is the contents of the `${CI_JOB_TOKEN}` CI/CD variable. - -An example setup is shown below: 1. Create a file named `example.auto.tfvars` with the following contents: @@ -243,7 +107,7 @@ An example setup is shown below: example_access_token=<GitLab Personal Access Token> ``` -1. Define the data source by adding the following code block in a `.tf` file (such as `data.tf`): +1. In a `.tf` file, define the data source by using [Terraform input variables](https://www.terraform.io/language/values/variables): ```hcl data "terraform_remote_state" "example" { @@ -257,21 +121,20 @@ An example setup is shown below: } ``` + - **address**: The URL of the remote state backend you want to use as a data source. + For example, `https://gitlab.com/api/v4/projects/<TARGET-PROJECT-ID>/terraform/state/<TARGET-STATE-NAME>`. + - **username**: The username to authenticate with the data source. If you are using + a [Personal Access Token](../../profile/personal_access_tokens.md) for + authentication, this value is your GitLab username. If you are using GitLab CI/CD, this value is `'gitlab-ci-token'`. + - **password**: The password to authenticate with the data source. If you are using a Personal Access Token for + authentication, this value is the token value. If you are using GitLab CI/CD, this value is the contents of the `${CI_JOB_TOKEN}` CI/CD variable. + Outputs from the data source can now be referenced in your Terraform resources using `data.terraform_remote_state.example.outputs.<OUTPUT-NAME>`. -You need at least the Developer role in the target project -to read the Terraform state. +To read the Terraform state in the target project, you need at least the Developer role. -## Migrating to GitLab-managed Terraform state - -Terraform supports copying the state when the backend is changed or -reconfigured. This can be useful if you need to migrate from another backend to -GitLab-managed Terraform state. Using a local terminal is recommended to run the commands needed for migrating to GitLab-managed Terraform state. - -The following example demonstrates how to change the state name, the same workflow is needed to migrate to GitLab-managed Terraform state from a different state storage backend. - -### Setting up the initial backend +### Set up the initial backend ```shell PROJECT_ID="<gitlab-project-id>" @@ -309,7 +172,7 @@ re-run this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. ``` -### Changing the backend +### Change the backend Now that `terraform init` has created a `.terraform/` directory that knows where the old state is, you can tell it about the new location: @@ -366,94 +229,54 @@ commands will detect it and remind you to do so if necessary. If you type `yes`, it copies your state from the old location to the new location. You can then go back to running it in GitLab CI/CD. -## Managing state files +## Manage Terraform state files > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/273592) in GitLab 13.8. -Users with at least the Developer role can view the -state files attached to a project at **Infrastructure > Terraform**. Users with the -Maintainer role can perform commands on the state files. The user interface -contains these fields: - - +To view Terraform state files: -- **Name**: The name of the environment, with a locked (**{lock}**) icon if the - state file is locked. -- **Pipeline**: A link to the most recent pipeline and its status. -- **Details**: Information about when the state file was created or changed. -- **Actions**: Actions you can take on the state file, including copying the `terraform init` command, - downloading, locking, unlocking, or [removing](#remove-a-state-file) the state file and versions. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Infrastructure > Terraform**. -NOTE: -Additional improvements to the -[graphical interface for managing state files](https://gitlab.com/groups/gitlab-org/-/epics/4563) -are planned. +[An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4563) to track improvements to this UI. -## Manage individual Terraform state versions +### Manage individual Terraform state versions > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207347) in GitLab 13.4. Individual state versions can be managed using the GitLab REST API. -Users with the [Developer role](../../permissions.md) can retrieve state versions using their serial number. To retrieve a version: +If you have at least the Developer role, you can retrieve state versions by using their serial number:: ```shell curl --header "Private-Token: <your_access_token>" "https://gitlab.example.com/api/v4/projects/<your_project_id>/terraform/state/<your_state_name>/versions/<version-serial>" ``` -Users with the [Maintainer role](../../permissions.md) can remove state versions using their serial number. To remove a version: +If you have at least the Maintainer role, you can remove state versions by using their serial number: ```shell curl --header "Private-Token: <your_access_token>" --request DELETE "https://gitlab.example.com/api/v4/projects/<your_project_id>/terraform/state/<your_state_name>/versions/<version-serial>" ``` -## Remove a state file - -Users with at least the Maintainer role can use the -following options to remove a state file: +### Remove a state file -- **GitLab UI**: Go to **Infrastructure > Terraform**. In the **Actions** column, - click the vertical ellipsis (**{ellipsis_v}**) button and select - **Remove state file and versions**. -- **GitLab REST API**: You can remove a state file by making a request to the - REST API. For example: +If you have at least the Maintainer role, you can remove a state file. - ```shell - curl --header "Private-Token: <your_access_token>" --request DELETE "https://gitlab.example.com/api/v4/projects/<your_project_id>/terraform/state/<your_state_name>" - ``` - -- [GitLab GraphQL API](#remove-a-state-file-with-the-gitlab-graphql-api). - -### Remove a state file with the GitLab GraphQL API - -You can remove a state file by making a GraphQL API request. For example: +1. On the left sidebar, select **Infrastructure > Terraform**. +1. In the **Actions** column, select **Actions** (**{ellipsis_v}**) and then **Remove state file and versions**. -```shell -mutation deleteState { - terraformStateDelete(input: { id: "<global_id_for_the_state>" }) { - errors - } -} -``` +### Remove a state file by using the API -You can obtain the `<global_id_for_the_state>` by querying the list of states: +You can remove a state file by making a request to the REST API. For example: ```shell -query ProjectTerraformStates { - project(fullPath: "<your_project_path>") { - terraformStates { - nodes { - id - name - } - } - } -} +curl --header "Private-Token: <your_access_token>" --request DELETE "https://gitlab.example.com/api/v4/projects/<your_project_id>/terraform/state/<your_state_name>" ``` -For those new to the GitLab GraphQL API, read -[Getting started with GitLab GraphQL API](../../../api/graphql/getting_started.md). +You can also use [the GraphQL API](../../../api/graphql/reference/index.md#mutationterraformstatedelete). ## Related topics - [Troubleshooting GitLab-managed Terraform state](troubleshooting.md). +- To use GitLab and Terraform to deploy an AWS EC2 instance in a custom VPC, + see [this sample project](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws). diff --git a/doc/user/infrastructure/iac/troubleshooting.md b/doc/user/infrastructure/iac/troubleshooting.md index bc0aa39bc70..9dfe58396e2 100644 --- a/doc/user/infrastructure/iac/troubleshooting.md +++ b/doc/user/infrastructure/iac/troubleshooting.md @@ -38,33 +38,37 @@ To workaround this issue, make sure to apply one of the following conditions: 1. Grant Maintainer or Owner role to the `terraform-user` user on `subgroup-B`. 1. The `terraform-user` inherited access to `subgroup-B` and `subgroup-B` contains at least one project. -### Invalid CI/CD syntax error when using the `latest` base template +### Invalid CI/CD syntax error when using the base template -On GitLab 14.2 and later, you might get a CI/CD syntax error when using the -`latest` Base Terraform template: +You might encounter a CI/CD syntax error when using the Terraform templates: + +- On GitLab 14.2 and later, using the `latest` template. +- On GitLab 15.0 and later, using any version of the template. + +For example: ```yaml include: + # On 14.2 and later, when using either of the following: - template: Terraform/Base.latest.gitlab-ci.yml + - template: Terraform.latest.gitlab-ci.yml + # On 15.0 and later, the following templates have also been updated: + - template: Terraform/Base.gitlab-ci.yml + - template: Terraform.gitlab-ci.yml -my-Terraform-job: - extends: .init +my-terraform-job: + extends: .apply ``` -The base template's [jobs were renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67719/) -with better Terraform-specific names. To resolve the syntax error, you can: - -- Use the stable `Terraform/Base.gitlab-ci.yml` template, which has not changed. -- Update your pipeline configuration to use the new job names in - `https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Terraform/Base.latest.gitlab-ci.yml`. - For example: +There are two different causes for the error: - ```yaml - include: - - template: Terraform/Base.latest.gitlab-ci.yml +- In the case of `.init`, the error occurs because the init stage and jobs [were removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71188) from the templates, since they are no longer required. To resolve the syntax error, you can safely remove any jobs extending `.init`. +- For all other jobs, the reason for the failure is that the base jobs have been [renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67719): A `.terraform:` prefix has been added to every job name. For example, `.apply` became `.terraform:apply`. To fix this error, you can update the base job names. For example: - my-Terraform-job: - extends: .terraform:init # The updated name. + ```diff + my-terraform-job: + - extends: .apply + + extends: .terraform:apply ``` ## Troubleshooting Terraform state @@ -80,7 +84,7 @@ This happens because the value of `$CI_JOB_TOKEN` is only valid for the duration As a workaround, use [http backend configuration variables](https://www.terraform.io/docs/language/settings/backends/http.html#configuration-variables) in your CI job, which is what happens behind the scenes when following the -[Get started using GitLab CI](terraform_state.md#get-started-using-gitlab-ci) instructions. +[Get started using GitLab CI](terraform_state.md#initialize-a-terraform-state-as-a-backend-by-using-gitlab-cicd) instructions. ### Error: "address": required field is not set diff --git a/doc/user/markdown.md b/doc/user/markdown.md index fc2f1de5ce2..bbcb0f62a5c 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -8,9 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > The abbreviation [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/24592) from `GFM` to `GLFM` in GitLab 14.10. -GitLab automatically renders Markdown content. For example, when you add a comment to an issue, -you type the text in the Markdown language. When you save the issue, the text is rendered -with a set of styles. These styles are described on this page. +When you enter text in the GitLab UI, GitLab assumes the text is in the Markdown language. +The text is rendered with a set of styles. These styles are called *GitLab Flavored Markdown*. For example, in Markdown, an unordered list looks like this: diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 59bb4a89b0b..fe7a41aa542 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -626,10 +626,10 @@ Use your own URLs to complete the following steps: docker pull gitlab.example.com/org/build/sample_project/cr:v2.9.1 ``` -NOTE: -For container registry authentication, use either a -[personal access token](../../profile/personal_access_tokens.md) or a -[deploy token](../../project/deploy_tokens/index.md). + NOTE: + For container registry authentication, use either a + [personal access token](../../profile/personal_access_tokens.md) or a + [deploy token](../../project/deploy_tokens/index.md). 1. Rename the images to match the new project name: @@ -695,7 +695,7 @@ There may be some errors not properly cached. Follow these steps to investigate `200 OK`, the body might have the error `AccessDenied`. This indicates a permission problem from the S3 side. -1. Ensure your S3 configuration has the `deleteObject` permisson scope. Here's an +1. Ensure your S3 configuration has the `deleteObject` permission scope. Here's an [example role for an S3 bucket](../../../administration/object_storage.md#iam-permissions). Once adjusted, trigger another tag deletion. You should be able to successfully delete tags. diff --git a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md index 5f678a661f8..9dcb4d7127d 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md +++ b/doc/user/packages/container_registry/reduce_container_registry_data_transfer.md @@ -7,11 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Reduce Container Registry data transfers **(FREE)** Depending on the frequency with which images or tags are downloaded from the Container Registry, -data transfers can exceed the GitLab limit. This page offers several recommendations and tips for +data transfers can exceed the GitLab.com limit. This page offers several recommendations and tips for reducing the amount of data you transfer with the Container Registry. -are downloaded from the Container Registry, data transfers can exceed the GitLab limit. This page -offers several recommendations and tips for reducing the amount of data you transfer with the -Container Registry. ## Check data transfer use @@ -115,9 +112,15 @@ images can be specified as a cache source by using multiple `--cache-from` argum up your builds and reduce the amount of data transferred. For more information, see the [documentation on Docker layer caching](../../../ci/docker/using_docker_build.md#make-docker-in-docker-builds-faster-with-docker-layer-caching). +## Check automation frequency + +We often create automation scripts bundled into container images to perform regular tasks on specific intervals. +You can reduce the frequency of those intervals in cases where the automation is pulling container images from +the GitLab Registry to a service outside of GitLab.com. + ## Move to GitLab Premium or Ultimate -GitLab data transfer limits are set at the tier level. If you need a higher limit, consider +GitLab.com data transfer limits are set at the tier level. If you need a higher limit, consider upgrading to [GitLab Premium or Ultimate](https://about.gitlab.com/upgrade/). ## Purchase additional data transfer diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index 2cfe99876fa..7b52a6a8ce3 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -24,6 +24,7 @@ which doesn't include the Container Registry. To track work on this, see the epi > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15398) in GitLab 12.8. > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/218737) from "expiration policy" to "cleanup policy" in GitLab 13.2. +> - [Required permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/350682) changed from developer to maintainer in GitLab 15.0. The cleanup policy is a scheduled job you can use to remove tags from the Container Registry. For the project where it's defined, tags matching the regex pattern are removed. @@ -158,12 +159,8 @@ Here are examples of regex patterns you may want to use: ### Set cleanup limits to conserve resources > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/288812) in GitLab 13.9 [with a flag](../../../administration/feature_flags.md) named `container_registry_expiration_policies_throttling`. Disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80815) in GitLab 14.9. - -FLAG: -By default this feature is available in GitLab 14.9. To disable the feature, an administrator can -[disable the feature flag](../../../administration/feature_flags.md) -named `container_registry_expiration_policies_throttling`. +> - [Enabled by default](https://gitlab.com/groups/gitlab-org/-/epics/2270) in GitLab 14.9. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/84996) the feature flag `container_registry_expiration_policies_throttling` in GitLab 15.0. Cleanup policies are executed as a background process. This process is complex, and depending on the number of tags to delete, the process can take time to finish. diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 5e66c8ed7a5..af54d928bec 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -37,7 +37,8 @@ For a list of planned additions, view the To enable or turn off the Dependency Proxy for a group: -1. Go to your group's **Settings > Packages & Registries**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Packages & Registries**. 1. Expand the **Dependency Proxy** section. 1. To enable the proxy, turn on **Enable Proxy**. To turn it off, turn the toggle off. @@ -49,7 +50,8 @@ for the entire GitLab instance. To view the Dependency Proxy: -- Go to your group's **Packages & Registries > Dependency Proxy**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Packages & Registries > Dependency Proxy**. The Dependency Proxy is not available for projects. @@ -63,17 +65,8 @@ Prerequisites: ### Authenticate with the Dependency Proxy -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) in GitLab 13.7. -> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](../../../administration/packages/dependency_proxy.md#disabling-authentication). **(FREE SELF)** - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. -The requirement to authenticate is a breaking change added in 13.7. An [administrator can temporarily -disable it](../../../administration/packages/dependency_proxy.md#disabling-authentication) if it -has disrupted your existing Dependency Proxy usage. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) in GitLab 13.7 [with a flag](../../../administration/feature_flags.md) named `dependency_proxy_for_private_groups`. Enabled by default. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/276777) the feature flag `dependency_proxy_for_private_groups` in GitLab 15.0. Because the Dependency Proxy is storing Docker images in a space associated with your group, you must authenticate against the Dependency Proxy. @@ -182,8 +175,9 @@ You can also use [custom CI/CD variables](../../../ci/variables/index.md#custom- To store a Docker image in Dependency Proxy storage: -1. Go to your group's **Packages & Registries > Dependency Proxy**. -1. Copy the **Dependency Proxy URL**. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Packages & Registries > Dependency Proxy**. +1. Copy the **Dependency Proxy image prefix**. 1. Use one of these commands. In these examples, the image is `alpine:latest`. 1. You can also pull images by digest to specify exactly which version of an image to pull. diff --git a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md index cd04d2e696b..839684da875 100644 --- a/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md +++ b/doc/user/packages/dependency_proxy/reduce_dependency_proxy_storage.md @@ -26,6 +26,8 @@ image or tag from Docker Hub. ## Cleanup policies +> [Required permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/350682) changed from developer to maintainer in GitLab 15.0. + ### Enable cleanup policies from within GitLab > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340777) in GitLab 14.6 diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index 9dc859a37e2..37e1f0c3eb1 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -101,7 +101,8 @@ API or the UI. #### Do not allow duplicate Generic packages -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/293755) in GitLab 13.12. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/293755) in GitLab 13.12. +> - [Required permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/350682) changed from developer to maintainer in GitLab 15.0. To prevent users from publishing duplicate generic packages, you can use the [GraphQl API](../../../api/graphql/reference/index.md#packagesettings) or the UI. diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index 29455fdbb35..d2edfcb94c5 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -106,6 +106,11 @@ the scope set to `api` or `read_api`. Open your [`~/.netrc`](https://everything.curl.dev/usingcurl/netrc) file and add the following text. Replace the variables in `< >` with your values. +WARNING: +If you use an environment variable called `NETRC`, Go will use its value +as a filename and ignore `~/.netrc`. If you intend to use `~/.netrc` in +the GitLab CI **do not use `NETRC` as an environment variable name**. + ```plaintext machine <url> login <username> password <token> ``` diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 6a515b78fc1..1e3b5bdc323 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -184,7 +184,7 @@ published to the GitLab Package Registry. Project name (default: test): ``` -1. Enter a project name or press Enter to use the directory name as project name. +1. Enter a project name or press <kbd>Enter</kbd> to use the directory name as project name. ## Authenticate to the Package Registry with Maven @@ -617,7 +617,8 @@ To delete these older package versions, consider using the Packages API or the U #### Do not allow duplicate Maven packages -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296895) in GitLab 13.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296895) in GitLab 13.9. +> - [Required permissions](https://gitlab.com/gitlab-org/gitlab/-/issues/350682) changed from developer to maintainer in GitLab 15.0. To prevent users from publishing duplicate Maven packages, you can use the [GraphQl API](../../../api/graphql/reference/index.md#packagesettings) or the UI. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 4d46032d229..eee6d55a3ce 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -325,7 +325,8 @@ python -m twine upload --repository <source_name> dist/<package_file> ### Publishing packages with the same name or version You cannot publish a package if a package of the same name and version already exists. -You must delete the existing package first. If you attempt to publish the same package +You must [delete the existing package](../../packages/package_registry/reduce_package_registry_storage.md#delete-a-package) first. +If you attempt to publish the same package more than once, a `400 Bad Request` error occurs. ## Install a PyPI package diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 2282a7d876e..d28cf75d11f 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -67,13 +67,12 @@ The following table lists project permissions available for each role: | [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/index.md#on-demand-scans) | | | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>Manage [security policy](application_security/policies/index.md) | | | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>View [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>View [threats list](application_security/threat_monitoring/index.md#threat-monitoring) | | | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>Create a [CVE ID Request](application_security/cve_id_request.md) | | | | ✓ | ✓ | | [Application security](application_security/index.md):<br>Create or assign [security policy project](application_security/policies/index.md) | | | | | ✓ | | [Clusters](infrastructure/clusters/index.md):<br>View [pod logs](project/clusters/kubernetes_pod_logs.md) | | | ✓ | ✓ | ✓ | | [Clusters](infrastructure/clusters/index.md):<br>View clusters | | | ✓ | ✓ | ✓ | | [Clusters](infrastructure/clusters/index.md):<br>Manage clusters | | | | ✓ | ✓ | -| [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete [cleanup policies](packages/container_registry/index.md#delete-images-by-using-a-cleanup-policy) | | | ✓ | ✓ | ✓ | +| [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete [cleanup policies](packages/container_registry/index.md#delete-images-by-using-a-cleanup-policy) | | | | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Push an image to the Container Registry | | | ✓ | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (*20*) | ✓ (*20*) | ✓ | ✓ | ✓ | | [Container Registry](packages/container_registry/index.md):<br>Remove a Container Registry image | | | ✓ | ✓ | ✓ | @@ -144,7 +143,7 @@ The following table lists project permissions available for each role: | [Projects](project/index.md):<br>Create [snippets](snippets.md) | | ✓ | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>Manage labels | | ✓ | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>View [project traffic statistics](../api/project_statistics.md) | | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Create, edit, delete [milestones](project/milestones/index.md). | | | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Create, edit, delete [milestones](project/milestones/index.md). | | ✓ | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>Create, edit, delete [releases](project/releases/index.md) | | | ✓ (*12*) | ✓ (*12*) | ✓ (*12*) | | [Projects](project/index.md):<br>Create, edit [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ | | [Projects](project/index.md):<br>Enable [Review Apps](../ci/review_apps/index.md) | | | ✓ | ✓ | ✓ | @@ -391,6 +390,7 @@ The following table lists group permissions available for each role: | Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | | Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | | Delete [packages](packages/index.md) | | | | ✓ | ✓ | +| Create/edit/delete [Maven and generic package duplicate settings](packages/generic_packages/index.md#do-not-allow-duplicate-generic-packages) | | | | ✓ | ✓ | | Pull a Container Registry image | ✓ (7) | ✓ | ✓ | ✓ | ✓ | | Remove a Container Registry image | | | ✓ | ✓ | ✓ | | View [Group DevOps Adoption](group/devops_adoption/index.md) | | ✓ | ✓ | ✓ | ✓ | @@ -398,11 +398,12 @@ The following table lists group permissions available for each role: | View [Productivity analytics](analytics/productivity_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | Create and edit [group wiki](project/wiki/group.md) pages | | | ✓ | ✓ | ✓ | | Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | -| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | -| Create/edit/delete iterations | | | ✓ | ✓ | ✓ | +| Create/edit/delete group milestones | | ✓ | ✓ | ✓ | ✓ | +| Create/edit/delete iterations | | ✓ | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | -| Enable/disable a dependency proxy | | | ✓ | ✓ | ✓ | +| Enable/disable a dependency proxy | | | | ✓ | ✓ | | Purge the dependency proxy for a group | | | | | ✓ | +| Create/edit/delete dependency proxy [cleanup policies](packages/dependency_proxy/reduce_dependency_proxy_storage.md#cleanup-policies) | | | | ✓ | ✓ | | Use [security dashboard](application_security/security_dashboard/index.md) | | | ✓ | ✓ | ✓ | | View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | | Create subgroup | | | | ✓ (1) | ✓ | @@ -445,7 +446,7 @@ The following table lists group permissions available for each role: ### Subgroup permissions When you add a member to a subgroup, they inherit the membership and -permission level from the parent group(s). This model allows access to +permission level from the parent groups. This model allows access to nested groups if you have membership in one of its parents. To learn more, read through the documentation on @@ -549,7 +550,7 @@ Auditor users are given read-only access to all projects, groups, and other resources on the GitLab instance. An Auditor user should be able to access all projects and groups of a GitLab instance -with the permissions described on the documentation on [auditor users permissions](../administration/auditor_users.md#permissions-and-restrictions-of-an-auditor-user). +with the permissions described on the documentation on [auditor users permissions](../administration/auditor_users.md#auditor-user-permissions-and-restrictions). [Read more about Auditor users.](../administration/auditor_users.md) diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index a86968654c7..d7eb5040416 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -381,6 +381,34 @@ Each of these contribution events is tracked: - Design - Wiki page +### Retrieve user activity as a feed + +GitLab provides RSS feeds of user activity. To subscribe to the +RSS feed of a user's activity: + +1. Go to the [user's profile](#access-your-user-profile). +1. In the top right, select the feed symbol **{rss}** to display the results as an RSS feed in Atom format. + +The URL of the result contains both a feed token, and +the user's activity that you're authorized to view. +You can add this URL to your feed reader. + +### Reset the user activity feed token + +Feed tokens are sensitive and can reveal information from confidential issues. +If you think your feed token has been exposed, you should reset it. + +To reset your feed token: + +1. On the top bar, in the top right corner, select your avatar. +1. Select **Edit profile**. +1. On the left sidebar, select **Access Tokens**. +1. Scroll down. In the **Feed token** section, select the + **reset this token** link. +1. On the confirmation box, select **OK**. + +A new token is generated. + ## Troubleshooting ### Why do I keep getting signed out? diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index d0e9b427f1c..3eda363d108 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -7,6 +7,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Notification emails **(FREE)** +> Enhanced email styling [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78604) in GitLab 14.9 [with a feature flag](../../administration/feature_flags.md) named `enhanced_notify_css`. Disabled by default. +> Enhanced email styling [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/355907) in GitLab 14.9. +> Enhanced email styling [enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/355907) in GitLab 15.0. + Stay informed about what's happening in GitLab with email notifications. You can receive updates about activity in issues, merge requests, epics, and designs. @@ -24,7 +28,7 @@ You might receive notifications for one of the following reasons: or edit, or someone mentions <sup>1</sup> you. - You've [enabled notifications in an issue, merge request, or epic](#notifications-on-issues-merge-requests-and-epics). - You've configured notifications for the [project](#change-level-of-project-notifications) or [group](#group-notifications). -- You're subscribed to group or project pipeline notifications via the pipeline emails [integration](../project/integrations/overview.md). +- You're subscribed to group or project pipeline notifications via the pipeline emails [integration](../project/integrations/index.md). 1. GitLab doesn't send a notification when [a comment is edited to include a user mention](../discussions/index.md#editing-a-comment-to-add-a-mention). diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 4c132094d24..09e71ce9133 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -23,7 +23,7 @@ Personal access tokens are: - Required when [two-factor authentication (2FA)](account/two_factor_authentication.md) is enabled. - Used with a GitLab username to authenticate with GitLab features that require usernames. For example, - [GitLab managed Terraform state backend](../infrastructure/iac/terraform_state.md#using-a-gitlab-managed-terraform-state-backend-as-a-remote-data-source) + [GitLab-managed Terraform state backend](../infrastructure/iac/terraform_state.md#use-your-gitlab-backend-as-a-remote-data-source) and [Docker container registry](../packages/container_registry/index.md#authenticate-with-the-container-registry), - Similar to [project access tokens](../project/settings/project_access_tokens.md) and [group access tokens](../group/settings/group_access_tokens.md), but are attached to a user rather than a project or group. @@ -109,9 +109,7 @@ Personal access tokens expire on the date you define, at midnight UTC. - GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that expire in the next seven days. The owners of these tokens are notified by email. - GitLab runs a check at 02:00 AM UTC every day to identify personal access tokens that expire on the current date. The owners of these tokens are notified by email. - In GitLab Ultimate, administrators can - [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens). -- In GitLab Ultimate, administrators can choose whether or not to - [enforce personal access token expiration](../admin_area/settings/account_and_limit_settings.md#allow-expired-personal-access-tokens-to-be-used-deprecated). + [limit the lifetime of access tokens](../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens). ## Create a personal access token programmatically **(FREE SELF)** diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index ecd6e83efa1..c654eb2b0e8 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -91,8 +91,8 @@ Introduced in GitLab 13.6, the themes [Solarized](https://gitlab.com/gitlab-org/ ## Diff colors -A diff compares the old/removed content with the new/added content (e.g. when -[reviewing a merge request](../project/merge_requests/reviews/index.md#review-a-merge-request) or in a +A diff compares the old/removed content with the new/added content (e.g. when +[reviewing a merge request](../project/merge_requests/reviews/index.md#review-a-merge-request) or in a [Markdown inline diff](../markdown.md#inline-diff)). Typically, the colors red and green are used for removed and added lines in diffs. The exact colors depend on the selected [syntax highlighting theme](#syntax-highlighting-theme). diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 344caed7449..a76517a7341 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -25,9 +25,6 @@ If there is a problem with the new version of the application, only a small percentage of users are affected and the change can either be fixed or quickly reverted. -Leveraging [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments), visualize your canary -deployments right inside the [deploy board](deploy_boards.md), without the need to leave GitLab. - ## Use cases Canary deployments can be used when you want to ship features to only a portion of @@ -45,38 +42,7 @@ may want to consider [setting `service.spec.sessionAffinity` to `ClientIP` in your Kubernetes service definitions](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies), but that is beyond the scope of this document. -## Enabling Canary Deployments - -Canary deployments require that you properly configure deploy boards: - -1. Follow the steps to [enable deploy boards](deploy_boards.md#enabling-deploy-boards). -1. To track canary deployments you must label your Kubernetes deployments and - pods with `track: canary`. To get started quickly, you can use the [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy) - template for canary deployments that GitLab provides. - -Depending on the deploy, the label should be either `stable` or `canary`. -GitLab assumes the track label is `stable` if the label is blank or missing. -Any other track label is considered `canary` (temporary). -This allows GitLab to discover whether a deployment is stable or canary (temporary). - -Once all of the above are set up and the pipeline has run at least once, -Go to the environments page under **Pipelines > Environments**. -As the pipeline executes, deploy boards clearly mark canary pods, enabling -quick and clear insight into the status of each environment and deployment. - -Canary deployments are marked with a yellow dot in the deploy board so that you -can quickly notice them. - - - -### Advanced traffic control with Canary Ingress (DEPRECATED) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in GitLab 13.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +## Advanced traffic control with Canary Ingress Canary deployments can be more strategic with [Canary Ingress](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary), which is an advanced traffic routing service that controls incoming HTTP @@ -84,7 +50,7 @@ requests between stable and canary deployments based on factors such as weight, and others. GitLab uses this service in its [Auto Deploy architecture](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#v2-chart-resource-architecture) to let users quickly and safely roll out their new deployments. -#### How to set up a Canary Ingress in a canary deployment +### How to set up a Canary Ingress in a canary deployment A Canary Ingress is installed by default if your Auto DevOps pipeline uses [`v2.0.0+` of `auto-deploy-image`](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#verify-dependency-versions). @@ -106,14 +72,51 @@ Here's an example setup flow from scratch: 1. [Run a new Auto DevOps pipeline](../../ci/pipelines/index.md#run-a-pipeline-manually) and make sure that the `canary` job succeeds and creates a canary deployment with Canary Ingress. -#### How to check the current traffic weight on a Canary Ingress +### Show Canary Ingress deployments on deploy boards (deprecated) + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in GitLab 13.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +To view canary deployments you must properly configure deploy boards: + +1. Follow the steps to [enable deploy boards](deploy_boards.md#enabling-deploy-boards). +1. To track canary deployments you must label your Kubernetes deployments and + pods with `track: canary`. To get started quickly, you can use the [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy) + template for canary deployments that GitLab provides. + +Depending on the deploy, the label should be either `stable` or `canary`. +GitLab assumes the track label is `stable` if the label is blank or missing. +Any other track label is considered `canary` (temporary). +This allows GitLab to discover whether a deployment is stable or canary (temporary). + +Once all of the above are set up and the pipeline has run at least once, +Go to the environments page under **Pipelines > Environments**. +As the pipeline executes, deploy boards clearly mark canary pods, enabling +quick and clear insight into the status of each environment and deployment. + +Canary deployments are marked with a yellow dot in the deploy board so that you +can quickly notice them. + + + +#### How to check the current traffic weight on a Canary Ingress (deprecated) + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. 1. Visit the [deploy board](../../user/project/deploy_boards.md). 1. View the current weights on the right.  -#### How to change the traffic weight on a Canary Ingress +#### How to change the traffic weight on a Canary Ingress (deprecated) + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. You can change the traffic weight in your environment's deploy board by using [GraphiQL](../../api/graphql/getting_started.md#graphiql), or by sending requests to the [GraphQL API](../../api/graphql/getting_started.md#command-line). diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 82106c9d1a9..935bc01bae7 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -226,7 +226,7 @@ on the running pod. If you are using a self-managed GitLab instance, you need to configure Amazon credentials. GitLab uses these credentials to assume an Amazon IAM role to create your cluster. -Create an IAM user and ensure it has permissions to assume the role(s) that +Create an IAM user and ensure it has permissions to assume the roles that your users need to create EKS clusters. For example, the following policy document allows assuming a role whose name starts with diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index f2d537513b7..c55c11151ce 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -27,7 +27,7 @@ To add any cluster to GitLab, you need: - Either a GitLab.com account or an account for a self-managed installation running GitLab 12.5 or later. - The Maintainer role for group-level and project-level clusters. -- Access to the Admin area for instance-level clusters. +- Access to the Admin Area for instance-level clusters. - A Kubernetes cluster. - Cluster administration access to the cluster with `kubectl`. @@ -230,3 +230,22 @@ kubectl create clusterrolebinding permissive-binding \ --user=kubelet \ --group=system:serviceaccounts ``` + +## Troubleshooting + +### `There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid` + +If you encounter this error while connecting a Kubernetes cluster, ensure you're +properly pasting the service token. Some shells may add a line break to the +service token, making it invalid. Ensure that there are no line breaks by +pasting your token into an editor and removing any additional spaces. + +You may also experience this error if your certificate is not valid. To check that your certificate's +subject alternative names contain the correct domain for your cluster's API, run this command: + +```shell +echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | +openssl x509 -inform pem -noout -text +``` + +The `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index a4f6dc325c8..8cdd1792e7f 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -10,60 +10,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -To create a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). - -NOTE: -Every new Google Cloud Platform (GCP) account receives -[$300 in credit upon sign up](https://console.cloud.google.com/freetrial). -In partnership with Google, GitLab is able to offer an additional $200 for new GCP -accounts to get started with the GitLab integration with Google Kubernetes Engine. -[Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) -to apply for credit. - -NOTE: -Watch the webcast [Scalable app deployment with GitLab and Google Cloud Platform](https://about.gitlab.com/webcast/scalable-app-deploy/) -and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform (GCP) -in a few clicks. - -## Create new cluster - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. - -As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac) -to **safely create new clusters from GitLab**. - -Creating clusters from GitLab using cluster certificates is still available on the -GitLab UI but was **deprecated** in GitLab 14.0 and is scheduled for removal in -GitLab 15.0. We don't recommend using this method. - -You can create a new cluster hosted in EKS, GKE, on premises, and with other -providers using cluster certificates: - -- [New cluster hosted on Google Kubernetes Engine (GKE)](add_gke_clusters.md). -- [New cluster hosted on Amazon Elastic Kubernetes Service (EKS)](add_eks_clusters.md). - -To host them on premises and with other providers, you can use Terraform -or your preferred tool of choice to create and connect a cluster with GitLab. -The [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs/resources/project_cluster) -supports connecting existing clusters using the certificate-based connection method. - -## Add existing cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to connect your cluster to GitLab. - -Alternatively, you can [add an existing cluster](add_existing_cluster.md) -through the certificate-based method, but we don't recommend using this method for [security implications](../../infrastructure/clusters/connect/index.md#security-implications-for-clusters-connected-with-certificates). - -## Configure your cluster - -As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) -to configure your cluster. +To create and manage a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). ## Disable a cluster -When you successfully create a new Kubernetes cluster or add an existing -one to GitLab, the cluster connection to GitLab becomes enabled. To disable it: +When you successfully connect an existing cluster using cluster certificates, the cluster connection to GitLab becomes enabled. To disable it: 1. Go to your: - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. @@ -95,26 +46,3 @@ To remove the Kubernetes cluster integration: 1. Go to your cluster details page. 1. Select the **Advanced Settings** tab. 1. Select either **Remove integration** or **Remove integration and resources**. - -## Access controls - -See [cluster access controls (RBAC or ABAC)](cluster_access.md). - -## Troubleshooting - -### There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid - -If you encounter this error while adding a Kubernetes cluster, ensure you're -properly pasting the service token. Some shells may add a line break to the -service token, making it invalid. Ensure that there are no line breaks by -pasting your token into an editor and removing any additional spaces. - -You may also experience this error if your certificate is not valid. To check that your certificate's -subject alternative names contain the correct domain for your cluster's API, run this: - -```shell -echo | openssl s_client -showcerts -connect kubernetes.example.com:443 2>/dev/null | -openssl x509 -inform pem -noout -text -``` - -Note that the `-connect` argument expects a `host:port` combination. For example, `https://kubernetes.example.com` would be `kubernetes.example.com:443`. diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 8ff0a275649..43ceb3673d8 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -28,7 +28,7 @@ Helm also creates additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application for details. -If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster), +If you are [adding an existing Kubernetes cluster](add_existing_cluster.md), ensure the token of the account has administrator privileges for the cluster. The resources created by GitLab differ depending on the type of cluster. diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index c1cdf754c11..fc41533b17c 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md). -To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_workflow.md). A Kubernetes cluster can be the destination for a deployment job. If diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index 9c5cc14f720..e295abf8d31 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -9,15 +9,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. > - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect your cluster to GitLab, use the [GitLab agent](../../../user/clusters/agent/index.md). To manage applications, use the [Cluster Project Management Template](../../../user/clusters/management_project_template.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `certificate_based_clusters`. + You can choose to allow GitLab to manage your cluster for you. If your cluster is managed by GitLab, resources for your projects are automatically created. See -the [Access controls](add_remove_clusters.md#access-controls) section for +the [Access controls](cluster_access.md) section for details about the created resources. If you choose to manage your own cluster, project-specific resources aren't created diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index b5e2a1bad51..b1158be9fb6 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -4,14 +4,23 @@ group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Kubernetes Logs (DEPRECATED) **(FREE)** +# Kubernetes Logs (DEPRECATED) **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4752) in GitLab 11.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26383) from GitLab Ultimate to GitLab Free 12.9. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/360182) behind a [feature flag](../../../administration/feature_flags.md) named `monitor_logging` in GitLab 15.0. Disabled by default. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: +This feature is in its end-of-life process. This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +It will be [removed completely](https://gitlab.com/gitlab-org/gitlab/-/issues/346485) in GitLab 15.2. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `monitor_logging` and the one named `certificate_based_clusters`. +On GitLab.com, this feature is not available. +This feature is not recommended for production use. GitLab makes it easy to view the logs of running pods in [connected Kubernetes clusters](index.md). By displaying the logs directly in GitLab diff --git a/doc/user/project/clusters/protect/container_host_security/index.md b/doc/user/project/clusters/protect/container_host_security/index.md deleted file mode 100644 index c897100f14e..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/index.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Host Security in GitLab provides Intrusion Detection and Prevention capabilities that can -monitor and (optionally) block activity inside the containers themselves. This is done by leveraging -an integration with Falco to provide the monitoring capabilities and an integration with Pod -Security Policies and AppArmor to provide blocking capabilities. - -## Overview - -Container Host Security can be used to monitor and block activity inside a container as well as to -enforce security policies across the entire Kubernetes cluster. Falco profiles allow for users to -define the activity they want to monitor for and detect. Among other things, this can include system -log entries, process starts, file activity, and network ports opened. AppArmor is used to block any -undesired activity via AppArmor profiles. These profiles are loaded into the cluster when -referenced by Pod Security Policies. - -By default, Container Host Security is deployed into the cluster in monitor mode only, with no -default profiles or rules running out-of-the-box. Activity monitoring and blocking begins only when -users define profiles for these technologies. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install the -Container Host Security capabilities. This guide shows the recommended way of installing Container -Host Security through the Cluster Management Project. However, it's also possible to do a manual -installation through our Helm chart. - -## Features - -- Prevent containers from starting as root. -- Limit the privileges and system calls available to containers. -- Monitor system logs, process starts, files read/written/deleted, and network ports opened. -- Optionally block processes from starting or files from being read/written/deleted. - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Host Security may function on Azure or self-managed Kubernetes instances, it isn't -officially tested and supported on those providers. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_host_security/) -for more information on the product direction of Container Host Security. diff --git a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md deleted file mode 100644 index af3128e3006..00000000000 --- a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Host Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Host Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Host Security. - -## Installation steps - -The following steps are recommended to install and use Container Host Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Falco](../../../../clusters/applications.md#install-falco-using-gitlab-cicd) - for activity monitoring. -1. [Install and configure AppArmor](../../../../clusters/applications.md#install-apparmor-using-gitlab-cicd) - for activity blocking. -1. [Configure Pod Security Policies](../../../../clusters/applications.md#using-podsecuritypolicy-in-your-deployments) - (required to be able to load AppArmor profiles). - -It's possible to install and manage Falco and AppArmor in other ways, such as installing them -manually in a Kubernetes cluster and then connecting it back to GitLab. These methods aren't -supported or documented. - -## Viewing the logs - -Falco logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l app=falco -``` - -## Troubleshooting - -### Trouble connecting to the cluster - -Your CI/CD pipeline may occasionally fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache) -1. If things still aren't working, a more assertive set of actions may help get things back to a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/#delete-a-stopped-environment) - in GitLab. - - Delete the relevant namespace in Kubernetes by running - `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/container_network_security/index.md b/doc/user/project/clusters/protect/container_network_security/index.md deleted file mode 100644 index b294859c660..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/index.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -Container Network Security in GitLab provides basic firewall functionality by leveraging Cilium -NetworkPolicies to filter traffic going in and out of the cluster as well as traffic between pods -inside the cluster. Container Network Security can be used to enforce L3, L4, and L7 policies and -can prevent an attacker with control over one pod from spreading laterally to access other pods in -the same cluster. Both Ingress and Egress rules are supported. - -By default, Cilium is deployed in Detection-only mode and only logs attack attempts. GitLab provides -a set of out-of-the-box policies as examples and to help users get started. These policies are -disabled by default, as they must usually be customized to match application-specific needs. - -## Installation - -See the [installation guide](quick_start_guide.md) for the recommended steps to install GitLab -Container Network Security. This guide shows the recommended way of installing Container Network -Security through the Cluster Management Project. However, it's also possible to install Cilium -manually through our Helm chart. - -## Features - -- GitLab managed installation of Cilium. -- Support for L3, L4, and L7 policies. -- Ability to export logs to a SIEM. -- Statistics page showing volume of packets processed and dropped over time (Ultimate users only). -- Management of NetworkPolicies through code in a project (Available for auto DevOps users only). -- Management of CiliumNetworkPolicies through a UI policy manager (Ultimate users only). - -## Supported container orchestrators - -Kubernetes v1.14+ is the only supported container orchestrator. OpenShift and other container -orchestrators aren't supported. - -## Supported Kubernetes providers - -The following cloud providers are supported: - -- Amazon EKS -- Google GKE - -Although Container Network Security may function on Azure or self-managed Kubernetes instances, it -isn't officially tested and supported on those providers. - -## Supported NetworkPolicies - -GitLab only supports the use of CiliumNetworkPolicies. Although generic Kubernetes NetworkPolicies -or other kinds of NetworkPolicies may work, GitLab doesn't test or support them. - -## Managing NetworkPolicies through GitLab vs your cloud provider - -Some cloud providers offer integrations with Cilium or offer other ways to manage NetworkPolicies in -Kubernetes. GitLab Container Network Security doesn't support deployments that have NetworkPolicies -managed by an external provider. By choosing to manage NetworkPolicies through GitLab, you can take -advantage of the following benefits: - -- Support for handling NetworkPolicy infrastructure as code. -- Full revision history and audit log of all changes made. -- Ability to revert back to a previous version at any time. - -## Roadmap - -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_network_security/) -for more information on the product direction of Container Network Security. diff --git a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md deleted file mode 100644 index 7671ed7eb73..00000000000 --- a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md +++ /dev/null @@ -1,230 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Getting started with Container Network Security **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Security is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The following steps are recommended for installing Container Network Security. - -## Installation steps - -The following steps are recommended to install and use Container Network Security through GitLab: - -1. [Install at least one runner and connect it to GitLab](https://docs.gitlab.com/runner/). -1. [Create a group](../../../../group/#create-a-group). -1. [Connect a Kubernetes cluster to the group](../../add_remove_clusters.md). -1. [Create a cluster management project and associate it with the Kubernetes cluster](../../../../clusters/management_project.md). - -1. Install and configure an Ingress node: - - - [Install the Ingress node via CI/CD (Cluster Management Project)](../../../../clusters/applications.md#install-ingress-using-gitlab-cicd). - - Navigate to the Kubernetes page and enter the [DNS address for the external endpoint](../../gitlab_managed_clusters.md#base-domain) - into the **Base domain** field on the **Details** tab. Save the changes to the Kubernetes - cluster. - -1. [Install and configure Cilium](#use-the-cluster-management-template-to-install-cilium). -1. Be sure to restart all pods that were running before Cilium was installed by running this command - in your cluster: - - `kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod` - - You can skip this step if `nodeinit.restartPods` is set to `true` on your Helm chart. - -It's possible to install and manage Cilium in other ways. For example, you could use the GitLab Helm -chart to install Cilium manually in a Kubernetes cluster, and then connect it back to GitLab. -However, such methods aren't documented or officially supported by GitLab. - -### Use the Cluster Management template to install Cilium - -[Cilium](https://cilium.io/) is a networking plug-in for Kubernetes that you can use to implement -support for [`NetworkPolicy`](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information, see [Network Policies](../../../../../topics/autodevops/stages.md#network-policy). - -You can use the [Cluster Management Project Template](../../../../clusters/management_project_template.md) -to install Cilium in your Kubernetes cluster. - -1. In your cluster management project, go to `helmfile.yaml` and uncomment `- path: applications/cilium/helmfile.yaml`. -1. In `applications/cilium/helmfile.yaml`, set `clusterType` to either `gke` or `eks` based on which Kubernetes provider your are using. - - ```yaml - environments: - default: - values: - # Set to "gke" or "eks" based on your cluster type - - clusterType: "" - ``` - -1. Merge or push these changes to the default branch of your cluster management project, -and [GitLab CI/CD](../../../../../ci/index.md) will automatically install Cilium. - -WARNING: -Installation and removal of the Cilium requires a **manual** -[restart](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-helm/#restart-unmanaged-pods) -of all affected pods in all namespaces to ensure that they are -[managed](https://docs.cilium.io/en/stable/operations/troubleshooting/#ensure-managed-pod) -by the correct networking plug-in. When Hubble is enabled, its related pod might require a -restart depending on whether it started prior to Cilium. For more information, see -[Failed Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#failed-deployment) -in the Kubernetes docs. - -NOTE: -Major upgrades might require additional setup steps. For more information, see -the official [upgrade guide](https://docs.cilium.io/en/stable/operations/upgrade/). - -Support for installing the Cilium application is provided by the -GitLab Container Security group. If you run into unknown issues, -[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at -least 2 people from the -[Container Security group](https://about.gitlab.com/handbook/product/categories/#container-security-group). - -### Configure the Cilium Helm chart - -You can customize Cilium's Helm variables by editing the `applications/cilium/values.yaml` -file in your cluster management project. Refer to the [Cilium Helm reference](https://docs.cilium.io/en/stable/helm-reference/) -for the available configuration options. - -By default, Cilium's -[audit mode](https://docs.cilium.io/en/stable/gettingstarted/policy-creation/#enable-policy-audit-mode) -is enabled. In audit mode, Cilium doesn't drop disallowed packets. You -can use `policy-verdict` log to observe policy-related decisions. You -can disable audit mode by setting `policyAuditMode: false` in -`applications/cilium/values.yaml`. - -The Cilium monitor log for traffic is logged out by the -`cilium-monitor` sidecar container. You can check these logs with the following command: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -You can disable the monitor log in `application/cilium/values.yaml`: - -```yaml -monitor: - enabled: false -``` - -The [Hubble](https://github.com/cilium/hubble) monitoring daemon is enabled by default -and it's set to collect per namespace flow metrics. This metrics are accessible on the -[Threat Monitoring](../../../../application_security/threat_monitoring/index.md) -dashboard. You can disable Hubble by adding the following to -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: false -``` - -You can also adjust Helm values for Hubble by using -`applications/cilium/values.yaml`: - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -## Managing Network Policies - -Managing NetworkPolicies through GitLab is advantageous over managing the policies in Kubernetes -directly. Kubernetes doesn't provide a GUI editor, a change control process, or a revision history. -Network Policies can be managed through GitLab in one of two ways: - -- Management through a YAML file in each application's project (for projects using Auto DevOps). For - more information, see the [Network Policy documentation](../../../../../topics/autodevops/stages.md#network-policy). -- Management through the GitLab Policy management UI (for projects not using Auto DevOps). For more - information, see the [Container Network Policy documentation](../../../../application_security/policies/index.md#container-network-policy) (Ultimate only). - -Each method has benefits and drawbacks: - -| | YAML method | UI method (Ultimate only) | -|--|:------------|:-------------------------------| -| **Benefits** | A change control process is possible by requiring [MR Approvals](../../../merge_requests/approvals/index.md). All changes are fully tracked and audited in the same way that Git tracks the history of any file in its repository. | The UI provides a simple rules editor for users who are less familiar with the YAML syntax of NetworkPolicies. This view is a live representation of the policies currently deployed in the Kubernetes cluster. The UI also allows for multiple network policies to be created per environment. | -| **Drawbacks** | Only one network policy can be deployed per environment (although that policy can be as detailed as needed). Also, if changes were made in Kubernetes directly rather than through the `auto-deploy-values.yaml` file, the YAML file's contents don't represent the actual state of policies deployed in Kubernetes. | Policy changes aren't audited and a change control process isn't available. | - -Users are encouraged to choose one of the two methods to manage their policies. If users attempt to -use both methods simultaneously, when the application project pipeline runs the contents of the -NetworkPolicy in the `auto-deploy-values.yaml` file may override policies configured in the UI -editor. - -## Monitoring throughput **(ULTIMATE)** - -To view statistics for Container Network Security, you must follow the installation steps above and -configure GitLab integration with Prometheus. Also, if you use custom Helm values for Cilium, you -must enable Hubble with flow metrics for each namespace by adding the following lines to -your [Cilium values](#use-the-cluster-management-template-to-install-cilium): - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -Additional information about the statistics page is available in the -[documentation that describes the Threat Management UI](../../../../application_security/policies/index.md#container-network-policy). - -## Forwarding logs to a SIEM - -Cilium logs can be forwarded to a SIEM or an external logging system through syslog protocol by -installing and configuring Fluentd. Fluentd can be installed through the GitLab -[Cluster Management Project](../../../../clusters/applications.md#install-fluentd-using-gitlab-cicd). - -## Viewing the logs - -Cilium logs can be viewed by running the following command in your Kubernetes cluster: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -## Troubleshooting - -### Traffic is not being blocked as expected - -By default, Cilium is installed in Audit mode only, meaning that NetworkPolicies log policy -violations but don't block any traffic. To set Cilium to Blocking mode, you must add the following -lines to the `applications/cilium/values.yaml` file in your cluster management project: - -```yaml -policyEnforcementMode: "always" - -monitor: - eventTypes: ["drop", "policy-verdict"] -``` - -### Traffic is not being allowed as expected - -Keep in mind that when Cilium is set to blocking mode (rather than Audit mode), NetworkPolicies -operate on an allow-list basis. If one or more NetworkPolicies apply to a node, then all traffic -that doesn't match at least one Policy is blocked. To resolve, add NetworkPolicies defining the -traffic that you want to allow in the node. - -### Trouble connecting to the cluster - -Occasionally, your CI/CD pipeline may fail or have trouble connecting to the cluster. Here are some -initial troubleshooting steps that resolve the most common problems: - -1. [Clear the cluster cache](../../gitlab_managed_clusters.md#clearing-the-cluster-cache). -1. If things still aren't working, a more assertive set of actions may help get things back into a - good state: - - - Stop and [delete the problematic environment](../../../../../ci/environments/index.md#delete-a-stopped-environment) in GitLab. - - Delete the relevant namespace in Kubernetes by running `kubectl delete namespaces <insert-some-namespace-name>` in your Kubernetes cluster. - - Rerun the application project pipeline to redeploy the application. - -**Related documentation links:** - -- [Cluster Management Project](../../../../clusters/management_project.md) diff --git a/doc/user/project/clusters/protect/index.md b/doc/user/project/clusters/protect/index.md deleted file mode 100644 index 6b89f7f1557..00000000000 --- a/doc/user/project/clusters/protect/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Protecting your deployed applications **(FREE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -The Container Network Security and Container Host Security features are in their end-of-life -processes. They're -[deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -GitLab makes it straightforward to protect applications deployed in [connected Kubernetes clusters](index.md). -These protections are available in the Kubernetes network layer and in the container itself. At -the network layer, the Container Network Security capabilities in GitLab provide basic firewall -functionality by leveraging Cilium NetworkPolicies to filter traffic going in and out of the cluster -and traffic between pods inside the cluster. Inside the container, Container Host Security provides -Intrusion Detection and Prevention capabilities that can monitor and block activity inside the -containers themselves. - -## Capabilities - -The following capabilities are available to protect deployed applications in Kubernetes: - -- Container Network Security - - [Overview](container_network_security/index.md) - - [Installation guide](container_network_security/quick_start_guide.md) -- Container Host Security - - [Overview](container_host_security/index.md) - - [Installation guide](container_host_security/quick_start_guide.md) diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 9d623518f72..086e1fccf6c 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -40,8 +40,8 @@ for an overview of how this is accomplished in GitLab! To create an executable runbook, you need: - **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the - applications. The simplest way to get started is to add a cluster using one - of the [GitLab integrations](../add_remove_clusters.md#create-new-cluster). + applications. The simplest way to get started is to connect a cluster using the + [GitLab agent](../../../clusters/agent/index.md). - **Ingress** - Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications. - **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index cf571abbf8a..93bc41dc24c 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -2,506 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Deploying AWS Lambda function using GitLab CI/CD **(FREE)** +# Deploying AWS Lambda function using GitLab CI/CD (removed) **(FREE)** -GitLab allows users to easily deploy AWS Lambda functions and create rich serverless applications. - -GitLab supports deployment of AWS Lambda functions through GitLab CI/CD using the following Serverless frameworks: - -- [Serverless Framework with AWS](#serverless-framework) -- [AWS' Serverless Application Model (SAM)](#aws-serverless-application-model) - -## Serverless Framework - -The [Serverless Framework can deploy to AWS](https://www.serverless.com/framework/docs/providers/aws/). - -We have prepared an example with a step-by-step guide to create a simple function and deploy it on AWS. - -Additionally, in the [How To section](#how-to), you can read about different use cases like: - -- Running a function locally. -- Working with secrets. -- Setting up CORS. - -Alternatively, you can quickly [create a new project with a template](../../working_with_projects.md#create-a-project). The [`Serverless Framework/JS` template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) already includes all parts described below. - -### Example - -This example shows you how to: - -1. Create a basic AWS Lambda Node.js function. -1. Link the function to an API Gateway `GET` endpoint. - -#### Steps - -The example consists of the following steps: - -1. Creating a Lambda handler function. -1. Creating a `serverless.yml` file. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your function. -1. Testing the deployed function. - -Lets take it step by step. - -#### Creating a Lambda handler function - -Your Lambda function is the primary handler of requests. In this case, create a very simple Node.js `hello` function: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -Place this code in the file `src/handler.js`. - -`src` is the standard location for serverless functions, but is customizable should you desire that. - -In our case, `module.exports.hello` defines the `hello` handler to reference later in the `serverless.yml`. - -You can learn more about the [AWS Lambda Node.js function handler](https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html) and all its various options in its documentation. - -#### Creating a `serverless.yml` file - -In the root of your project, create a `serverless.yml` file containing configuration specifics for the Serverless Framework. - -Put the following code in the file: - -```yaml -service: gitlab-example -provider: - name: aws - runtime: nodejs14.x - -functions: - hello: - handler: src/handler.hello - events: - - http: GET hello -``` - -Our function contains a handler and a event. - -The handler definition provisions the Lambda function using the source code located `src/handler.hello`. - -The `events` declaration creates an AWS API Gateway `GET` endpoint to receive external requests and hand them over to the Lambda function via a service integration. - -You can read more about the [available properties and additional configuration possibilities](https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml/) of the Serverless Framework. - -#### Crafting the `.gitlab-ci.yml` file - -In a `.gitlab-ci.yml` file in the root of your project, place the following code: - -```yaml -image: node:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - npm config set prefix /usr/local - - npm install -g serverless - script: - - serverless deploy --stage production --verbose - environment: production -``` - -This example code does the following: - -1. Uses the `node:latest` image for all GitLab CI/CD builds -1. The `deploy` stage: - - Installs the Serverless Framework. - - Deploys the serverless function to your AWS account using the AWS credentials - defined above. - -#### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. -For more information please see [Create a custom variable in the UI](../../../../ci/variables/index.md#custom-variables-validated-by-gitlab). - - The AWS credentials you provide must include IAM policies that provision correct - access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -#### Deploying your function - -`git push` the changes to your GitLab repository and the GitLab build pipeline deploys your function. - -Your GitLab deploy stage log contains output containing your AWS Lambda endpoint URL, -with log lines similar to this: - -```plaintext -endpoints: - GET - https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello -``` - -#### Manually testing your function - -Running the following `curl` command should trigger your function. -Your URL should be the one retrieved from the GitLab deploy stage log: - -```shell -curl "https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello" -``` - -That should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -Hooray! You now have a AWS Lambda function deployed via GitLab CI/CD. - -Nice work! - -### How To - -In this section, we show you how to build on the basic example to: - -- Run the function locally. -- Set up secret variables. -- Set up CORS. - -#### Running function locally - -The `serverless-offline` plugin allows to run your code locally. To run your code locally: - -1. Add the following to your `serverless.yml`: - - ```yaml - plugins: - - serverless-offline - ``` - -1. Start the service by running the following command: - - ```shell - serverless offline - ``` - -Running the following `curl` command should trigger your function. - -```shell -curl "http://localhost:3000/hello" -``` - -It should output: - -```json -{ - "message": "Your function executed successfully!" -} -``` - -#### Secret variables - -Secrets are injected into your functions using environment variables. - -By defining variables in the provider section of the `serverless.yml`, you add them to -the environment of the deployed function: - -```yaml -provider: - # Other configuration omitted - # ... - environment: - A_VARIABLE: ${env:A_VARIABLE} -``` - -From there, you can reference them in your functions as well. -Remember to add `A_VARIABLE` to your GitLab CI/CD variables under **Settings > CI/CD > Variables** to be picked up and deployed with your function. - -NOTE: -Anyone with access to the AWS environment may be able to see the values of those -variables persisted in the lambda definition. - -#### Setting up CORS - -If you want to set up a web page that makes calls to your function, like we have done in the [template](https://gitlab.com/gitlab-org/project-templates/serverless-framework/), you need to deal with the Cross-Origin Resource Sharing (CORS). - -The quick way to do that is to add the `cors: true` flag to the HTTP endpoint in your `serverless.yml`: - -```yaml -functions: - hello: - handler: src/handler.hello - events: - - http: # Rewrite this part to enable CORS - path: hello - method: get - cors: true # <-- CORS here -``` - -You also need to return CORS specific headers in your function response: - -```javascript -'use strict'; - -module.exports.hello = async event => { - return { - statusCode: 200, - headers: { - // Uncomment the line below if you need access to cookies or authentication - // 'Access-Control-Allow-Credentials': true, - 'Access-Control-Allow-Origin': '*' - }, - body: JSON.stringify( - { - message: 'Your function executed successfully!' - }, - null, - 2 - ), - }; -}; -``` - -For more information, see the [Your CORS and API Gateway survival guide](https://www.serverless.com/blog/cors-api-gateway-survival-guide/) -blog post written by the Serverless Framework team. - -#### Writing automated tests - -The [Serverless Framework](https://gitlab.com/gitlab-org/project-templates/serverless-framework/) -example project shows how to use Jest, Axios, and `serverless-offline` plugin to do -automated testing of both local and deployed serverless function. - -### Examples and template - -The example code is available: - -- As a [clonable repository](https://gitlab.com/gitlab-org/serverless/examples/serverless-framework-js). -- In a version with [tests and secret variables](https://gitlab.com/gitlab-org/project-templates/serverless-framework/). - -You can also use a [template](../../working_with_projects.md#create-a-project) -(based on the version with tests and secret variables) from within the GitLab UI (see -the `Serverless Framework/JS` template). - -## AWS Serverless Application Model - -AWS Serverless Application Model is an open source framework for building serverless -applications. It makes it easier to build and deploy serverless applications. For more -details, please take a look at AWS documentation on [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/). - -### Deploying AWS Lambda function using AWS SAM and GitLab CI/CD - -GitLab allows developers to build and deploy serverless applications using the combination of: - -- [AWS Serverless Application Model (AWS SAM)](https://aws.amazon.com/serverless/sam/). -- GitLab CI/CD. - -### Example - -This example shows you how to: - -- Install SAM CLI. -- Create a sample SAM application including a Lambda function and API Gateway. -- Build and deploy the application to your AWS account using GitLab CI/CD. - -### Steps - -The example consists of the following steps: - -1. Installing SAM CLI. -1. Creating an AWS SAM application using SAM CLI. -1. Crafting the `.gitlab-ci.yml` file. -1. Setting up your AWS credentials with your GitLab account. -1. Deploying your application. -1. Testing the deployed function. - -### Installing SAM CLI - -AWS SAM provides a CLI called AWS SAM CLI to make it easier to create and manage -applications. - -Some steps in this documentation use SAM CLI. Follow the instructions for -[installing SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -to install and configure SAM CLI. - -If you use [AWS Cloud9](https://aws.amazon.com/cloud9/) as your integrated development -environment (IDE), the following are installed for you: - -- [AWS Command Line Interface](https://docs.aws.amazon.com/en_pv/cli/latest/userguide/cli-chap-install.html) -- [SAM CLI](https://docs.aws.amazon.com/en_pv/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) -- [Docker](https://docs.docker.com/install/) and necessary Docker images. - -### Creating an AWS SAM application using SAM CLI - -To create a new AWS SAM application: - -1. Create a new GitLab project. -1. `git clone` the project into your local environment. -1. Change to the newly cloned project and create a new SAM app using the following command: - - ```shell - sam init -r python3.8 -n gitlabpoc --app-template "hello-world" - ``` - -1. `git push` the application back to the GitLab project. - -This creates a SAM app named `gitlabpoc` using the default configuration, a single -Python 3.8 function invoked by an [Amazon API Gateway](https://aws.amazon.com/api-gateway/) -endpoint. To see additional runtimes supported by SAM and options for `sam init`, run: - -```shell -sam init -h -``` - -### Setting up your AWS credentials with your GitLab account - -In order to interact with your AWS account, the GitLab CI/CD pipelines require both -`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be set in the project's CI/CD variables. - -To set these: - -1. Navigate to the project's **Settings > CI/CD**. -1. Expand the **Variables** section and create entries for `AWS_ACCESS_KEY_ID` and - `AWS_SECRET_ACCESS_KEY`. -1. Mask the credentials so they do not show in logs using the **Masked** toggle. - -The AWS credentials you provide must include IAM policies that provision correct access -control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. - -### Crafting the `.gitlab-ci.yml` file - -In a [`.gitlab-ci.yml`](../../../../ci/yaml/index.md) file in the root of your project, -add the following and replace `<S3_bucket_name>` with the name of the S3 bucket where you -want to store your package: - -```yaml -image: python:latest - -stages: - - deploy - -production: - stage: deploy - before_script: - - apt-get update - - apt-get install -y python3-pip - - pip3 install awscli --upgrade - - pip3 install aws-sam-cli --upgrade - script: - - sam build - - sam package --output-template-file packaged.yaml --s3-bucket <S3_bucket_name> - - sam deploy --template-file packaged.yaml --stack-name gitlabpoc --s3-bucket <S3_bucket_name> --capabilities CAPABILITY_IAM --region us-east-1 - environment: production -``` - -Let's examine the configuration file more closely: - -- `image` specifies the Docker image to use for this build. This is the latest Python - image since the sample application is written in Python. -- AWS CLI and AWS SAM CLI are installed in the `before_script` section. -- SAM build, package, and deploy commands are used to build, package, and deploy the - application. - -### Deploying your application - -Push changes to your GitLab repository and the GitLab build pipeline -deploys your application. If your: - -- Build and deploy are successful, [test your deployed application](#testing-the-deployed-application). -- Build fails, look at the build log to see why the build failed. Some common reasons - the build might fail are: - - - Incompatible versions of software. For example, Python runtime version might be - different from the Python on the build machine. Address this by installing the - required versions of the software. - - You may not be able to access your AWS account from GitLab. Check the CI/CD variables - you set up with AWS credentials. - - You may not have permission to deploy a serverless application. Make sure you - provide all required permissions to deploy a serverless application. - -### Testing the deployed application - -To test the application you deployed, please go to the build log and follow the following steps: - -1. Click on "Show complete raw" on the upper right-hand corner: - -  - -1. Look for HelloWorldApi – API Gateway endpoint similar to shown below: - -  - -1. Use curl to test the API. For example: - - ```shell - curl "https://py4rg7qtlg.execute-api.us-east-1.amazonaws.com/Prod/hello/" - ``` - -Output should be: - -```json -{"message": "hello world"} -``` - -### Testing Locally - -AWS SAM provides functionality to test your applications locally. You must have AWS SAM -CLI installed locally for you to test locally. - -First, test the function. - -SAM provides a default event in `events/event.json` that includes a message body of: - -```plaintext -{\"message\": \"hello world\"} -``` - -If you pass that event into the `HelloWorldFunction`, it should respond with the same -body. - -Invoke the function by running: - -```shell -sam local invoke HelloWorldFunction -e events/event.json -``` - -Output should be: - -```json -{"message": "hello world"} -``` - -After you confirm that Lambda function is working as expected, test the API Gateway -using following steps. - -Start the API locally by running: - -```shell -sam local start-api -``` - -SAM again launches a Docker container, this time with a mocked Amazon API Gateway -listening on `localhost:3000`. - -Call the `hello` API by running: - -```shell -curl "http://127.0.0.1:3000/hello" -``` - -Output again should be: - -```json -{"message": "hello world"} -``` +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. diff --git a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png b/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png Binary files differdeleted file mode 100644 index a19d236fc39..00000000000 --- a/doc/user/project/clusters/serverless/img/function-details-loaded_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-endpoint.png b/doc/user/project/clusters/serverless/img/function-endpoint.png Binary files differdeleted file mode 100644 index a38fe2cb6c2..00000000000 --- a/doc/user/project/clusters/serverless/img/function-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-execution.png b/doc/user/project/clusters/serverless/img/function-execution.png Binary files differdeleted file mode 100644 index f60dd277081..00000000000 --- a/doc/user/project/clusters/serverless/img/function-execution.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/function-list_v12_7.png b/doc/user/project/clusters/serverless/img/function-list_v12_7.png Binary files differdeleted file mode 100644 index f2a27ce7b0f..00000000000 --- a/doc/user/project/clusters/serverless/img/function-list_v12_7.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png b/doc/user/project/clusters/serverless/img/sam-api-endpoint.png Binary files differdeleted file mode 100644 index 3407b2684fd..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-api-endpoint.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/sam-complete-raw.png b/doc/user/project/clusters/serverless/img/sam-complete-raw.png Binary files differdeleted file mode 100644 index 1130cd29d56..00000000000 --- a/doc/user/project/clusters/serverless/img/sam-complete-raw.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png b/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png Binary files differdeleted file mode 100644 index f88eb4bdcd2..00000000000 --- a/doc/user/project/clusters/serverless/img/serverless-page_v14_0.png +++ /dev/null diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 29164da307b..432caa8476f 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -2,818 +2,10 @@ stage: Configure group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +remove_date: '2022-08-22' +redirect_to: '../../../../update/removals.md#gitlab-serverless' --- -# Serverless (DEPRECATED) **(FREE)** +# Serverless (removed) **(FREE)** -> - Introduced in GitLab 11.5. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3. - -WARNING: -Serverless is currently in [alpha](../../../../policy/alpha-beta-support.md#alpha-features). - -## Overview - -Serverless architectures offer Operators and Developers the ability write highly scalable applications without provisioning a single server. - -GitLab supports several ways deploy Serverless applications in both Kubernetes Environments and also major cloud Function as a Service (FaaS) environments. - -Currently we support: - -- [Knative](#knative): Build Knative applications with Knative and `gitlabktl` on GKE and EKS. -- [AWS Lambda](aws.md): Create serverless applications via the Serverless Framework and GitLab CI/CD. - -## Knative - -Run serverless workloads on Kubernetes using [Knative](https://cloud.google.com/knative/). - -Knative extends Kubernetes to provide a set of middleware components that are useful to build -modern, source-centric, container-based applications. Knative brings some significant benefits out -of the box through its main components: - -- [Serving](https://github.com/knative/serving): Request-driven compute that can scale to zero. -- [Eventing](https://github.com/knative/eventing): Management and delivery of events. - -For more information on Knative, visit the [Knative docs repository](https://github.com/knative/docs). - -With GitLab Serverless, you can deploy both FaaS and serverless applications. - -## Prerequisites - -To run Knative on GitLab, you need: - -1. **Existing GitLab project:** You need a GitLab project to associate all resources. The simplest way to get started: - - If you are planning on [deploying functions](#deploying-functions), - clone the [functions example project](https://gitlab.com/knative-examples/functions) to get - started. - - If you are planning on [deploying a serverless application](#deploying-serverless-applications), - clone the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get - started. -1. **Kubernetes Cluster:** An RBAC-enabled Kubernetes cluster is required to deploy Knative. - The simplest way to get started is to add a cluster using the GitLab [GKE integration](../add_remove_clusters.md). - The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. -1. **GitLab Runner:** A runner is required to run the CI jobs that deploy serverless - applications or functions onto your cluster. You can install GitLab Runner - onto the [existing Kubernetes cluster](https://docs.gitlab.com/runner/install/kubernetes.html). -1. **Domain Name:** Knative provides its own load balancer using Istio, and an - external IP address or hostname for all the applications served by Knative. Enter a - wildcard domain to serve your applications. Configure your DNS server to use the - external IP address or hostname for that domain. -1. **`.gitlab-ci.yml`:** GitLab uses [Kaniko](https://github.com/GoogleContainerTools/kaniko) - to build the application. We also use [GitLab Knative tool](https://gitlab.com/gitlab-org/gitlabktl) - CLI to simplify the deployment of services and functions to Knative. -1. **`serverless.yml`** (for [functions only](#deploying-functions)): When using serverless to deploy functions, the `serverless.yml` file - contains the information for all the functions being hosted in the repository as well as a reference - to the runtime being used. -1. **`Dockerfile`** (for [applications only](#deploying-serverless-applications)): Knative requires a - `Dockerfile` in order to build your applications. It should be included at the root of your - project's repository and expose port `8080`. `Dockerfile` is not require if you plan to build serverless functions - using our [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes). -1. **Prometheus** (optional): The [Prometheus cluster integration](../../../clusters/integrations.md#prometheus-cluster-integration) - allows you to monitor the scale and traffic of your serverless function/application. -1. **Logging** (optional): Configuring logging allows you to view and search request logs for your serverless function/application. - See [Configuring logging](#configuring-logging) for more information. - -## Configuring Knative - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58941) in GitLab 12.0. - -1. Follow the steps to - [add a Kubernetes - cluster](../add_remove_clusters.md). - -1. Ensure GitLab can manage Knative: - - For a non-GitLab managed cluster, ensure that the service account for the token - provided can manage resources in the `serving.knative.dev` API group. - - For a GitLab managed cluster, if you added the cluster in [GitLab 12.1 or later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30235), - then GitLab already has the required access and you can proceed to the next step. - - Otherwise, you need to manually grant the GitLab service account the ability to manage - resources in the `serving.knative.dev` API group. Since every GitLab service account - has the `edit` cluster role, the simplest way to do this is with an - [aggregated ClusterRole](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles) - adding rules to the default `edit` cluster role: First, save the following YAML as - `knative-serving-only-role.yaml`: - - ```yaml - apiVersion: rbac.authorization.k8s.io/v1 - kind: ClusterRole - metadata: - name: knative-serving-only-role - labels: - rbac.authorization.k8s.io/aggregate-to-edit: "true" - rules: - - apiGroups: - - serving.knative.dev - resources: - - configurations - - configurationgenerations - - routes - - revisions - - revisionuids - - autoscalers - - services - verbs: - - get - - list - - create - - update - - delete - - patch - - watch - ``` - - Then run the following command: - - ```shell - kubectl apply -f knative-serving-only-role.yaml - ``` - - Alternatively, permissions can be granted on a per-service account basis - using `Role`s and `RoleBinding`s (see the [Kubernetes RBAC - documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) - for more information). - -1. Follow the steps to deploy [functions](#deploying-functions) - or [serverless applications](#deploying-serverless-applications) onto your - cluster. - -1. **Optional:** For invocation metrics to show in GitLab, additional Istio metrics need to be configured in your cluster. For example, with Knative v0.9.0, you can use [this manifest](https://gitlab.com/gitlab-org/charts/knative/-/raw/v0.10.0/vendor/istio-metrics.yml). - -## Supported runtimes - -Serverless functions for GitLab can be run using: - -- [GitLab-managed](#gitlab-managed-runtimes) runtimes. -- [OpenFaaS](#openfaas-runtimes) runtimes. - -If a runtime is not available for the required programming language, consider deploying a -[serverless application](#deploying-serverless-applications). - -### GitLab-managed runtimes - -The following GitLab-managed [runtimes](https://gitlab.com/gitlab-org/serverless/runtimes) -are available: - -- `go` (proof of concept) -- `nodejs` -- `ruby` - -You must provide a `Dockerfile` to run serverless functions if no runtime is specified. - -### OpenFaaS runtimes - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29253) in GitLab 12.5. - -[OpenFaaS classic runtimes](https://github.com/openfaas/templates#templates-in-store) can be used with GitLab serverless. - -OpenFaas runtimes are available for the following languages: - -- C# -- Go -- NodeJS -- PHP -- Python -- Ruby - -Runtimes are specified using the pattern: `openfaas/classic/<template_name>`. The following -example shows how to define a function in `serverless.yml` using an OpenFaaS runtime: - -```yaml -hello: - source: ./hello - runtime: openfaas/classic/ruby - description: "Ruby function using OpenFaaS classic runtime" -``` - -`handler` is not needed for OpenFaaS functions. The location of the handler is defined -by the conventions of the runtime. - -See the [`ruby-openfaas-function`](https://gitlab.com/knative-examples/ruby-openfaas-function) -project for an example of a function using an OpenFaaS runtime. - -## Deploying functions - -> Introduced in GitLab 11.6. - -You can find and import all the files referenced in this doc in the -**[functions example project](https://gitlab.com/knative-examples/functions)**. - -Follow these steps to deploy a function using the Node.js runtime to your -Knative instance (you can skip these steps if you've cloned the example -project): - -1. Create a directory to house the function. In this example we will - create a directory called `echo` at the root of the project. - -1. Create the file to contain the function code. In this example, our file is called `echo.js` and is located inside the `echo` directory. If your project is: - - Public, continue to the next step. - - Private, you must [create a GitLab deploy token](../../deploy_tokens/index.md#creating-a-deploy-token) with `gitlab-deploy-token` as the name and the `read_registry` scope. - -1. `.gitlab-ci.yml`: this defines a pipeline used to deploy your functions. - It must be included at the root of your repository: - - ```yaml - include: - - template: Serverless.gitlab-ci.yml - - functions:build: - extends: .serverless:build:functions - environment: production - - functions:deploy: - extends: .serverless:deploy:functions - environment: production - ``` - - This `.gitlab-ci.yml` creates jobs that invoke some predefined commands to - build and deploy your functions to your cluster. - - `Serverless.gitlab-ci.yml` is a template that allows customization. - You can either import it with `include` parameter and use `extends` to - customize your jobs, or you can inline the entire template by choosing it - from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through - the user interface. - -1. `serverless.yml`: this file contains the metadata for your functions, - such as name, runtime, and environment. - - It must be included at the root of your repository. - The following is a sample `echo` function which shows the required structure - for the file. - - You can find the relevant files for this project in the [functions example project](https://gitlab.com/knative-examples/functions). - - ```yaml - service: functions - description: "GitLab Serverless functions using Knative" - - provider: - name: triggermesh - envs: - FOO: value - secrets: - - my-secrets - - functions: - echo-js: - handler: echo-js - source: ./echo-js - runtime: gitlab/runtimes/nodejs - description: "node.js runtime function" - envs: - MY_FUNCTION: echo-js - secrets: - - my-secrets - ``` - -Explanation of the fields used above: - -### `service` - -| Parameter | Description | -|-----------|-------------| -| `service` | Name for the Knative service which serves the function. | -| `description` | A short description of the `service`. | - -### `provider` - -| Parameter | Description | -|-----------|-------------| -| `name` | Indicates which provider is used to execute the `serverless.yml` file. In this case, the TriggerMesh middleware. | -| `envs` | Includes the environment variables to be passed as part of function execution for **all** functions in the file, where `FOO` is the variable name and `BAR` are the variable contents. You may replace this with your own variables. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for **all** functions in the file. The secrets are expected in `INI` format. | - -### `functions` - -In the `serverless.yml` example above, the function name is `echo` and the -subsequent lines contain the function attributes. - -| Parameter | Description | -|-----------|-------------| -| `handler` | The function's name. | -| `source` | Directory with sources of a functions. | -| `runtime` (optional)| The runtime to be used to execute the function. This can be a runtime alias (see [Runtime aliases](#runtime-aliases)), or it can be a full URL to a custom runtime repository. When the runtime is not specified, we assume that `Dockerfile` is present in the function directory specified by `source`. | -| `description` | A short description of the function. | -| `envs` | Sets an environment variable for the specific function only. | -| `secrets` | Includes the contents of the Kubernetes secret as environment variables accessible to be passed as part of function execution for the specific function only. The secrets are expected in `INI` format. | - -### Deployment - -#### Runtime aliases - -The optional `runtime` parameter can refer to one of the following runtime aliases (also see [Supported runtimes](#supported-runtimes)): - -| Runtime alias | Maintained by | -|-------------|---------------| -| `gitlab/runtimes/go` | GitLab | -| `gitlab/runtimes/nodejs` | GitLab | -| `gitlab/runtimes/ruby` | GitLab | -| `openfaas/classic/csharp` | OpenFaaS | -| `openfaas/classic/go` | OpenFaaS | -| `openfaas/classic/node` | OpenFaaS | -| `openfaas/classic/php7` | OpenFaaS | -| `openfaas/classic/python` | OpenFaaS | -| `openfaas/classic/python3` | OpenFaaS | -| `openfaas/classic/ruby` | OpenFaaS | - -After the `.gitlab-ci.yml` template has been added and the `serverless.yml` file -has been created, pushing a commit to your project results in a CI pipeline -being executed which deploys each function as a Knative service. After the -deploy stage has finished, additional details for the function display -under **Infrastructure > Serverless platform**. - - - -This page contains all functions available for the project, the description for -accessing the function, and, if available, the function's runtime information. -The details are derived from the Knative installation inside each of the project's -Kubernetes cluster. Click on each function to obtain detailed scale and invocation data. - -The function details can be retrieved directly from Knative on the cluster: - -```shell -kubectl -n "$KUBE_NAMESPACE" get services.serving.knative.dev -``` - -The sample function can now be triggered from any HTTP client using a simple `POST` call: - - 1. Using curl (replace the URL on the last line with the URL of your application): - - ```shell - curl \ - --header "Content-Type: application/json" \ - --request POST \ - --data '{"GitLab":"FaaS"}' \ - "http://functions-echo.functions-1.functions.example.com/" - ``` - - 1. Using a web-based tool (such as Postman or Restlet) - -  - -### Secrets - -To access your Kubernetes secrets from within your function, the secrets should be created under the namespace of your serverless deployment and specified in your `serverless.yml` file as above. -You can create secrets in several ways. The following sections show some examples. - -#### CLI example - -```shell -kubectl create secret generic my-secrets -n "$KUBE_NAMESPACE" --from-literal MY_SECRET=imverysecure -``` - -#### Part of deployment job - -You can extend your `.gitlab-ci.yml` to create the secrets during deployment using the [CI/CD variables](../../../../ci/variables/index.md) -stored securely under your GitLab project. - -```yaml -deploy:function: - stage: deploy - environment: production - extends: .serverless:deploy:functions - before_script: - - kubectl create secret generic my-secret - --from-literal MY_SECRET="$GITLAB_SECRET_VARIABLE" - --namespace "$KUBE_NAMESPACE" - --dry-run -o yaml | kubectl apply -f - -``` - -### Running functions locally - -Running a function locally is a good way to quickly verify behavior during development. - -Running functions locally requires: - -- Go 1.12 or newer installed. -- Docker Engine installed and running. -- `gitlabktl` installed using the Go package manager: - - ```shell - GO111MODULE=on go get gitlab.com/gitlab-org/gitlabktl - ``` - -To run a function locally: - -1. Navigate to the root of your GitLab serverless project. -1. Build your function into a Docker image: - - ```shell - gitlabktl serverless build - ``` - -1. Run your function in Docker: - - ```shell - docker run -itp 8080:8080 <your_function_name> - ``` - -1. Invoke your function: - - ```shell - curl "http://localhost:8080" - ``` - -## Deploying Serverless applications - -> Introduced in GitLab 11.5. - -Serverless applications are an alternative to [serverless functions](#deploying-functions). -They're useful in scenarios where an existing runtime does not meet the needs of -an application, such as one written in a language that has no runtime available. -Note though that serverless applications should be stateless. - -You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) -to get started. Add the following `.gitlab-ci.yml` to the root of your repository -(you may skip this step if you've previously cloned the previously mentioned, -sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app)): - -```yaml -include: - - template: Serverless.gitlab-ci.yml - -build: - extends: .serverless:build:image - -deploy: - extends: .serverless:deploy:image -``` - -`Serverless.gitlab-ci.yml` is a template that allows customization. -You can either import it with `include` parameter and use `extends` to -customize your jobs, or you can inline the entire template by choosing it -from **Apply a template** dropdown when editing the `.gitlab-ci.yml` file through -the user interface. - -A `serverless.yml` file is not required when deploying serverless applications. - -### Deploy the application with Knative - -With all the pieces in place, the next time a CI pipeline runs the Knative application deploys. Navigate to -**CI/CD > Pipelines** and click the most recent pipeline. - -### Function details - -Go to the **Infrastructure > Serverless platform** page to see the final URL of your functions. - - - -### Invocation metrics - -On the same page as above, click on one of the function -rows to bring up the function details page. - - - -The pod count gives you the number of pods running the serverless function instances on a given cluster. - -For the Knative function invocations to appear, the -[Prometheus cluster integration must be enabled](../../../clusters/integrations.md#prometheus-cluster-integration). - -Once Prometheus is enabled, a message may appear indicating that the metrics data _is -loading or is not available at this time._ It appears upon the first access of the -page, but should go away after a few seconds. If the message does not disappear, then it -is possible that GitLab is unable to connect to the Prometheus instance running on the -cluster. - -## Configuring logging - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33330) in GitLab 12.5. - -### Prerequisites - -- A GitLab-managed cluster. -- `kubectl` installed and working. - -Running `kubectl` commands on your cluster requires setting up access to the -cluster first. For clusters created on: - -- GKE, see [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl) -- Other platforms, see [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/). - -### Enable request log template - -Run the following command to enable request logs: - -```shell -kubectl edit cm -n knative-serving config-observability -``` - -Copy the `logging.request-log-template` from the `data._example` field to the data field one level up in the hierarchy. - -### Enable request logs - -Run the following commands to install Elasticsearch, Kibana, and Filebeat into a `kube-logging` namespace and configure all nodes to forward logs using Filebeat: - -```shell -kubectl apply -f https://gitlab.com/gitlab-org/serverless/configurations/knative/raw/v0.7.0/kube-logging-filebeat.yaml -kubectl label nodes --all beta.kubernetes.io/filebeat-ready="true" -``` - -### Viewing request logs - -To view request logs: - -1. Run `kubectl proxy`. -1. Navigate to [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). - -Or: - -1. Open the [Kibana UI](http://localhost:8001/api/v1/namespaces/kube-logging/services/kibana/proxy/app/kibana). -1. Click on **Discover**, then select `filebeat-*` from the dropdown on the left. -1. Enter `kubernetes.container.name:"queue-proxy" AND message:/httpRequest/` into the search box. - -## Enabling TLS for Knative services - -By default, a GitLab serverless deployment is served over `http`. To serve -over `https`, you must manually obtain and install TLS certificates. - -The simplest way to accomplish this is to use Certbot to -[manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). -Certbot is a free, open source software tool for automatically using Let's Encrypt -certificates on manually-administered websites to enable HTTPS. - -The following instructions relate to installing and running Certbot on a Linux -server that has Python 3 installed, and may not work on other operating systems -or with other versions of Python. - -1. Install Certbot by running the - [`certbot-auto` wrapper script](https://eff-certbot.readthedocs.io/install.html#certbot-auto). - On the command line of your server, run the following commands: - - ```shell - wget https://dl.eff.org/certbot-auto - sudo mv certbot-auto /usr/local/bin/certbot-auto - sudo chown root /usr/local/bin/certbot-auto - sudo chmod 0755 /usr/local/bin/certbot-auto - /usr/local/bin/certbot-auto --help - ``` - - To check the integrity of the `certbot-auto` script, run: - - ```shell - wget -N https://dl.eff.org/certbot-auto.asc - gpg2 --keyserver ipv4.pool.sks-keyservers.net --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc /usr/local/bin/certbot-auto - ``` - - The output of the last command should look something like: - - ```shell - gpg: Signature made Mon 10 Jun 2019 06:24:40 PM EDT - gpg: using RSA key A2CFB51FA275A7286234E7B24D17C995CD9775F2 - gpg: key 4D17C995CD9775F2 marked as ultimately trusted - gpg: checking the trustdb - gpg: marginals needed: 3 completes needed: 1 trust model: pgp - gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u - gpg: next trustdb check due at 2027-11-22 - gpg: Good signature from "Let's Encrypt Client Team <letsencrypt-client@eff.org>" [ultimate] - ``` - -1. Run the following command to use Certbot to request a certificate - using DNS challenge during authorization: - - ```shell - /usr/local/bin/certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' - ``` - - Where `<namespace>` is the namespace created by GitLab for your serverless project (composed of `<project_name>-<project_id>-<environment>`) and - `example.com` is the domain being used for your project. If you are unsure what the namespace of your project is, navigate - to the **Infrastructure > Serverless platform** page of your project and inspect - the endpoint provided for your function/app. - -  - - In the above image, the namespace for the project is `node-function-11909507` and the domain is `knative.info`, thus - certificate request line would look like this: - - ```shell - ./certbot-auto certonly --manual --preferred-challenges dns -d '*.node-function-11909507.knative.info' - ``` - - The Certbot tool walks you through the steps of validating that you own each domain that you specify by creating TXT records in those domains. - After this process is complete, the output should look something like this: - - ```shell - IMPORTANT NOTES: - - Congratulations! Your certificate and chain have been saved at: - /etc/letsencrypt/live/namespace.example.com/fullchain.pem - Your key file has been saved at: - /etc/letsencrypt/live/namespace.example/privkey.pem - Your cert will expire on 2019-09-19. To obtain a new or tweaked - version of this certificate in the future, simply run certbot-auto - again. To non-interactively renew *all* of your certificates, run - "certbot-auto renew" - -----BEGIN PRIVATE KEY----- - - Your account credentials have been saved in your Certbot - configuration directory at /etc/letsencrypt. You should make a - secure backup of this folder now. This configuration directory will - also contain certificates and private keys obtained by Certbot so - making regular backups of this folder is ideal. - ``` - -1. Create certificate and private key files. Using the contents of the files - returned by Certbot, create two files in order to create the - Kubernetes secret: - - Run the following command to see the contents of `fullchain.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/node-function-11909507.knative.info/fullchain.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b4ag== - -----END CERTIFICATE----- - -----BEGIN CERTIFICATE----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - K2fcb195768c39e9a94cec2c2e30Qg== - -----END CERTIFICATE----- - ``` - - Create a file with the name `cert.pem` with the contents of the entire output. - - Once `cert.pem` is created, run the following command to see the contents of `privkey.pem`: - - ```shell - sudo cat /etc/letsencrypt/live/namespace.example/privkey.pem - ``` - - Output should look like this: - - ```shell - -----BEGIN PRIVATE KEY----- - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df - -----BEGIN CERTIFICATE----- - fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 - 4f294d1eaca42b8692017b4262== - -----END PRIVATE KEY----- - ``` - - Create a new file with the name `cert.pk` with the contents of the entire output. - -1. Create a Kubernetes secret to hold your TLS certificate, `cert.pem`, and - the private key `cert.pk`: - - NOTE: - Running `kubectl` commands on your cluster requires setting up access to the cluster first. - For clusters created on GKE, see - [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl). - For other platforms, [install `kubectl`](https://kubernetes.io/docs/tasks/tools/). - - ```shell - kubectl create --namespace istio-system secret tls istio-ingressgateway-certs \ - --key cert.pk \ - --cert cert.pem - ``` - - Where `cert.pem` and `cert.pk` are your certificate and private key files. Note that the `istio-ingressgateway-certs` secret name is required. - -1. Configure Knative to use the new secret that you created for HTTPS - connections. Run the - following command to open the Knative shared `gateway` in edit mode: - - ```shell - kubectl edit gateway knative-ingress-gateway --namespace knative-serving - ``` - - Update the gateway to include the following `tls:` section and configuration: - - ```shell - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - Example: - - ```shell - apiVersion: networking.istio.io/v1alpha3 - kind: Gateway - metadata: - # ... skipped ... - spec: - selector: - istio: ingressgateway - servers: - - hosts: - - "*" - port: - name: http - number: 80 - protocol: HTTP - - hosts: - - "*" - port: - name: https - number: 443 - protocol: HTTPS - tls: - mode: SIMPLE - privateKey: /etc/istio/ingressgateway-certs/tls.key - serverCertificate: /etc/istio/ingressgateway-certs/tls.crt - ``` - - After your changes are running on your Knative cluster, you can begin using the HTTPS protocol for secure access your deployed Knative services. - In the event a mistake is made during this process and you need to update the cert, you must edit the gateway `knative-ingress-gateway` - to switch back to `PASSTHROUGH` mode. Once corrections are made, edit the file again so the gateway uses the new certificates. - -## Using an older version of `gitlabktl` - -There may be situations where you want to run an older version of `gitlabktl`. This -requires setting an older version of the `gitlabktl` image in the `.gitlab-ci.yml` file. - -To set an older version, add `image:` to the `functions:deploy` block. For example: - -```yaml -functions:deploy: - extends: .serverless:deploy:functions - environment: production - image: registry.gitlab.com/gitlab-org/gitlabktl:0.5.0 -``` - -Different versions are available by changing the version tag at the end of the registry URL in the -format `registry.gitlab.com/gitlab-org/gitlabktl:<version>`. - -For a full inventory of available `gitlabktl` versions, see the `gitlabktl` project's -[container registry](https://gitlab.com/gitlab-org/gitlabktl/container_registry). +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/6) in GitLab 14.3 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86267) in GitLab 15.0. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 567c15c7d5f..42c1b8b0a62 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -16,12 +16,16 @@ type: howto, reference > This is [fixed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60525) in > GitLab 13.12. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +> - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/353410) in GitLab 15.0. WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/2493) to add this functionality to the [agent](../index.md). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. + GitLab deploy boards offer a consolidated view of the current health and status of each CI [environment](../../ci/environments/index.md) running on [Kubernetes](https://kubernetes.io), displaying the status of the pods in the deployment. Developers and other teammates can view the diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index 8bb716d8122..17d717bdc61 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -24,8 +24,8 @@ The following list illustrates the main differences between CVS and Git: whole, or they fail without any changes. In CVS, commits (and other operations) are not atomic. If an operation on the repository is interrupted in the middle, the repository can be left in an inconsistent state. -- **Storage method.** Changes in CVS are per file (changeset), while in Git - a committed file(s) is stored in its entirety (snapshot). That means it's +- **Storage method.** Changes in CVS are per file (changeset), while in Git, + committed files are stored in their entirety (snapshot). This means it is very easy in Git to revert or undo a whole change. - **Revision IDs.** The fact that in CVS changes are per files, the revision ID is depicted by version numbers, for example `1.4` reflects how many times a diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 329d91916e8..a190edb179d 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -7,66 +7,56 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import your project from GitHub to GitLab **(FREE)** -Using the importer, you can import your GitHub repositories to GitLab.com or to -your self-managed GitLab instance. - -The following aspects of a project are imported: - -- Repository description (GitLab.com & 7.7+) -- Git repository data (GitLab.com & 7.7+) -- Issues (GitLab.com & 7.7+) -- Pull requests (GitLab.com & 8.4+) -- Wiki pages (GitLab.com & 8.4+) -- Milestones (GitLab.com & 8.7+) -- Labels (GitLab.com & 8.7+) -- Release note descriptions (GitLab.com & 8.12+) -- Pull request review comments (GitLab.com & 10.2+) -- Pull request reviews (GitLab.com & 13.7+) -- Pull request "merged by" information (GitLab.com & 13.7+) -- Regular issue and pull request comments -- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md) -- Pull request comments replies in discussions ([GitLab.com & 14.5+](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)) -- Diff Notes suggestions ([GitLab.com & 14.7+](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)) - -References to pull requests and issues are preserved (GitLab.com & 8.7+), and -each imported repository maintains visibility level unless that [visibility -level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), -in which case it defaults to the default project visibility. - -The namespace is a user or group in GitLab, such as `gitlab.com/janedoe` or -`gitlab.com/customer-success`. You can do some bulk actions to move projects to -different namespaces in the rails console. - -This process does not migrate or import any types of groups or organizations -from GitHub to GitLab. - -## Use cases - -The steps you take depend on whether you are importing from GitHub.com or -GitHub Enterprise. The steps also depend on whether you are importing to GitLab.com or -self-managed GitLab instance. - -- If you're importing to GitLab.com, you can alternatively import GitHub repositories - using a [personal access token](#use-a-github-token). We do not recommend - this method, as it does not associate all user activity (such as issues and - pull requests) with matching GitLab users. -- If you're importing to a self-managed GitLab instance, you can alternatively use the - [GitHub Rake task](../../../administration/raketasks/github_import.md) to import - projects without the constraints of a [Sidekiq](../../../development/sidekiq_style_guide.md) worker. -- If you're importing from GitHub Enterprise to your self-managed GitLab instance: - - You must first enable [GitHub integration](../../../integration/github.md). - - To import projects from GitHub Enterprise to GitLab.com, use the [Import API](../../../api/import.md). - - If GitLab is behind a HTTP/HTTPS proxy you must populate the [allowlist for local requests](../../../security/webhooks.md#allowlist-for-local-requests) - with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue - [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941) -- If you're importing from GitHub.com to your self-managed GitLab instance, - setting up GitHub integration is not required. You can use the [Import API](../../../api/import.md). - -## How it works +You can import your GitHub repositories: + +- From either GitHub.com or GitHub Enterprise. +- To either GitLab.com or a self-managed GitLab instance. + +This process does not migrate or import any types of groups or organizations from GitHub to GitLab. + +The namespace is a user or group in GitLab, such as `gitlab.com/sidney-jones` or +`gitlab.com/customer-success`. You can use bulk actions in the rails console to move projects to +different namespaces. + +If you are importing to a self-managed GitLab instance, you can use the +[GitHub Rake task](../../../administration/raketasks/github_import.md) instead. This allows you to import projects +without the constraints of a [Sidekiq](../../../development/sidekiq/index.md) worker. + +If you are importing from GitHub Enterprise to a self-managed GitLab instance: + +- You must first enable [GitHub integration](../../../integration/github.md). +- To import projects from GitHub Enterprise to GitLab.com, use the [Import API](../../../api/import.md). +- If GitLab is behind a HTTP/HTTPS proxy, you must populate the [allowlist for local requests](../../../security/webhooks.md#allowlist-for-local-requests) + with `github.com` and `api.github.com` to solve the hostname. For more information, read the issue + [Importing a GitHub project requires DNS resolution even when behind a proxy](https://gitlab.com/gitlab-org/gitlab/-/issues/37941). + +If you are importing from GitHub.com to a self-managed GitLab instance: + +- Setting up GitHub integration is not required. +- You can use the [Import API](../../../api/import.md). + +When importing projects: + +- If a user referenced in the project is not found in the GitLab database, the project creator is set as the author and + assignee. The project creator is usually the user that initiated the import process. A note on the issue mentioning the + original GitHub author is added. +- The importer creates any new namespaces (or groups) if they do not exist, or, if the namespace is taken, the + repository is imported under the namespace of the user who initiated the import process. The namespace or repository + name can also be edited, with the proper permissions. +- The importer also imports branches on forks of projects related to open pull requests. These branches are + imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to + a discrepancy in branches compared to those of the GitHub repository. + +For additional technical details, refer to the [GitHub Importer](../../../development/github_importer.md) +developer documentation. + +For an overview of the import process, see the video [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). + +## Prerequisites When issues and pull requests are being imported, the importer attempts to find -their GitHub authors and assignees in the database of the GitLab instance (note -that pull requests are called "merge requests" in GitLab). +their GitHub authors and assignees in the database of the GitLab instance. Pull requests are called _merge requests_ in +GitLab. For this association to succeed, each GitHub author and assignee in the repository must meet one of the following conditions prior to the import: @@ -75,32 +65,9 @@ must meet one of the following conditions prior to the import: - Have a GitHub account with a [public-facing email address](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-user-account/managing-email-preferences/setting-your-commit-email-address) that matches their GitLab account's email address. - NOTE: - GitLab content imports that use GitHub accounts require that the GitHub - public-facing email address is populated so that all comments and - contributions are properly mapped to the same user in GitLab. GitHub - Enterprise (on premise) does not require this field to be populated to use the - product, so you may have to add it on existing accounts for the imported - content to be properly mapped to the user in the new system. Refer to GitHub - documentation for instructions on how to add this email address. - -If a user referenced in the project is not found in the GitLab database, the project creator (typically the user -that initiated the import process) is set as the author/assignee, but a note on the issue mentioning the original -GitHub author is added. - -The importer creates any new namespaces (groups) if they do not exist, or, if the namespace is taken, the -repository is imported under the namespace of the user who initiated the import process. The namespace/repository -name can also be edited, with the proper permissions. - -The importer will also import branches on forks of projects related to open pull requests. These branches will be -imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to -a discrepancy in branches compared to those of the GitHub repository. - -For additional technical details, you can refer to the -[GitHub Importer](../../../development/github_importer.md "Working with the GitHub importer") -developer documentation. - -For an overview of the import process, see the video [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). +GitLab content imports that use GitHub accounts require that the GitHub public-facing email address is populated. This means +all comments and contributions are properly mapped to the same user in GitLab. GitHub Enterprise does not require this +field to be populated so you may have to add it on existing accounts. ## Import your GitHub repository into GitLab @@ -108,10 +75,12 @@ For an overview of the import process, see the video [Migrating from GitHub to G Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: -- A GitLab account that has logged in using the GitHub icon - \- or - +- A GitLab account that has logged in using the GitHub icon. - A GitLab account with an email address that matches the [publicly visible email address](https://docs.github.com/en/rest/reference/users#get-a-user) in the profile of the GitHub user +If you are importing to GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#use-a-github-token). +We do not recommend this method, as it does not associate all user activity (such as issues and pull requests) with matching GitLab users. + User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with the user account that is performing the import. @@ -119,10 +88,10 @@ NOTE: If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured [GitHub integration](../../../integration/github.md). -1. From the top navigation bar, click **+** and select **New project**. +1. From the top navigation bar, select **+** and select **New project**. 1. Select the **Import project** tab and then select **GitHub**. 1. Select the first button to **List your GitHub repositories**. You are redirected to a page on [GitHub](https://github.com) to authorize the GitLab application. -1. Click **Authorize GitlabHQ**. You are redirected back to the GitLab Import page and all of your GitHub repositories are listed. +1. Select **Authorize GitlabHQ**. You are redirected back to the GitLab Import page and all of your GitHub repositories are listed. 1. Continue on to [selecting which repositories to import](#select-which-repositories-to-import). ### Use a GitHub token @@ -134,14 +103,13 @@ associate all user activity (such as issues and pull requests) with matching Git If you are an administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, you cannot use a personal access token. The [GitHub integration method (above)](#use-the-github-integration) is recommended for all users. -Read more in the [How it works](#how-it-works) section. If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: 1. Go to <https://github.com/settings/tokens/new> 1. Enter a token description. 1. Select the repository scope. -1. Click **Generate token**. +1. Select **Generate token**. 1. Copy the token hash. 1. Go back to GitLab and provide the token to the GitHub importer. 1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. @@ -161,7 +129,7 @@ your GitHub repositories are listed. you can filter projects by name. If filter is applied, **Import all repositories** only imports matched repositories. 1. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will update in real-time or you can return to it later. -1. Once a repository has been imported, click its GitLab path to open its GitLab URL. +1. Once a repository has been imported, select its GitLab path to open its GitLab URL.  @@ -197,6 +165,30 @@ Reducing the time spent in cloning a repository can be done by increasing networ performance (by using high performance SSDs, for example) of the disks that store the Git repositories (for your GitLab instance). Increasing the number of Sidekiq workers does *not* reduce the time spent cloning repositories. +## Imported data + +The following items of a project are imported: + +- Repository description. +- Git repository data. +- Issues. +- Pull requests. +- Wiki pages. +- Milestones. +- Labels. +- Release note descriptions. +- Pull request review comments. +- Regular issue and pull request comments. +- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md). +- Pull request reviews (GitLab.com and GitLab 13.7 and later). +- Pull request "merged by" information (GitLab.com and GitLab 13.7 and later). +- Pull request comments replies in discussions ([GitLab.com and GitLab 14.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/336596)). +- Diff Notes suggestions ([GitLab.com and GitLab 14.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/340624)). + +References to pull requests and issues are preserved. Each imported repository maintains visibility level unless that +[visibility level is restricted](../../public_access.md#restrict-use-of-public-or-internal-projects), in which case it +defaults to the default project visibility. + ## Alternative way to import notes and diff notes When GitHub Importer runs on extremely large projects not all notes & diff notes can be imported due to GitHub API `issues_comments` & `pull_requests_comments` endpoints limitation. diff --git a/doc/user/project/import/img/gitlab_import_history_page_v14_10.png b/doc/user/project/import/img/gitlab_import_history_page_v14_10.png Binary files differindex c93b5ed2b27..812696a8faa 100644 --- a/doc/user/project/import/img/gitlab_import_history_page_v14_10.png +++ b/doc/user/project/import/img/gitlab_import_history_page_v14_10.png diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index bf343078634..22e6d45dd96 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -46,7 +46,7 @@ integration in GitLab. 1. Select **Atlassian Bamboo**. 1. Ensure the **Active** checkbox is selected. 1. Enter the base URL of your Bamboo server. For example, `https://bamboo.example.com`. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](overview.md#ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). 1. Enter the [build key](#identify-the-bamboo-build-plan-build-key) from your Bamboo build plan. 1. If necessary, enter a username and password for a Bamboo user that has @@ -71,7 +71,7 @@ Bamboo. For example, `https://bamboo.example.com/browse/PROJ-PLAN`. ### Builds not triggered If builds are not triggered, ensure you entered the right GitLab IP address in -Bamboo under **Trigger IP addresses**. Also check [service hook logs](overview.md#troubleshooting-integrations) for request failures. +Bamboo under **Trigger IP addresses**. Also check [service hook logs](index.md#troubleshooting-integrations) for request failures. ### Advanced Atlassian Bamboo features not available in GitLab UI diff --git a/doc/user/project/integrations/bugzilla.md b/doc/user/project/integrations/bugzilla.md index 4a9a8d62098..0f7ce182e1a 100644 --- a/doc/user/project/integrations/bugzilla.md +++ b/doc/user/project/integrations/bugzilla.md @@ -57,4 +57,4 @@ internal issue tracker, the internal issue is linked. ## Troubleshooting -To see recent service hook deliveries, check [service hook logs](overview.md#troubleshooting-integrations). +To see recent service hook deliveries, check [service hook logs](index.md#troubleshooting-integrations). diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 2dae02dc093..dc56c2669f8 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: The GitLab Slack application is only configurable for GitLab.com. It will **not** work for on-premises installations where you can configure the -[Slack slash commands](slack_slash_commands.md) service instead. We're planning +[Slack slash commands](slack_slash_commands.md) integration instead. We're planning to make this configurable for all GitLab installations, but there's no ETA - see [#28164](https://gitlab.com/gitlab-org/gitlab/-/issues/28164). @@ -31,17 +31,21 @@ Alternatively, you can configure the Slack application with a project's integration settings. Keep in mind that you must have the appropriate permissions for your Slack -team to be able to install a new application, read more in Slack's +workspace to be able to install a new application. Read more in Slack's documentation on [Adding an app to your workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). -To enable the GitLab service for your Slack team: +To enable the GitLab integration for your Slack workspace: 1. Go to your project's **Settings > Integration > Slack application** (only visible on GitLab.com). -1. Select **Add to Slack**. +1. Select **Install Slack app**. +1. Select **Allow** on Slack's confirmation screen. That's all! You can now start using the Slack slash commands. +You can also select **Reinstall Slack app** to update the app in your Slack workspace +to the latest version. See the [Version history](#version-history) for details. + ## Create a project alias for Slack To create a project alias on GitLab.com for Slack integration: @@ -62,7 +66,7 @@ GitLab error: project or alias not found ## Usage -After confirming the installation, you, and everyone else in your Slack team, +After confirming the installation, you, and everyone else in your Slack workspace, can use all the [slash commands](../../../integration/slash_commands.md). When you perform your first slash command, you are asked to authorize your @@ -78,3 +82,11 @@ project, you would do: ```plaintext /gitlab gitlab-org/gitlab issue show 1001 ``` + +## Version history + +### 15.0+ + +In GitLab 15.0 the Slack app is updated to [Slack's new granular permissions app model](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). + +There is no change in functionality. A reinstall is not required but recommended. diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index 9764c4d44a0..7af2e431157 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -6,26 +6,114 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project integrations **(FREE)** -You can find the available integrations under your project's -**Settings > Integrations** page. You need to have at least -the Maintainer role on the project. +You can integrate your GitLab projects with other applications. Integrations are +like plugins, and give you the freedom to add +functionality to GitLab. -## Integrations +## View project integrations -Like plugins, integrations allow you to integrate GitLab with other applications, adding additional features. -For more information, read the -[overview of integrations](overview.md) or learn how to manage your integrations: +Prerequisites: -- *For GitLab 13.3 and later,* read [Project integration management](../../admin_area/settings/project_integration_management.md). -- *For GitLab 13.2 and earlier,* read [Integration Management](../../admin_area/settings/project_integration_management.md), - which replaced the deprecated Service Templates [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268032) - in GitLab 14.0. +- You must have at least the Maintainer role for the project. -## Project webhooks +To view the available integrations for your project: -Project webhooks allow you to trigger a URL if for example new code is pushed or -a new issue is created. You can configure webhooks to listen for specific events -like pushes, issues or merge requests. GitLab sends a POST request with data -to the webhook URL. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. + +You can also view and manage integration settings across [all projects in an instance or group](../../admin_area/settings/project_integration_management.md). +For a single project, you can choose to inherit the instance or group configuration, +or provide custom settings. + +NOTE: +Instance and group-based integration management replaces service templates, which +were [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268032) in GitLab 14.0. + +## Manage SSL verification + +By default, the SSL certificate for outgoing HTTP requests is verified based on +an internal list of Certificate Authorities. This means the certificate cannot +be self-signed. + +You can turn off SSL verification in the configuration settings for [webhooks](webhooks.md#configure-a-webhook-in-gitlab) +and some integrations. + +## Available integrations + +You can configure the following integrations. + +| Integration | Description | Integration hooks | +|-----------------------------------------------------------------------------|-----------------------------------------------------------------------|------------------------| +| [Asana](asana.md) | Add commit messages as comments to Asana tasks. | **{dotted-circle}** No | +| Assembla | Manage projects. | **{dotted-circle}** No | +| [Atlassian Bamboo CI](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | +| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No | +| Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes | +| Campfire | Connect to chat. | **{dotted-circle}** No | +| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Use Confluence Cloud Workspace as an internal wiki. | **{dotted-circle}** No | +| [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | +| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | +| [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | +| Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | +| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | +| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No | +| [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No | +| [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No | +| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | +| [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat. | **{dotted-circle}** No | +| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | +| [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | +| [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | +| JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | +| [Jira](../../../integration/jira/index.md) | Use Jira as the issue tracker. | **{dotted-circle}** No | +| [Mattermost notifications](mattermost.md) | Send notifications about project events to Mattermost channels. | **{dotted-circle}** No | +| [Mattermost slash commands](mattermost_slash_commands.md) | Perform common tasks with slash commands. | **{dotted-circle}** No | +| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | +| Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | +| [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | +| [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | +| [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | +| Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | +| [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | +| [Slack application](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | +| [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | +| [Slack slash commands](slack_slash_commands.md) | Enable slash commands in a workspace. | **{dotted-circle}** No | +| [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | +| [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | +| [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | +| [ZenTao](zentao.md) | Use ZenTao as the issue tracker. | **{dotted-circle}** No | + +### Project webhooks + +You can configure a project webhook to listen for specific events +like pushes, issues, or merge requests. When the webhook is triggered, GitLab +sends a POST request with data to a specified webhook URL. Learn more [about webhooks](webhooks.md). + +## Push hooks limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17874) in GitLab 12.4. + +If a single push includes changes to more than three branches or tags, integrations +supported by `push_hooks` and `tag_push_hooks` events aren't executed. + +You can change the number of supported branches or tags by changing the +[`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). + +## Troubleshooting integrations + +Some integrations use hooks to integrate with external applications. To confirm which ones use integration hooks, see the [available integrations](#available-integrations). Learn more about [troubleshooting integration hooks](webhooks.md#troubleshoot-webhooks). + +### `Test Failed. Save Anyway` error + +Some integrations fail with an error `Test Failed. Save Anyway` when you set them +up on uninitialized repositories. This error occurs because the integration uses +push data to build the test payload, and there are no push events in the project. + +To resolve this error, initialize the repository by pushing a test file to the project +and set up the integration again. + +## Contribute to integrations + +To add a new integration, see the [Integrations development guide](../../../development/integrations/index.md). diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 081780e6277..9625edcd8f9 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -1,117 +1,11 @@ --- -stage: Ecosystem -group: Integrations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2022-07-20' --- -# Integrations **(FREE)** +This document was moved to [another location](index.md). -Integrations allow you to integrate GitLab with other applications. They -are a bit like plugins in that they allow a lot of freedom in adding -functionality to GitLab. - -## Accessing integrations - -To find the available integrations for your project: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Integrations**. - -There are more than 20 integrations to integrate with. Select the one that you -want to configure. - -## Integrations listing - -Click on the integration links to see further configuration instructions and details. - -| Integration | Description | Integration hooks | -| --------------------------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------- | -| [Asana](asana.md) | Add commit messages as comments to Asana tasks. | **{dotted-circle}** No | -| Assembla | Manage projects. | **{dotted-circle}** No | -| [Atlassian Bamboo CI](bamboo.md) | Run CI/CD pipelines with Atlassian Bamboo. | **{check-circle}** Yes | -| [Bugzilla](bugzilla.md) | Use Bugzilla as the issue tracker. | **{dotted-circle}** No | -| Buildkite | Run CI/CD pipelines with Buildkite. | **{check-circle}** Yes | -| Campfire | Connect to chat. | **{dotted-circle}** No | -| [Confluence Workspace](../../../api/integrations.md#confluence-integration) | Replace the link to the internal wiki with a link to a Confluence Cloud Workspace. | **{dotted-circle}** No | -| [Custom issue tracker](custom_issue_tracker.md) | Use a custom issue tracker. | **{dotted-circle}** No | -| [Datadog](../../../integration/datadog.md) | Trace your GitLab pipelines with Datadog. | **{check-circle}** Yes | -| [Discord Notifications](discord_notifications.md) | Send notifications about project events to a Discord channel. | **{dotted-circle}** No | -| Drone CI | Run CI/CD pipelines with Drone. | **{check-circle}** Yes | -| [Emails on push](emails_on_push.md) | Send commits and diff of each push by email. | **{dotted-circle}** No | -| [EWM](ewm.md) | Use IBM Engineering Workflow Management as the issue tracker. | **{dotted-circle}** No | -| [External wiki](../wiki/index.md#link-an-external-wiki) | Link an external wiki. | **{dotted-circle}** No | -| [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No | -| [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | -| [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat.| **{dotted-circle}** No | -| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | -| [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | -| [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | -| JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | -| [Jira](../../../integration/jira/index.md) | Use Jira as the issue tracker. | **{dotted-circle}** No | -| [Mattermost notifications](mattermost.md) | Send notifications about project events to Mattermost channels. | **{dotted-circle}** No | -| [Mattermost slash commands](mattermost_slash_commands.md) | Perform common tasks with slash commands. | **{dotted-circle}** No | -| [Microsoft Teams notifications](microsoft_teams.md) | Receive event notifications. | **{dotted-circle}** No | -| Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | -| [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | -| [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | -| [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | -| Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | -| [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | -| [Slack application](gitlab_slack_application.md) | Use Slack's official GitLab application. | **{dotted-circle}** No | -| [Slack notifications](slack.md) | Send notifications about project events to Slack. | **{dotted-circle}** No | -| [Slack slash commands](slack_slash_commands.md) | Enable slash commands in workspace. | **{dotted-circle}** No | -| [Unify Circuit](unify_circuit.md) | Send notifications about project events to Unify Circuit. | **{dotted-circle}** No | -| [Webex Teams](webex_teams.md) | Receive events notifications. | **{dotted-circle}** No | -| [YouTrack](youtrack.md) | Use YouTrack as the issue tracker. | **{dotted-circle}** No | -| [ZenTao](zentao.md) | Use ZenTao as the issue tracker. | **{dotted-circle}** No | - -## Push hooks limit - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17874) in GitLab 12.4. - -If a single push includes changes to more than three branches or tags, integrations -supported by `push_hooks` and `tag_push_hooks` events aren't executed. - -The number of branches or tags supported can be changed via -[`push_event_hooks_limit` application setting](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). - -## Project integration management - -Project integration management lets you control integration settings across all projects -of an instance. On the project level, administrators you can choose whether to inherit the -instance configuration or provide custom settings. - -Read more about [Project integration management](../../admin_area/settings/project_integration_management.md). - -## SSL verification - -By default, the SSL certificate for outgoing HTTP requests is verified based on -an internal list of Certificate Authorities. This means the certificate cannot -be self-signed. - -You can turn off SSL verification in the configuration settings for [webhooks](webhooks.md#configure-a-webhook-in-gitlab) -and some integrations. - -## Troubleshooting integrations - -Some integrations use hooks for integration with external applications. To confirm which ones use integration hooks, see the [integrations listing](#integrations-listing) above. Learn more about [troubleshooting integration hooks](webhooks.md#troubleshoot-webhooks). - -### Uninitialized repositories - -Some integrations fail with an error `Test Failed. Save Anyway` when you attempt to set them up on -uninitialized repositories. Some integrations use push data to build the test payload, -and this error occurs when no push events exist in the project yet. - -To resolve this error, initialize the repository by pushing a test file to the project and set up -the integration again. - -## Contributing to integrations - -Because GitLab is open source we can ship with the code and tests for all -plugins. This allows the community to keep the plugins up to date so that they -always work in newer GitLab versions. - -For an overview of what integrations are available, please see the -[integrations source directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/app/models/integrations). - -Contributions are welcome! +<!-- This redirect file can be deleted after 2022-07-20. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index f4f5b3f545b..ac7d447961c 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -57,7 +57,7 @@ You can configure a webhook for a group or a project. The URL must be percent-encoded if it contains one or more special characters. 1. In **Secret token**, enter the [secret token](#validate-payloads-by-using-a-secret-token) to validate payloads. 1. In the **Trigger** section, select the [events](webhook_events.md) to trigger the webhook. -1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](overview.md#ssl-verification). +1. Optional. Clear the **Enable SSL verification** checkbox to disable [SSL verification](index.md#manage-ssl-verification). 1. Select **Add webhook**. ## Configure your webhook receiver endpoint @@ -244,7 +244,7 @@ To view the table: - **Failed to connect** if it is misconfigured, and needs manual intervention to re-enable it. - **Fails to connect** if it is temporarily disabled and will retry later. - +  1. Select **Edit** for the webhook you want to view. diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 15130523da6..9d23a63b940 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -36,7 +36,7 @@ The second way is to locate the Confidentiality section in the sidebar and click | Turn off confidentiality | Turn on confidentiality | | :-----------: | :----------: | -|  |  | +|  |  | Every change from regular to confidential and vice versa, is indicated by a system note in the issue's comments. @@ -97,5 +97,5 @@ sees in the project's search results respectively. - [Merge requests for confidential issues](../merge_requests/confidential.md) - [Make an epic confidential](../../group/epics/manage_epics.md#make-an-epic-confidential) -- [Mark a comment as confidential](../../discussions/index.md#mark-a-comment-as-confidential) +- [Add an internal note](../../discussions/index.md#add-an-internal-note) - [Security practices for confidential merge requests](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) at GitLab diff --git a/doc/user/project/issues/csv_import.md b/doc/user/project/issues/csv_import.md index e4b8efd27ed..a3f6ee5e61e 100644 --- a/doc/user/project/issues/csv_import.md +++ b/doc/user/project/issues/csv_import.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Importing issues from CSV **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/23532) in GitLab 11.7. - Issues can be imported to a project by uploading a CSV file with the columns `title` and `description`. Other columns are **not** imported. If you want to retain columns such as labels and milestones, consider the [Move Issue feature](managing_issues.md#move-an-issue). @@ -25,33 +23,29 @@ You must have at least the Developer role for a project to import issues. To import issues: -1. Navigate to a project's Issues list page. -1. If existing issues are present, click the import icon at the top right, next to the **Edit issues** button. -1. For a project without any issues, click the button labeled **Import CSV** in the middle of the page. -1. Select the file and click the **Import issues** button. +1. Go to your project's Issues list page. +1. Open the import feature, depending if the project has issues: + - Existing issues are present: Select the import icon at the top right, next to **Edit issues**. + - Project has no issues: Select **Import CSV** in the middle of the page. +1. Select the file you want to import, and then select **Import issues**. -The file is processed in the background and a notification email is sent -to you once the import is complete. +The file is processed in the background, and a notification email is sent +to you after the import is complete. ## CSV file format -When importing issues from a CSV file, it must be formatted in a certain way: +To import issues, GitLab requires CSV files have a specific format: -- **header row:** CSV files must include the following headers: -`title` and `description`. The case of the headers does not matter. -- **columns:** Data from columns beyond `title` and `description` are not imported. -- **separators:** The column separator is automatically detected from the header row. - Supported separator characters are: commas (`,`), semicolons (`;`), and tabs (`\t`). - The row separator can be either `CRLF` or `LF`. -- **double-quote character:** The double-quote (`"`) character is used to quote fields, - enabling the use of the column separator within a field (see the third line in the - sample CSV data below). To insert a double-quote (`"`) within a quoted - field, use two double-quote characters in succession (`""`). -- **data rows:** After the header row, succeeding rows must follow the same column - order. The issue title is required while the description is optional. +| Element | Format | +|------------------------|--------| +| header row | CSV files must include the following headers: `title` and `description`. The case of the headers does not matter. | +| columns | Data from columns beyond `title` and `description` are not imported. | +| separators | The column separator is detected from the header row. Supported separator characters are commas (`,`), semicolons (`;`), and tabs (`\t`). The row separator can be either `CRLF` or `LF`. | +| double-quote character | The double-quote (`"`) character is used to quote fields, enabling the use of the column separator in a field (see the third line in the sample CSV data below). To insert a double-quote (`"`) in a quoted field use two double-quote characters in succession (`""`). | +| data rows | After the header row, following rows must use the same column order. The issue title is required, but the description is optional. | -If you have special characters _within_ a field, (such as `\n` or `,`), -wrap the characters in double quotes. +If you have special characters in a field, (such as `\n` or `,`), surround the +characters with double quotes (`"`). Sample CSV data: @@ -64,6 +58,7 @@ Another Title,"A description, with a comma" ### File size -The limit depends on the configuration value of Max Attachment Size for the GitLab instance. +The limit depends on how your GitLab instance is hosted: -For GitLab.com, it is set to 10 MB. +- Self-managed: Set by the configuration value of `Max Attachment Size` for the GitLab instance. +- GitLab SaaS: On GitLab.com, it's set to 10 MB. diff --git a/doc/user/project/issues/img/confidential_issues_issue_page.png b/doc/user/project/issues/img/confidential_issues_issue_page.png Binary files differindex 0f5c774d258..b349149aa98 100644 --- a/doc/user/project/issues/img/confidential_issues_issue_page.png +++ b/doc/user/project/issues/img/confidential_issues_issue_page.png diff --git a/doc/user/project/issues/img/design_management_v14_10.png b/doc/user/project/issues/img/design_management_v14_10.png Binary files differindex a10be15eafd..133c0a52d6b 100644 --- a/doc/user/project/issues/img/design_management_v14_10.png +++ b/doc/user/project/issues/img/design_management_v14_10.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differindex a320f4dcfe5..0ef61c7f1b0 100644 --- a/doc/user/project/issues/img/sidebar_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/turn_off_confidentiality.png b/doc/user/project/issues/img/turn_off_confidentiality.png Binary files differdeleted file mode 100644 index 04a85933071..00000000000 --- a/doc/user/project/issues/img/turn_off_confidentiality.png +++ /dev/null diff --git a/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png b/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png Binary files differnew file mode 100644 index 00000000000..37cbe0f4fd9 --- /dev/null +++ b/doc/user/project/issues/img/turn_off_confidentiality_v15_0.png diff --git a/doc/user/project/issues/img/turn_on_confidentiality.png b/doc/user/project/issues/img/turn_on_confidentiality.png Binary files differdeleted file mode 100644 index fac360ca6dc..00000000000 --- a/doc/user/project/issues/img/turn_on_confidentiality.png +++ /dev/null diff --git a/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png b/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png Binary files differnew file mode 100644 index 00000000000..498867d1933 --- /dev/null +++ b/doc/user/project/issues/img/turn_on_confidentiality_v15_0.png diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 4508ef30ac5..7db66dd013b 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -225,6 +225,8 @@ When you're creating a new issue, you can complete the following fields: ## Edit an issue +> Reordering list items [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15260) in GitLab 15.0. + You can edit an issue's title and description. Prerequisites: @@ -237,6 +239,14 @@ To edit an issue: 1. Edit the available fields. 1. Select **Save changes**. +You can also reorder list items, which include bullet, numerical, and task list items. +To reorder list items: + +1. Hover over the list item row to make the drag icon visible. +1. Click and hold the drag icon. +1. Drag the row to the new position in the list. +1. Release the drag icon. + ### Bulk edit issues from a project > - Assigning epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in GitLab 13.2. diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index ff4677eddde..28bd353d8cc 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -189,7 +189,7 @@ You can filter and sort members in a project. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Inherited`. -1. Press Enter. +1. Press <kbd>Enter</kbd>.  @@ -198,7 +198,7 @@ You can filter and sort members in a project. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. 1. In the **Filter members** box, select `Membership` `=` `Direct`. -1. Press Enter. +1. Press <kbd>Enter</kbd>.  diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md index 5826ebcab49..d06c8182e22 100644 --- a/doc/user/project/merge_requests/allow_collaboration.md +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -60,10 +60,10 @@ In the following example: To change or add a commit to the contributor's merge request: -1. Open the merge request page, and select the **Overview** tab. -1. Scroll to the merge request widget, and select **Check out branch**. +1. Go to the merge request. +1. In the upper right corner, select **Code**, then select **Check out branch**. 1. In the modal window, select **Copy** (**{copy-to-clipboard}**). -1. In your terminal, navigate to your cloned version of the repository, and +1. In your terminal, go to your cloned version of the repository, and paste the commands. For example: ```shell diff --git a/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png b/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png Binary files differnew file mode 100644 index 00000000000..b28d216f180 --- /dev/null +++ b/doc/user/project/merge_requests/approvals/img/security_approvals_v15_0.png diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index e940426dc67..f0ab4d606ad 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -102,8 +102,8 @@ Without the approvals, the work cannot merge. Required approvals enable multiple - Use the [code owners of changed files](rules.md#code-owners-as-eligible-approvers), to determine who should review the work. - Require an [approval before merging code that causes test coverage to decline](../../../../ci/pipelines/settings.md#coverage-check-approval-rule) -- [Require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) - before merging code that could introduce a vulnerability. +- Users on GitLab Ultimate can also [require approval from a security team](../../../application_security/index.md#security-approvals-in-merge-requests) + before merging code that could introduce a vulnerability. ## Related topics diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index 01772e59127..17a42e1b540 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -242,3 +242,14 @@ the API. For more information about this validation error, read [issue 285129](https://gitlab.com/gitlab-org/gitlab/-/issues/285129). + +## Security Approvals **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. + +You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. +Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. + + + +These policies are both created and edited in the [security policy editor](../../../application_security/policies/index.md#policy-editor). diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 0ede9310393..9c2b54888fb 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -119,16 +119,9 @@ when more changes are added to it: 1. Select the **Remove all approvals when commits are added to the source branch** checkbox. 1. Select **Save changes**. -Approvals aren't reset when a merge request is [rebased from the UI](../fast_forward_merge.md) +Approvals aren't reset when a merge request is [rebased from the UI](../methods/index.md#rebasing-in-semi-linear-merge-methods) However, approvals are reset if the target branch is changed. -## Security approvals in merge requests **(ULTIMATE)** - -You can require that a member of your security team approves a merge request if a -merge request could introduce a vulnerability. - -To learn more, see [Security approvals in merge requests](../../../application_security/index.md#security-approvals-in-merge-requests). - ## Code coverage check approvals You can require specific approvals if a merge request would result in a decline in code test diff --git a/doc/user/project/merge_requests/changes.md b/doc/user/project/merge_requests/changes.md index 8796ea0635b..5016a33ed28 100644 --- a/doc/user/project/merge_requests/changes.md +++ b/doc/user/project/merge_requests/changes.md @@ -14,7 +14,7 @@ changes. By default, the diff view compares the versions of files in the merge request source branch to the files in the target branch, and shows only the parts of a file that have changed. - + ## Show all changes in a merge request diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 10fc778d5ae..7e8ef9272d4 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -72,7 +72,7 @@ This example shows how to run Code Quality on your code by using GitLab CI/CD an In either configuration, the runner must have enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB. -Once you set up GitLab Runner, include the Code Quality template in your CI configuration: +Once you set up GitLab Runner, include the [Code Quality template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Code-Quality.gitlab-ci.yml) in your CI configuration: ```yaml include: @@ -257,9 +257,9 @@ The template has these [`rules`](../../../ci/yaml/index.md#rules) for the `code ```yaml code_quality: rules: - - if: '$CODE_QUALITY_DISABLED' + - if: $CODE_QUALITY_DISABLED when: never - - if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH' + - if: $CI_COMMIT_TAG || $CI_COMMIT_BRANCH ``` If you are using merge request pipelines, your `rules` (or [`workflow: rules`](../../../ci/yaml/index.md#workflow)) @@ -268,9 +268,9 @@ might look like this example: ```yaml job1: rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run job1 in merge request pipelines - - if: '$CI_COMMIT_BRANCH == "main"' # Run job1 in pipelines on the main branch (but not in other branch pipelines) - - if: '$CI_COMMIT_TAG' # Run job1 in pipelines for tags + - if: $CI_PIPELINE_SOURCE == "merge_request_event" # Run job1 in merge request pipelines + - if: $CI_COMMIT_BRANCH == "main" # Run job1 in pipelines on the main branch (but not in other branch pipelines) + - if: $CI_COMMIT_TAG # Run job1 in pipelines for tags ``` To make these work together, you need to overwrite the code quality `rules` @@ -282,11 +282,11 @@ include: code_quality: rules: - - if: '$CODE_QUALITY_DISABLED' + - if: $CODE_QUALITY_DISABLED when: never - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run code quality job in merge request pipelines - - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH' # Run code quality job in pipelines on the default branch (but not in other branch pipelines) - - if: '$CI_COMMIT_TAG' # Run code quality job in pipelines for tags + - if: $CI_PIPELINE_SOURCE == "merge_request_event" # Run code quality job in merge request pipelines + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH # Run code quality job in pipelines on the default branch (but not in other branch pipelines) + - if: $CI_COMMIT_TAG # Run code quality job in pipelines for tags ``` ### Configure Code Quality to use a private container image registry @@ -541,7 +541,7 @@ This can be due to multiple reasons: - If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks), nothing is displayed. - The [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) CI/CD - setting can cause the Code Quality artifact(s) to expire faster than desired. + setting can cause the Code Quality artifacts to expire faster than desired. - The widgets use the pipeline of the latest commit to the target branch. If commits are made to the default branch that do not run the code quality job, this may cause the merge request widget to have no base report for comparison. - If you use the [`REPORT_STDOUT` environment variable](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables), no report file is generated and nothing displays in the merge request. - Large `gl-code-quality-report.json` files (esp. >10 MB) are [known to prevent the report from being displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/2737). diff --git a/doc/user/project/merge_requests/confidential.md b/doc/user/project/merge_requests/confidential.md index 6900880417f..5b17ec009e4 100644 --- a/doc/user/project/merge_requests/confidential.md +++ b/doc/user/project/merge_requests/confidential.md @@ -74,5 +74,5 @@ Open a merge request - [Confidential issues](../issues/confidential_issues.md) - [Make an epic confidential](../../group/epics/manage_epics.md#make-an-epic-confidential) -- [Mark a comment as confidential](../../discussions/index.md#mark-a-comment-as-confidential) +- [Add an internal note](../../discussions/index.md#add-an-internal-note) - [Security practices for confidential merge requests](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) at GitLab diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md index 77162aa0b83..048421a3a5b 100644 --- a/doc/user/project/merge_requests/fast_forward_merge.md +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -1,74 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, concepts +redirect_to: 'methods/index.md' +remove_date: '2022-08-09' --- -# Fast-forward merge requests **(FREE)** +This document was moved to [another location](methods/index.md). -Sometimes, a workflow policy might mandate a clean commit history without -merge commits. In such cases, the fast-forward merge is the perfect candidate. - -With fast-forward merge requests, you can retain a linear Git history and a way -to accept merge requests without creating merge commits. - -When the fast-forward merge -([`--ff-only`](https://git-scm.com/docs/git-merge#git-merge---ff-only)) setting -is enabled, no merge commits are created and all merges are fast-forwarded, -which means that merging is only allowed if the branch can be fast-forwarded. - -When a fast-forward merge is not possible, the user is given the option to rebase. - -NOTE: -Projects using the fast-forward merge strategy can't filter merge requests -[by deployment date](../../search/index.md#filtering-merge-requests-by-environment-or-deployment-date), -because no merge commit is created. - -## Enabling fast-forward merges - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge requests**. -1. In the **Merge method** section, select **Fast-forward merge**. -1. Select **Save changes**. - -Now, when you visit the merge request page, you can accept it -**only if a fast-forward merge is possible**. - - - -If a fast-forward merge is not possible but a conflict free rebase is possible, -a rebase button is offered. - -You can also rebase without running a CI/CD pipeline. -[Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) GitLab 14.7. - -The rebase action is also available as a [quick action command: `/rebase`](../../../topics/git/git_rebase.md#rebase-from-the-gitlab-ui). - - - -If the target branch is ahead of the source branch and a conflict free rebase is -not possible, you need to rebase the -source branch locally before you can do a fast-forward merge. - - - -## Fast-forward merges prevent squashing commits - -If your project has enabled fast-forward merges, to merge cleanly, the code in a -merge request cannot use [squashing during merge](squash_and_merge.md). Squashing -is available only when accepting a merge request. Rebasing may be required before -squashing, even though squashing can itself be considered equivalent to rebasing. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2022-08-09>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index fd1751585d5..c1986a80ca0 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -51,9 +51,9 @@ Learn the various ways to [create a merge request](creating_merge_requests.md). When you start a new merge request, you can immediately include the following options. You can also add them later by either selecting **Edit** on the merge request's page at the top-right side, or by using -[keyboard shortcuts for merge requests](../../shortcuts.md#issues-and-merge-requests): +[keyboard shortcuts for merge requests](../../shortcuts.md#merge-requests): -- [Assign](#assignee) the merge request to a colleague for review. With [multiple assignees](#multiple-assignees), you can assign it to more than one person at a time. +- [Assign](index.md#assign-a-user-to-a-merge-request) the merge request to a colleague for review. With [multiple assignees](index.md#assign-multiple-users), you can assign it to more than one person at a time. - Set a [milestone](../milestones/index.md) to track time-sensitive changes. - Add [labels](../labels.md) to help contextualize and filter your merge requests over time. - [Require approval](approvals/index.md#required-approvals) from your team. @@ -76,43 +76,11 @@ After you have created the merge request, you can also: Many of these options can be set: -- From the merge request page, with [keyboard shortcuts](../../shortcuts.md#issues-and-merge-requests). +- From the merge request page, with [keyboard shortcuts](../../shortcuts.md#merge-requests). - When pushing changes from the command line, with [Git push options](../push_options.md). See also other [features associated to merge requests](reviews/index.md#associated-features). -### Assignee - -Choose an assignee to designate someone as the person responsible -for the first [review of the merge request](reviews/index.md). -Open the drop down box to search for the user you wish to assign, -and the merge request is added to their -[assigned merge request list](../../search/index.md#search-issues-and-merge-requests). - -#### Multiple assignees **(PREMIUM)** - -> Moved to GitLab Premium in 13.9 - -Multiple people often review merge requests at the same time. -GitLab allows you to have multiple assignees for merge requests -to indicate everyone that is reviewing or accountable for it. - - - -To assign multiple assignees to a merge request: - -1. From a merge request, expand the right sidebar and locate the **Assignees** section. -1. Click on **Edit** and from the dropdown menu, select as many users as you want - to assign the merge request to. - -Similarly, assignees are removed by deselecting them from the same -dropdown menu. - -It is also possible to manage multiple assignees: - -- When creating a merge request. -- Using [quick actions](../quick_actions.md#issues-merge-requests-and-epics). - ### Reviewer WARNING: diff --git a/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png b/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png Binary files differnew file mode 100644 index 00000000000..323fd03ffa2 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_ff_v15_0.png diff --git a/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png b/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png Binary files differnew file mode 100644 index 00000000000..b880c2c0e04 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_merge_commit_v15_0.png diff --git a/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png b/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png Binary files differnew file mode 100644 index 00000000000..9eab71e9d3c --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_method_merge_commit_with_semi_linear_history_v15_0.png diff --git a/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png b/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png Binary files differdeleted file mode 100644 index 1984defde9a..00000000000 --- a/doc/user/project/merge_requests/img/mr-diff-example_v14_8.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/mr-diff-example_v15.png b/doc/user/project/merge_requests/img/mr-diff-example_v15.png Binary files differnew file mode 100644 index 00000000000..8fdf3935906 --- /dev/null +++ b/doc/user/project/merge_requests/img/mr-diff-example_v15.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index a3b9fb52f0d..510dcd82907 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -70,6 +70,72 @@ change and whether you need access to a development environment: - [Push changes from the command line](../../../gitlab-basics/start-using-git.md), if you are familiar with Git and the command line. +## Assign a user to a merge request + +When a merge request is created, it's assigned by default to the person who created it. +This person owns the merge request, but isn't responsible for [reviewing it](reviews/index.md). +To assign the merge request to someone else, use the `/assign @user` +[quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area in +a merge request, or: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. +1. Select **Edit**. +1. Search for the user you want to assign, and select the user. + +The merge request is added to the user's +[assigned merge request list](../../search/index.md#search-issues-and-merge-requests). + +### Assign multiple users **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +GitLab enables multiple assignees for merge requests, if multiple people are +accountable for it: + + + +To assign multiple assignees to a merge request, use the `/assign @user` +[quick action](../quick_actions.md#issues-merge-requests-and-epics) in a text area, or: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Merge requests** and find your merge request. +1. On the right sidebar, expand the right sidebar and locate the **Assignees** section. +1. Select **Edit** and, from the dropdown list, select all users you want + to assign the merge request to. + +To remove an assignee, clear the user from the same dropdown list. + +## Close a merge request + +If you decide to permanently stop work on a merge request, +GitLab recommends you close the merge request rather than +[delete it](#delete-a-merge-request). The author and assignees of a merge request, and users with +Developer, Maintainer, or Owner [roles](../../permissions.md) in a project +can close merge requests in the project: + +1. Go to the merge request you want to close. +1. Scroll to the comment box at the bottom of the page. +1. Following the comment box, select **Close merge request**. + +GitLab closes the merge request, but preserves records of the merge request, +its comments, and any associated pipelines. + +### Delete a merge request + +GitLab recommends you close, rather than delete, merge requests. + +WARNING: +You cannot undo the deletion of a merge request. + +To delete a merge request: + +1. Sign in to GitLab as a user with the project Owner role. + Only users with this role can delete merge requests in a project. +1. Go to the merge request you want to delete, and select **Edit**. +1. Scroll to the bottom of the page, and select **Delete merge request**. + ## Request attention to a merge request > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343528) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `mr_attention_requests`. Disabled by default. @@ -117,35 +183,6 @@ only one attention request, which is synced across both duties. If the attention request is removed from you, either as an assignee or a reviewer, it is removed from both your duties. -## Close a merge request - -If you decide to permanently stop work on a merge request, -GitLab recommends you close the merge request rather than -[delete it](#delete-a-merge-request). The author and assignees of a merge request, and users with -Developer, Maintainer, or Owner [roles](../../permissions.md) in a project -can close merge requests in the project: - -1. Go to the merge request you want to close. -1. Scroll to the comment box at the bottom of the page. -1. Following the comment box, select **Close merge request**. - -GitLab closes the merge request, but preserves records of the merge request, -its comments, and any associated pipelines. - -### Delete a merge request - -GitLab recommends you close, rather than delete, merge requests. - -WARNING: -You cannot undo the deletion of a merge request. - -To delete a merge request: - -1. Sign in to GitLab as a user with the project Owner role. - Only users with this role can delete merge requests in a project. -1. Go to the merge request you want to delete, and select **Edit**. -1. Scroll to the bottom of the page, and select **Delete merge request**. - ## Merge request workflows For a software developer working in a team: diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index 7861e1e28fc..a5fff4a38be 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -189,7 +189,7 @@ review: paths: - review.env rules: - - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. load_performance: dependencies: @@ -197,5 +197,5 @@ load_performance: variables: K6_DOCKER_OPTIONS: '--env-file review.env' rules: - - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. + - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. ``` diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index 256dde4fa17..ac1c61f2e72 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -50,7 +50,7 @@ You can prevent merge requests from being merged if: This works for both: - GitLab CI/CD pipelines -- Pipelines run from an [external CI integration](../integrations/overview.md#integrations-listing) +- Pipelines run from an [external CI integration](../integrations/index.md#available-integrations) As a result, [disabling GitLab CI/CD pipelines](../../../ci/enable_or_disable_ci.md) does not disable this feature, as it is possible to use pipelines from external @@ -81,13 +81,13 @@ it could allow code that fails tests to be merged: ```yaml branch-pipeline-job: rules: - - if: '$CI_PIPELINE_SOURCE == "push"' + - if: $CI_PIPELINE_SOURCE == "push" script: - echo "Code testing scripts here, for example." merge-request-pipeline-job: rules: - - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_PIPELINE_SOURCE == "merge_request_event" script: - echo "No tests run, but this pipeline always succeeds and enables merge." - echo true diff --git a/doc/user/project/merge_requests/methods/index.md b/doc/user/project/merge_requests/methods/index.md new file mode 100644 index 00000000000..adfa5288f81 --- /dev/null +++ b/doc/user/project/merge_requests/methods/index.md @@ -0,0 +1,116 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, concepts +--- + +# Merge methods **(FREE)** + +The merge method you select for your project determines how the changes in your +merge requests are merged into an existing branch. + +## Configure a project's merge method + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. In the **Merge method** section, select your desired merge method. +1. Select **Save changes**. + +## Merge commit + +This setting is the default. It always creates a separate merge commit, +even when using [squash](../squash_and_merge.md). An example commit graph generated using this merge method: + + + +- For regular merges, it is equivalent to the command `git merge --no-ff <source-branch>`. +- For squash merges, it squashes all commits in the source branch before merging it normally. It performs actions similar to: + + ```shell + git checkout `git merge-base <source-branch> <target-branch>` + git merge --squash <source-branch> + SOURCE_SHA=`git rev-parse HEAD` + git checkout <target-branch> + git merge --no-ff $SOURCE_SHA + ``` + +## Merge commit with semi-linear history + +A merge commit is created for every merge, but the branch is only merged if +a fast-forward merge is possible. This ensures that if the merge request build +succeeded, the target branch build also succeeds after the merge. An example commit graph generated using this merge method: + + + +When you visit the merge request page with `Merge commit with semi-linear history` +method selected, you can accept it **only if a fast-forward merge is possible**. +When a fast-forward merge is not possible, the user is given the option to rebase, see +[Rebasing in (semi-)linear merge methods](#rebasing-in-semi-linear-merge-methods). + +This method is equivalent to the same Git commands as in the **Merge commit** method. However, +if your source branch is based on an out-of-date version of the target branch (such as `main`), +you must rebase your source branch. +This merge method creates a cleaner-looking history, while still enabling you to +see where every branch began and was merged. + +## Fast-forward merge + +Sometimes, a workflow policy might mandate a clean commit history without +merge commits. In such cases, the fast-forward merge is appropriate. With +fast-forward merge requests, you can retain a linear Git history and a way +to accept merge requests without creating merge commits. An example commit graph +generated using this merge method: + + + +This method is equivalent to `git merge --ff <source-branch>` for regular merges, and to +`git merge -squash <source-branch>` for squash merges. + +When the fast-forward merge +([`--ff-only`](https://git-scm.com/docs/git-merge#git-merge---ff-only)) setting +is enabled, no merge commits are created and all merges are fast-forwarded, +which means that merging is only allowed if the branch can be fast-forwarded. +When a fast-forward merge is not possible, the user is given the option to rebase, see +[Rebasing in (semi-)linear merge methods](#rebasing-in-semi-linear-merge-methods). + +NOTE: +Projects using the fast-forward merge strategy can't filter merge requests +[by deployment date](../../../search/index.md#filtering-merge-requests-by-environment-or-deployment-date), +because no merge commit is created. + +When you visit the merge request page with `Fast-forward merge` +method selected, you can accept it **only if a fast-forward merge is possible**. + + + +## Rebasing in (semi-)linear merge methods + +> Rebasing without running a CI/CD pipeline [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) in GitLab 14.7. + +In these merge methods, you can merge only when your source branch is up-to-date with the target branch: + +- Merge commit with semi-linear history. +- Fast-forward merge. + +If a fast-forward merge is not possible but a conflict-free rebase is possible, +GitLab offers you the [`/rebase` quick action](../../../../topics/git/git_rebase.md#rebase-from-the-gitlab-ui), +and the ability to **Rebase** from the user interface: + + + +In [GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/118825) and later, you can also rebase without running a CI/CD pipeline. + +If the target branch is ahead of the source branch and a conflict-free rebase is +not possible, you must rebase the source branch locally before you can do a fast-forward merge. + + + +Rebasing may be required before squashing, even though squashing can itself be +considered equivalent to rebasing. + +## Related topics + +- [Commits history](../commits.md) +- [Squash and merge](../squash_and_merge.md) diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 6441ccb73fe..7b4a41f9339 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -14,7 +14,7 @@ by clicking the **Revert** button in merge requests and commit details. NOTE: The **Revert** button is shown only for projects that use the merge method "Merge Commit", which can be set under the project's -**Settings > General > Merge request**. [Fast-forward commits](fast_forward_merge.md) +**Settings > General > Merge request**. [Fast-forward commits](methods/index.md#fast-forward-merge) can not be reverted by using the merge request view. After the merge request has been merged, use the **Revert** button diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 512faae82a9..eb5a54e6119 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -131,17 +131,6 @@ the author of the merge request can request a new review from the reviewer: GitLab creates a new [to-do item](../../../todos.md) for the reviewer, and sends them a notification email. -## Semi-linear history merge requests - -A merge commit is created for every merge, but the branch is only merged if -a fast-forward merge is possible. This ensures that if the merge request build -succeeded, the target branch build also succeeds after the merge. - -1. Go to your project and select **Settings > General**. -1. Expand **Merge requests**. -1. In the **Merge method** section, select **Merge commit with semi-linear history**. -1. Select **Save changes**. - ## Comment on multiple lines > - [Introduced](https://gitlab.com/gitlab-org/ux-research/-/issues/870) in GitLab 13.2. @@ -211,17 +200,17 @@ These features are associated with merge requests: - [Cherry-pick changes](../cherry_pick_changes.md): Cherry-pick any commit in the UI by selecting the **Cherry-pick** button in a merged merge requests or a commit. -- [Fast-forward merge requests](../fast_forward_merge.md): +- [Fast-forward merge requests](../methods/index.md#fast-forward-merge): For a linear Git history and a way to accept merge requests without creating merge commits - [Find the merge request that introduced a change](../versions.md): - When viewing the commit details page, GitLab links to the merge request(s) containing that commit. + When viewing the commit details page, GitLab links to the merge requests containing that commit. - [Merge requests versions](../versions.md): Select and compare the different versions of merge request diffs - [Resolve conflicts](../conflicts.md): GitLab can provide the option to resolve certain merge request conflicts in the GitLab UI. - [Revert changes](../revert_changes.md): Revert changes from any commit from a merge request. -- [Keyboard shortcuts](../../../shortcuts.md#issues-and-merge-requests): +- [Keyboard shortcuts](../../../shortcuts.md#merge-requests): Access and modify specific parts of a merge request with keyboard commands. ## Troubleshooting @@ -365,3 +354,7 @@ All the above can be done with the [`git-mr`](https://gitlab.com/glensc/git-mr) In a group, the sidebar displays the total count of open merge requests. This value is cached if it's greater than than 1000. The cached value is rounded to thousands (or millions) and updated every 24 hours. + +## Related topics + +- [Merge methods](../methods/index.md) diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index a1d6959b75e..7e37990b9bf 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -18,8 +18,8 @@ in your Git repository by using the _squash and merge_ strategy. Each time a branch merges into your base branch, up to two commits are added: - The single commit created by squashing the commits from the branch. -- A merge commit, unless you have [enabled fast-forward merges](fast_forward_merge.md#enabling-fast-forward-merges) - in your project. Fast-forward merges disable both merge commits and squashing. +- A merge commit, unless you have enabled [fast-forward merges](methods/index.md#fast-forward-merge) + in your project. Fast-forward merges disable merge commits. By default, squashed commits contain the following metadata: @@ -74,7 +74,7 @@ To configure the default squashing behavior for all merge requests in your proje ## Related topics - [Commit message templates](commit_templates.md) -- [Fast-forward merges](fast_forward_merge.md) +- [Merge methods](methods/index.md) <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 9f1e5ae7046..85b5bbea284 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -28,7 +28,7 @@ between pipeline completion and the visualization loading on the page. For the coverage analysis to work, you have to provide a properly formatted [Cobertura XML](https://cobertura.github.io/cobertura/) report to -[`artifacts:reports:cobertura`](../../../ci/yaml/artifacts_reports.md#artifactsreportscobertura-deprecated). +[`artifacts:reports:coverage_report`](../../../ci/yaml/artifacts_reports.md#artifactsreportscoverage_report). This format was originally developed for Java, but most coverage analysis frameworks for other languages have plugins to add support for it, like: @@ -196,7 +196,9 @@ coverage-jdk11: needs: ["test-jdk11"] artifacts: reports: - cobertura: target/site/cobertura.xml + coverage_report: + coverage_format: cobertura + path: target/site/cobertura.xml ``` #### Gradle example @@ -232,7 +234,9 @@ coverage-jdk11: needs: ["test-jdk11"] artifacts: reports: - cobertura: build/cobertura.xml + coverage_report: + coverage_format: cobertura + path: build/cobertura.xml ``` ### Python example @@ -251,9 +255,12 @@ run tests: - coverage run -m pytest - coverage report - coverage xml + coverage: '/TOTAL.*\s([.\d]+)%/' artifacts: reports: - cobertura: coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage.xml ``` ### PHP example @@ -263,7 +270,7 @@ to collect test coverage data and generate the report. With a minimal [`phpunit.xml`](https://phpunit.readthedocs.io/en/9.5/configuration.html) file (you may reference [this example repository](https://gitlab.com/yookoala/code-coverage-visualization-with-php/)), you can run the test and -generate the coverage xml: +generate the `coverage.xml`: ```yaml run tests: @@ -283,7 +290,9 @@ run tests: - php ./vendor/bin/phpunit --coverage-text --coverage-cobertura=coverage.cobertura.xml artifacts: reports: - cobertura: coverage.cobertura.xml + coverage_report: + coverage_format: cobertura + path: coverage.cobertura.xml ``` [Codeception](https://codeception.com/), through PHPUnit, also supports generating Cobertura report with @@ -318,7 +327,9 @@ run tests: name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA} expire_in: 2 days reports: - cobertura: build/coverage.xml + coverage_report: + coverage_format: cobertura + path: build/coverage.xml ``` ### Go example @@ -345,7 +356,9 @@ run tests: - go run github.com/boumenot/gocover-cobertura < coverage.txt > coverage.xml artifacts: reports: - cobertura: coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage.xml ``` ### Ruby example @@ -372,5 +385,7 @@ run tests: - bundle exec rspec artifacts: reports: - cobertura: coverage/coverage.xml + coverage_report: + coverage_format: cobertura + path: coverage/coverage.xml ``` diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 4501cf500b0..c2b85a2183c 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -53,10 +53,14 @@ If you're in a project and select **Issues > Milestones**, GitLab displays only ## Creating milestones -Users with at least the Developer role can create milestones. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. Milestones can be created either at project or group level. +Prerequisites: + +- You must have at least the Reporter role for a group. + To create a milestone: 1. On the top bar, select **Menu > Projects** and find your project or **Menu > Groups** and find your group. @@ -69,7 +73,13 @@ To create a milestone: ## Editing milestones -Users with at least the Developer role can edit milestones. +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/343889) the minimum user role from Developer to Reporter in GitLab 15.0. + +Users with at least the Reporter role can edit milestones. + +Prerequisites: + +- You must have at least the Reporter role for a group. To edit a milestone: diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index f08afc924f3..5fd17b5c07e 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -202,7 +202,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: script: @@ -222,7 +222,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: script: @@ -233,7 +233,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" ``` ### Specify a stage to deploy @@ -253,7 +253,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: stage: deploy @@ -265,7 +265,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" ``` Now add another job to the CI file, telling it to @@ -276,7 +276,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH pages: stage: deploy @@ -288,7 +288,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -300,7 +300,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` When the `test` job runs in the `test` stage, Jekyll @@ -327,7 +327,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH before_script: - gem install bundler @@ -341,7 +341,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -351,7 +351,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` ### Build faster with cached dependencies @@ -367,7 +367,7 @@ image: ruby:2.7 workflow: rules: - - if: '$CI_COMMIT_BRANCH' + - if: $CI_COMMIT_BRANCH cache: paths: @@ -385,7 +385,7 @@ pages: paths: - public rules: - - if: '$CI_COMMIT_BRANCH == "main"' + - if: $CI_COMMIT_BRANCH == "main" test: stage: test @@ -395,7 +395,7 @@ test: paths: - test rules: - - if: '$CI_COMMIT_BRANCH != "main"' + - if: $CI_COMMIT_BRANCH != "main" ``` In this case, you need to exclude the `/vendor` diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index f274338c2fd..5846ceeb1b6 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -304,7 +304,7 @@ Find more details in the [Pages administration documentation](../../../administr Safari requires the web server to support the [Range request header](https://developer.apple.com/library/archive/documentation/AppleApplications/Reference/SafariWebContent/CreatingVideoforSafarioniPhone/CreatingVideoforSafarioniPhone.html#//apple_ref/doc/uid/TP40006514-SW6) in order to play your media content. For GitLab Pages to serve -HTTP Range requests, you should use the following two variables in your `.gitlab-ci.yaml` file: +HTTP Range requests, you should use the following two variables in your `.gitlab-ci.yml` file: ```yaml pages: diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 06396b5cd62..56f5a2d24ff 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -10,7 +10,7 @@ In GitLab, [permissions](../permissions.md) are fundamentally defined around the idea of having read or write permission to the repository and branches. To impose further restrictions on certain branches, they can be protected. -The default branch for your repository is protected by default. +The [default branch](repository/branches/default.md) for your repository is protected by default. ## Who can modify a protected branch @@ -39,7 +39,8 @@ Prerequisite: To protect a branch: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. @@ -50,13 +51,18 @@ The protected branch displays in the list of protected branches. ## Configure multiple protected branches by using a wildcard +If both a specific rule and a wildcard rule apply to the same branch, the most +permissive rule controls how the branch behaves. For merge controls to work properly, +set **Allowed to push** to a broader set of users than **Allowed to merge**. + Prerequisite: - You must have at least the Maintainer role. To protect multiple branches at the same time: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, type the branch name and a wildcard. For example: @@ -88,7 +94,8 @@ from the command line or from a Git client application. To create a new branch through the user interface: -1. Go to **Repository > Branches**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. 1. Select **New branch**. 1. Fill in the branch name and select an existing branch, tag, or commit to base the new branch on. Only existing protected branches and commits @@ -96,21 +103,28 @@ To create a new branch through the user interface: ## Require everyone to submit merge requests for a protected branch -You can force everyone to submit a merge request, rather than allowing them to check in directly -to a protected branch. This is compatible with workflows like the [GitLab workflow](../../topics/gitlab_flow.md). +You can force everyone to submit a merge request, rather than allowing them to +check in directly to a protected branch. This setting is compatible with workflows +like the [GitLab workflow](../../topics/gitlab_flow.md). -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select **Developers + Maintainers**. 1. From the **Allowed to push** list, select **No one**. + + NOTE: + Setting a role, group or user as **Allowed to push** also allows those users to merge. + 1. Select **Protect**. ## Allow everyone to push directly to a protected branch You can allow everyone with write access to push to the protected branch. -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select **Developers + Maintainers**. @@ -137,7 +151,8 @@ Prerequisites: To allow a deploy key to push to a protected branch: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select the deploy key. @@ -155,7 +170,8 @@ protected branches. To protect a new branch and enable force push: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. @@ -166,7 +182,8 @@ To protect a new branch and enable force push: To enable force pushes on branches that are already protected: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Allowed to force push** toggle. @@ -181,7 +198,8 @@ For a protected branch, you can require at least one approval by a [Code Owner]( To protect a new branch and enable Code Owner's approval: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. @@ -190,7 +208,8 @@ To protect a new branch and enable Code Owner's approval: To enable Code Owner's approval on branches that are already protected: -1. Go to your project and select **Settings > Repository**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Code owner approval** toggle. @@ -221,9 +240,11 @@ for details about the pipelines security model. Users with at least the Maintainer role can manually delete protected branches by using the GitLab web interface: -1. Go to **Repository > Branches**. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Repository > Branches**. 1. Next to the branch you want to delete, select **Delete** (**{remove}**). -1. On the confirmation dialog, type the branch name and select **Delete protected branch**. +1. On the confirmation dialog, type the branch name. +1. Select **Yes, delete protected branch**. Protected branches can only be deleted by using GitLab either from the UI or API. This prevents accidentally deleting a branch through local Git commands or diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index d04f79d11c7..6ef8477b6b6 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -63,6 +63,7 @@ time as pushing changes: | `merge_request.remove_source_branch` | Set the merge request to remove the source branch when it's merged. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.title="<title>"` | Set the title of the merge request. Ex: `git push -o merge_request.title="The title I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | | `merge_request.description="<description>"` | Set the description of the merge request. Ex: `git push -o merge_request.description="The description I want"`. | [12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/64320) | +| `merge_request.draft` | Mark the merge request as a draft. Ex: `git push -o merge_request.draft`. | [15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/296673) | | `merge_request.milestone="<milestone>"` | Set the milestone of the merge request. Ex: `git push -o merge_request.milestone="3.0"`. | [14.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63960) | | `merge_request.label="<label>"` | Add labels to the merge request. If the label does not exist, it is created. For example, for two labels: `git push -o merge_request.label="label1" -o merge_request.label="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | | `merge_request.unlabel="<label>"` | Remove labels from the merge request. For example, for two labels: `git push -o merge_request.unlabel="label1" -o merge_request.unlabel="label2"`. | [12.3](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31831) | diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 26b36198bde..c0a6fa9c301 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -91,6 +91,7 @@ To create a release in the Releases page: - [Title](#title). - [Milestones](#associate-milestones-with-a-release). - [Release notes](#release-notes-description). + - Whether or not to include the [Tag message](../../../topics/git/tags.md). - [Asset links](#links). 1. Select **Create release**. @@ -439,8 +440,11 @@ Every release has a description. You can add any text you like, but we recommend including a changelog to describe the content of your release. This helps users quickly scan the differences between each release you publish. -[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and -Release note descriptions are unrelated. Description supports [Markdown](../../markdown.md). +[Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) can +be included in Release note descriptions by selecting **Include tag message in +the release notes**. + +Description supports [Markdown](../../markdown.md). ### Release assets @@ -529,7 +533,7 @@ The physical location of the asset can change at any time and the direct link re > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16821) in GitLab 14.9. -The `filepath` from [permanent links to release assets](#permanent-links-to-release-assets) can be used in combination with [permanent link to the latest release](#permanent-link-to-latest-release). It is useful when we want to link a permanant URL to download an asset from the *latest release*. +The `filepath` from [permanent links to release assets](#permanent-links-to-release-assets) can be used in combination with [permanent link to the latest release](#permanent-link-to-latest-release). It is useful when we want to link a permanent URL to download an asset from the *latest release*. The format of the URL is: diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 00998c9a227..b2a6c8848ce 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -11,7 +11,7 @@ GPG ([GNU Privacy Guard](https://gnupg.org/)) key. When you add a cryptographic signature to your commit, you provide extra assurance that a commit originated from you, rather than an impersonator. If GitLab can verify a commit author's identity with a public GPG key, the commit is marked **Verified** in the -GitLab UI. You can then configure [push rules](../push_rules.md#enabling-push-rules) +GitLab UI. You can then configure [push rules](../push_rules.md) for your project to reject individual commits not signed with GPG, or reject all commits from unverified users. diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 6ece6e3e4e0..02b5639cae8 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -244,6 +244,11 @@ When you [rename a user](../../profile/index.md#change-your-username), - The redirects are available as long as the original path is not claimed by another group, user, or project. +WARNING: +The [CI/CD `includes` keyword](../../../ci/yaml/includes.md) can't follow project +redirects. Pipelines fail with a syntax error when configured to use `includes` +to fetch configuration from a project that is renamed or moved. + ## Related topics - [GitLab Workflow VS Code extension](vscode.md). diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md index 39b57f89ceb..d013d2802bf 100644 --- a/doc/user/project/repository/jupyter_notebooks/index.md +++ b/doc/user/project/repository/jupyter_notebooks/index.md @@ -23,21 +23,25 @@ it's rendered into HTML when you view it: Interactive features, including JavaScript plots, don't work when viewed in GitLab. -## Cleaner diffs +## Cleaner diffs and raw diffs > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 as an [Alpha](../../../../policy/alpha-beta-support.md#alpha-features) release [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed. > - [Reintroduced toggle](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85079) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. Enabled by default. +> - Selecting between raw and cleaner diffs [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85203) in GitLab 15.0 [with a flag](../../../../administration/feature_flags.md) named `rendered_diffs_viewer`. Enabled by default. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. +On self-managed GitLab, by default semantic diffs are available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `ipynb_semantic_diff`. On GitLab.com, this feature is available. -This feature is ready for production use. + +FLAG: +On self-managed GitLab, by default the ability to switch between raw and rendered diffs is available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md)named `rendered_diffs_viewer`. On GitLab.com, this feature is available. When commits include changes to Jupyter Notebook files, GitLab: - Transforms the machine-readable `.ipynb` file into a human-readable Markdown file. - Displays a cleaner version of the diff that includes syntax highlighting. +- Enables switching between raw and rendered diffs on the Commit and Compare pages. (Not available on merge request pages.) Code suggestions are not available on diffs and merge requests for `.ipynb` files. diff --git a/doc/user/project/repository/managing_large_repositories.md b/doc/user/project/repository/managing_large_repositories.md new file mode 100644 index 00000000000..d6f88697c54 --- /dev/null +++ b/doc/user/project/repository/managing_large_repositories.md @@ -0,0 +1,51 @@ +--- +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: "Documentation on large repositories." +--- + +# Managing large repositories **(FREE SELF)** + +GitLab, like any Git based system, is subject to similar performance restraints when it comes to large +repositories that size into the gigabytes. + +On this page we detail several best practices to improve performance with these large repositories on GitLab. + +## Large File System (LFS) + +It's *strongly* recommended in any Git system that binary or blob files (for example, packages, audio, video, graphics, etc.) are stored as Large File Storage (LFS) objects. In such setup, the Objects are stored elsewhere, such as in Object Storage, and this can reduce the repository size significantly, thus improving performance. + +Refer to the [Git LFS docs for more info](../../../topics/git/lfs/index.md). + +## Gitaly Pack Objects Cache + +Gitaly, the service that provides storage for Git repositories, can be configured to cache a short rolling window of Git fetch responses. This is recommended for large repositories as it can notably reduce server load when your server receives lots of fetch traffic. + +Refer to the [Gitaly Pack Objects Cache for more info](../../../administration/gitaly/configure_gitaly.md#pack-objects-cache). + +## Reference Architectures + +Large repositories tend to be found in larger organisations with many users. The GitLab Quality and Support teams provide several [Reference Architectures](../../../administration/reference_architectures/index.md) that are the recommended way to deploy GitLab at scale. + +In these types of setups it's recommended that the GitLab environment used matches a Reference Architecture to improve performance. + +## Gitaly Cluster + +Gitaly Cluster can notably improve large repository performance as it holds multiple replicas of the repository across several nodes. As a result, Gitaly Cluster can load balance read requests against those repositories and is also fault tolerant. + +It's recommended for large repositories, however, Gitaly Cluster is a large solution with additional complexity of setup and management. Refer to the [Gitaly Cluster docs for more info](../../../administration/gitaly/index.md), specifically the [Before deploying Gitaly Cluster](../../../administration/gitaly/index.md#before-deploying-gitaly-cluster) section. + +## Keep GitLab up to date + +Performance improvements and fixes are added continuously in GitLab. As such, it's recommended you keep GitLab updated to the latest version where possible to benefit from these. + +## Reduce concurrent clones in CI/CD + +Large repositories tend to be monorepos. This in turn typically means that these repositories get a lot of traffic not only from users, but from CI/CD. + +CI/CD loads tend to be concurrent as pipelines are scheduled during set times. As a result, the Git requests against the repositories can spike notably during these times and lead to reduced performance for both CI and users alike. + +When designing CI/CD pipelines, it's advisable to reduce their concurrency by staggering them to run at different times, for example, a set running at one time and then another set running several minutes later. + +There's several other actions that can be explored to improve CI/CD performance with large repositories. Refer to the [Runner docs for more info](../../../ci/large_repositories/index.md). diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index e1017e78437..1645cf7244e 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -228,10 +228,19 @@ This error can occur when a firewall performs a `Deep SSH Inspection` on outgoin If you receive this error after creating a new project using [GitLab CI/CD for external repositories](../../../../ci/ci_cd_for_external_repos/): -```plaintext -"2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': -terminal prompts disabled\n": exit status 128." -``` +- In Bitbucket Cloud: + + ```plaintext + "2:fetch remote: "fatal: could not read Username for 'https://bitbucket.org': + terminal prompts disabled\n": exit status 128." + ``` + +- In Bitbucket Server (self-managed): + + ```plaintext + "2:fetch remote: "fatal: could not read Username for 'https://lab.example.com': + terminal prompts disabled\n": exit status 128. + ``` Check if the repository owner is specified in the URL of your mirrored repository: @@ -239,13 +248,21 @@ Check if the repository owner is specified in the URL of your mirrored repositor 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Mirroring repositories**. 1. If no repository owner is specified, delete and add the URL again in this format, - replacing `OWNER`, `ACCOUNTNAME`, and `REPONAME` with your values: + replacing `OWNER`, `ACCOUNTNAME`, `PATH_TO_REPO`, and `REPONAME` with your values: - ```plaintext - https://OWNER@bitbucket.org/ACCOUNTNAME/REPONAME.git - ``` + - In Bitbucket Cloud: + + ```plaintext + https://OWNER@bitbucket.org/ACCOUNTNAME/REPONAME.git + ``` + + - In Bitbucket Server (self-managed): + + ```plaintext + https://OWNER@lab.example.com/PATH_TO_REPO/REPONAME.git + ``` -When connecting to the repository for mirroring, Bitbucket requires the repository owner in the string. +When connecting to the Cloud or self-managed Bitbucket repository for mirroring, the repository owner is required in the string. ### Pull mirror is missing LFS files @@ -257,3 +274,31 @@ In some cases, pull mirroring does not transfer LFS files. This issue occurs whe [Fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/335123) in GitLab 14.0.6. - You mirror an external repository using object storage. An issue exists [to fix this problem](https://gitlab.com/gitlab-org/gitlab/-/issues/335495). + +### `The repository is being updated`, but neither fails nor succeeds visibly + +In rare cases, mirroring slots on Redis can become exhausted, +possibly because Sidekiq workers are reaped due to out-of-memory (OoM) events. +When this occurs, mirroring jobs start and complete quickly, but they neither +fail nor succeed. They also do not leave a clear log. To check for this problem: + +1. Enter the [Rails console](../../../../administration/operations/rails_console.md) + and check Redis' mirroring capacity: + + ```ruby + current = Gitlab::Redis::SharedState.with { |redis| redis.scard('MIRROR_PULL_CAPACITY') }.to_i + maximum = Gitlab::CurrentSettings.mirror_max_capacity + available = maximum - current + ``` + +1. If the mirroring capacity is `0` or very low, you can drain all stuck jobs with: + + ```ruby + Gitlab::Redis::SharedState.with { |redis| redis.smembers('MIRROR_PULL_CAPACITY') }.each do |pid| + Gitlab::Redis::SharedState.with { |redis| redis.srem('MIRROR_PULL_CAPACITY', pid) } + end + ``` + +1. After you run the command, the [background jobs page](../../../admin_area/index.md#background-jobs) + should show new mirroring jobs being scheduled, especially when + [triggered manually](#update-a-mirror). diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index 4c437d0f8c8..73103a9af3d 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -62,7 +62,8 @@ Prerequisite: 1. Enter the **Git repository URL**. Include the username in the URL, if required: `https://MYUSERNAME@github.com/GROUPNAME/PROJECTNAME.git` 1. In **Mirror direction**, select **Pull**. -1. In **Authentication method**, select your authentication method. +1. In **Authentication method**, select your authentication method. To learn more, read + [Authentication methods for mirrors](index.md#authentication-methods-for-mirrors). 1. Select any of the options you need: - [**Overwrite diverged branches**](#overwrite-diverged-branches) - [**Trigger pipelines for mirror updates**](#trigger-pipelines-for-mirror-updates) @@ -114,6 +115,21 @@ and mirroring attempts stop. This failure is visible in either the: To resume project mirroring, [force an update](index.md#force-an-update). +If many projects are affected by this problem, such as after a long network or +server outage, you can use the [Rails console](../../../../administration/operations/rails_console.md) +to identify and update all affected projects with this command: + +```ruby +Project.find_each do |p| + if p.import_state && p.import_state.retry_count >= 14 + puts "Resetting mirroring operation for #{p.full_path}" + p.import_state.reset_retry_count + p.import_state.set_next_execution_to_now(prioritized: true) + p.import_state.save! + end +end +``` + ## Related topics - Configure [pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index ebc4430ac53..c00ebf415c9 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -38,8 +38,8 @@ To set up push mirroring for an existing project: 1. Expand **Mirroring repositories**. 1. Enter a repository URL. 1. In the **Mirror direction** dropdown list, select **Push**. -1. Select an **Authentication method**. - You can authenticate with either a password or an [SSH key](index.md#ssh-authentication). +1. Select an **Authentication method**. To learn more, read + [Authentication methods for mirrors](index.md#authentication-methods-for-mirrors). 1. Select **Only mirror protected branches**, if necessary. 1. Select **Keep divergent refs**, if desired. 1. To save the configuration, select **Mirror repository**. diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md index bb473a2830b..994cf78d98a 100644 --- a/doc/user/project/repository/push_rules.md +++ b/doc/user/project/repository/push_rules.md @@ -6,148 +6,136 @@ info: "To determine the technical writer assigned to the Stage/Group associated # Push rules **(PREMIUM)** -Gain additional control over what can and can't be pushed to your repository by using -regular expressions to reject pushes based on commit contents, branch names or file details. - -GitLab already offers [protected branches](../protected_branches.md), but there are -cases when you need some specific rules. Some common scenarios: preventing Git tag removal, or -enforcing a special format for commit messages. - Push rules are [pre-receive Git hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) you -can enable in a user-friendly interface. They are defined either: +can enable in a user-friendly interface. Push rules give you more control over what +can and can't be pushed to your repository. While GitLab offers +[protected branches](../protected_branches.md), you may need more specific rules, such as: -- Globally if you are an administrator. -- Per project, so you can have different rules applied to different - projects depending on your needs. +- Evaluating the contents of a commit. +- Confirming commit messages match expected formats. +- Enforcing branch name rules. +- Evaluating the details of files. +- Preventing Git tag removal. -## Use cases +GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions +in push rules. You can test them at the [regex101 regex tester](https://regex101.com/). -Every push rule could have its own use case, but let's consider some examples. +For custom push rules use [server hooks](../../../administration/server_hooks.md). -### Commit messages with a specific reference +## Enable global push rules -Let's assume you have the following requirements for your workflow: +You can create push rules for all new projects to inherit, but they can be overridden +at the project level or the [group level](../../group/index.md#group-push-rules). -- every commit should reference a Jira issue, for example: `Refactored css. Fixes JIRA-123.` -- users should not be able to remove Git tags with `git push` +Prerequisite: -Write a regular expression that requires the mention -of a Jira issue in the commit message, like `JIRA\-\d+`. +- You must be an administrator. -Now when a user tries to push a commit with a message `Bugfix`, their push is -declined. Only pushing commits with messages like `Bugfix according to JIRA-123` -is accepted. +To create global push rules: -### Restrict branch names +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Push Rules**. +1. Expand **Push rules**. +1. Set the rule you want. +1. Select **Save push rules**. -If your company has a strict policy for branch names, you may want the branches to start -with a certain name. This approach enables different -GitLab CI/CD jobs (such as `feature`, `hotfix`, `docker`, `android`) that rely on the -branch name. +## Override global push rules per project -Your developers may not remember that policy, so they might push to -various branches, and CI pipelines might not work as expected. By restricting the -branch names globally in Push Rules, such mistakes are prevented. -All branch names that don't match your push rule are rejected. +The push rule of an individual project overrides the global push rule. +To override global push rules for a specific project: -Note that the name of your default branch is always allowed, regardless of the branch naming -regular expression (regex) specified. GitLab is configured this way -because merges typically have the default branch as their target. -If you have other target branches, include them in your regex. (See [Enabling push rules](#enabling-push-rules)). +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Push rules**. +1. Set the rule you want. +1. Select **Save push rules**. -The default branch also defaults to being a [protected branch](../protected_branches.md), -which already limits users from pushing directly. +## Verify users -Some example regular expressions you can use in push rules: +Use these rules to validate users who make commits. -- `^JIRA-` Branches must start with `JIRA-`. -- `-JIRA$` Branches must end with `-JIRA`. -- `^[a-z0-9\\-]{4,15}$` Branches must be between `4` and `15` characters long, - accepting only lowercase letters, numbers and dashes. +- **Reject unverified users**: Users must have a [confirmed email address](../../../security/user_email_confirmation.md). +- **Check whether the commit author is a GitLab user**: The commit author and committer must have an email address that's been verified by GitLab. +- **Commit author's email**: Both the author's and committer's email addresses must match the regular expression. + To allow any email address, leave empty. -#### Default restricted branch names +## Validate commit messages -> Introduced in GitLab 12.10. +Use these rules for your commit messages. -By default, GitLab restricts certain formats of branch names for security purposes. -40-character hexadecimal names, similar to Git commit hashes, are prohibited. +- **Require expression in commit messages**: Messages must match the + expression. To allow any commit message, leave empty. + Uses multiline mode, which can be disabled by using `(?-m)`. -### Custom Push Rules **(PREMIUM SELF)** + For example, if every commit should reference a Jira issue + (like `Refactored css. Fixes JIRA-123.`), the regular expression would be + `JIRA\-\d+`. +- **Reject expression in commit messages**: Commit messages must not match + the expression. To allow any commit message, leave empty. + Uses multiline mode, which can be disabled by using `(?-m)`. -It's possible to create custom push rules rather than the push rules available in -**Admin Area > Push Rules** by using more advanced server hooks. +## Validate branch names -See [server hooks](../../../administration/server_hooks.md) for more information. +To validate your branch names, enter a regular expression for **Branch name**. +To allow any branch name, leave empty. Your +[default branch](branches/default.md) is always allowed. -## Enabling push rules +Examples: -You can create push rules for all new projects to inherit, but they can be overridden -at the project level or the [group level](../../group/index.md#group-push-rules). +- Branches must start with `JIRA-`. -To create global push rules: + ```plaintext + `^JIRA-` + ``` -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Push Rules**. +- Branches must end with `-JIRA`. -To override global push rules in a project's settings: + ```plaintext + `-JIRA$` + ``` -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Repository**. -1. Expand **Push rules**. -1. Set the rule you want. -1. Select **Save push rules**. +- Branches must be between `4` and `15` characters long, + accepting only lowercase letters, numbers and dashes. -The following options are available: + ```plaintext + `^[a-z0-9\\-]{4,15}$` + ``` -| Push rule | Description | -|---------------------------------|-------------| -| Removal of tags with `git push` | Forbid users to remove Git tags with `git push`. Tags can be deleted through the web UI. | -| Check whether the commit author is a GitLab user | Restrict commits to existing GitLab users (checked against their emails). <sup>1</sup> | -| Reject unverified users | GitLab rejects any commit that was not committed by the same user as the user who pushed it, or where the committer's email address is not [confirmed](../../../security/user_email_confirmation.md). | -| Check whether commit is signed through GPG | Reject commit when it is not signed through GPG. Read [signing commits with GPG](gpg_signed_commits/index.md). | -| Prevent pushing secret files | GitLab rejects any files that are likely to contain secrets. See the [forbidden file names](#prevent-pushing-secrets-to-the-repository). | -| Require expression in commit messages | Only commit messages that match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | -| Reject expression in commit messages | Only commit messages that do not match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | -| Restrict by branch name | Only branch names that match this regular expression are allowed to be pushed. <sup>2</sup> Leave empty to allow all branch names. | -| Restrict by commit author's email | Only commit author's email that match this regular expression are allowed to be pushed. <sup>1</sup> <sup>2</sup> Leave empty to allow any email. | -| Prohibited file names | Any committed filenames that match this regular expression and do not already exist in the repository are not allowed to be pushed. <sup>2</sup> Leave empty to allow any filenames. See [common examples](#prohibited-file-names). | -| Maximum file size | Pushes that contain added or updated files that exceed this file size (in MB) are rejected. Set to 0 to allow files of any size. Files tracked by Git LFS are exempted. | +NOTE: +In GitLab 12.10 and later, certain formats of branch names are restricted by +default for security purposes. 40-character hexadecimal names, similar to Git +commit hashes, are prohibited. -1. Checks both the commit author and committer. -1. GitLab uses [RE2 syntax](https://github.com/google/re2/wiki/Syntax) for regular expressions in push rules, and you can test them at the [regex101 regex tester](https://regex101.com/). +## Prevent unintended consequences -### Caveat to "Reject unsigned commits" push rule +Use these rules to prevent unintended consequences. -This push rule ignores commits that are authenticated and created by GitLab -(either through the UI or API). When the **Reject unsigned commits** push rule is -enabled, unsigned commits may still show up in the commit history if a commit was -created **in** GitLab itself. As expected, commits created outside GitLab and -pushed to the repository are rejected. For more information about how GitLab -plans to fix this issue, read [issue #19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). +- **Reject unsigned commits**: Commit must be signed through [GPG](gpg_signed_commits/index.md). This rule + can block some legitimate commits [created in the Web IDE](#reject-unsigned-commits-push-rule-disables-web-ide), + and allow [unsigned commits created in the GitLab UI](#unsigned-commits-created-in-the-gitlab-ui). +- **Do not allow users to remove Git tags with `git push`**: Users cannot use `git push` to remove Git tags. + Users can still delete tags in the UI. -#### "Reject unsigned commits" push rule disables Web IDE +## Validate files -In 13.10, if a project has the "Reject unsigned commits" push rule, the user is not allowed to -commit through GitLab Web IDE. +Use these rules to validate files contained in the commit. -To allow committing through the Web IDE on a project with this push rule, a GitLab administrator -must disable the feature flag `reject_unsigned_commits_by_gitlab`. This can be done through a -[rails console](../../../administration/operations/rails_console.md) and running: +- **Prevent pushing secret files**: Files must not contain [secrets](#prevent-pushing-secrets-to-the-repository). +- **Prohibited file names**: Files that do not exist in the repository + must not match the regular expression. To allow all file names, leave empty. See [common examples](#prohibit-files-by-name). +- **Maximum file size**: Added or updated files must not exceed this + file size (in MB). To allow files of any size, set to `0`. Files tracked by Git LFS are exempted. -```ruby -Feature.disable(:reject_unsigned_commits_by_gitlab) -``` - -## Prevent pushing secrets to the repository +### Prevent pushing secrets to the repository > Moved to GitLab Premium in 13.9. -Secrets, such as credential files and SSH private keys, should never be committed to a version control +Never commit secrets, such as credential files and SSH private keys, to a version control system. In GitLab, you can use a predefined list of files to block those files from a -repository. Any merge request containing a file matching the list is blocked from being merged. -Files already committed to the repository are not restricted by this push rule. +repository. Merge requests that contain a file that matches the list are blocked. +This push rule does not restrict files already committed to the repository. -Files blocked by this rule are listed below. For a complete list of criteria, see +Files blocked by this rule are listed below. For a complete list of criteria, refer to [`files_denylist.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/gitlab/checks/files_denylist.yml). - AWS CLI credential blobs: @@ -211,78 +199,85 @@ Files blocked by this rule are listed below. For a complete list of criteria, se - `*.history` - `*_history` -### Prevent pushing secrets to all projects +### Prohibit files by name -To set a global push rule to prevent pushing secrets to all projects: +> Moved to GitLab Premium in 13.9. -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Push Rules**. -1. Expand **Push rules**. -1. Select **Prevent pushing secret files**. -1. Select **Save push rules**. +In Git, filenames include both the file's name, and all directories preceding the name. +When you `git push`, each filename in the push is compared to the regular expression +in **Prohibited file names**. -### Prevent pushing secrets to a project +The regular expression in your **Prohibited file names** push rule can contain multiple, +independent matches to exclude. You can match file names broadly to any location in +your repository, or restrict only in certain locations. Filename matches can also +be partial, and exclude file types by extension. -The push rule of a project overrides the global push rule. +These examples use regex (regular expressions) string boundary characters to match +the beginning of a string (`^`), and its end (`$`). They also include instances +where either the directory path or the filename can include `.` or `/`. Both of +these special regex characters must be escaped with a backslash `\\` if you want +to use them as normal characters in a match condition. -To prevent pushing secrets to a project: +- **Prevent pushing `.exe` files to any location in the repository** - This regex + matches any filename that contains `.exe` at the end: -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > Repository**. -1. Expand **Push rules**. -1. Select **Prevent pushing secret files**. -1. Select **Save push rules**. + ```plaintext + \.exe$ + ``` -## Prohibited file names +- **Prevent pushing a specific configuration file in the repository root** -> Moved to GitLab Premium in 13.9. + ```plaintext + ^config\.yml$ + ``` -Each filename contained in a Git push is compared to the regular expression in this field. Filenames in Git consist of both the file's name and any directory that may precede it. A singular regular expression can contain multiple independent matches used as exclusions. File names can be broadly matched to any location in the repository, or restricted to specific locations. Filenames can also be partial matches used to exclude file types by extension. +- **Prevent pushing a specific configuration file in a known directory** -The following examples make use of regex string boundary characters which match the beginning of a string (`^`), and the end (`$`). They also include instances where either the directory path or the filename can include `.` or `/`. Both of these special regex characters have to be escaped with a backslash `\\` to be used as normal characters in a match condition. + ```plaintext + ^directory-name\/config\.yml$ + ``` -Example: prevent pushing any `.exe` files to any location in the repository. This uses a partial match, which matches any filename that contains `.exe` at the end: +- **Prevent pushing a specific file to any location in the repository** - This example tests + for any file named `install.exe`. The parenthesized expression `(^|\/)` matches either + a file following a directory separator, or a file in the root directory of the repository: -```plaintext -\.exe$ -``` + ```plaintext + (^|\/)install\.exe$ + ``` -Example: prevent a specific configuration file in the repository root from being pushed: +- **Combine all previous expressions into one expression** - The preceding expressions rely + on the end-of-string character `$`. We can move that part of each expression to the + end of the grouped collection of match conditions, where it is appended to all matches: -```plaintext -^config\.yml$ -``` + ```plaintext + (\.exe|^config\.yml|^directory-name\/config\.yml|(^|\/)install\.exe)$ + ``` -Example: prevent a specific configuration file in a known directory from being pushed: +## Related topics -```plaintext -^directory-name\/config\.yml$ -``` +- [Server hooks](../../../administration/server_hooks.md), to create complex custom push rules +- [Signing commits with GPG](gpg_signed_commits/index.md) +- [Protected branches](../protected_branches.md) -Example: prevent the specific file named `install.exe` from being pushed to any -location in the repository. The parenthesized expression `(^|\/)` matches either -a file following a directory separator or a file in the root directory of the repository: +## Troubleshooting -```plaintext -(^|\/)install\.exe$ -``` +### Reject unsigned commits push rule disables Web IDE -Example: combining all of the above in a single expression. The preceding expressions rely -on the end-of-string character `$`. We can move that part of each expression to the -end of the grouped collection of match conditions where it is appended to all matches: +In GitLab 13.10, if a project has the **Reject unsigned commits** push rule, the user cannot +create commits through the GitLab Web IDE. -```plaintext -(\.exe|^config\.yml|^directory-name\/config\.yml|(^|\/)install\.exe)$ -``` +To allow committing through the Web IDE on a project with this push rule, a GitLab administrator +must disable the feature flag `reject_unsigned_commits_by_gitlab`. [with a flag](../../../administration/feature_flags.md) -<!-- ## Troubleshooting +```ruby +Feature.disable(:reject_unsigned_commits_by_gitlab) +``` -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Unsigned commits created in the GitLab UI -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +The **Reject unsigned commits** push rule ignores commits that are authenticated +and created by GitLab (either through the UI or API). When this push rule is +enabled, unsigned commits may still appear in the commit history if a commit was +created in GitLab itself. As expected, commits created outside GitLab and +pushed to the repository are rejected. For more information about this issue, +read [issue #19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185). diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 6eed1717507..747bd690911 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -159,7 +159,7 @@ repository project, GitLab performs these actions: - Creates a default branch. - Commits a blank `README.md` file to it. - Creates and redirects you to a new branch based on the issue title. -- _If your project is [configured with a deployment service](../integrations/overview.md) like Kubernetes,_ +- _If your project is [configured with a deployment service](../integrations/index.md) like Kubernetes,_ GitLab prompts you to set up [auto deploy](../../../topics/autodevops/stages.md#auto-deploy) by helping you create a `.gitlab-ci.yml` file. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 30261ed5082..9b37e147a52 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -41,7 +41,7 @@ Prerequisites: To export a project and its data, follow these steps: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings**. +1. On the left sidebar, select **Settings > General**. 1. Expand **Advanced**. 1. Select **Export project**. 1. After the export is generated, you should receive an email with a link to download the file. @@ -82,6 +82,7 @@ The following items are **not** exported: - Merge Request Approvers and [the number of required approvals](https://gitlab.com/gitlab-org/gitlab/-/issues/221088) - Repository size limits - Deploy keys allowed to push to protected branches +- Secure Files These content rules also apply to creating projects from templates on the [group](../../group/custom_project_templates.md) diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 31cda756a78..1e63472763d 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -88,11 +88,11 @@ read-only view to discourage this behavior. Compliance framework pipelines allow group owners to define a compliance pipeline in a separate repository that gets -executed in place of the local project's `gitlab-ci.yml` file. As part of this pipeline, an -`include` statement can reference the local project's `gitlab-ci.yml` file. This way, the compliance +executed in place of the local project's `.gitlab-ci.yml` file. As part of this pipeline, an +`include` statement can reference the local project's `.gitlab-ci.yml` file. This way, the compliance pipeline jobs can run alongside the project-specific jobs any time the pipeline runs. Jobs and variables defined in the compliance -pipeline can't be changed by variables in the local project's `gitlab-ci.yml` file. +pipeline can't be changed by variables in the local project's `.gitlab-ci.yml` file. When you set up the compliance framework, use the **Compliance pipeline configuration** box to link the compliance framework to specific CI/CD configuration. Use the @@ -314,7 +314,7 @@ related to the project by selecting the **Disable email notifications** checkbox Set up your project's merge request settings: -- Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.md)). +- Set up the [merge request method](../merge_requests/methods/index.md) (merge commit, fast-forward merge). - Add merge request [description templates](../description_templates.md#description-templates). - Enable [merge request approvals](../merge_requests/approvals/index.md). - Enable [status checks](../merge_requests/status_checks.md). @@ -409,10 +409,13 @@ NOTE: Only project owners and administrators have the [permissions](../../permissions.md#project-members-permissions) to transfer a project. -You can transfer an existing project into a [group](../../group/index.md). +You can transfer an existing project to another [group](../../group/index.md), +or you can transfer a [personal project](../working_with_projects.md#view-personal-projects) to a group. Prerequisites: +- A group for your project. You can [view your existing groups](../../group/index.md#view-groups) + to find a suitable group. If you don't have a group, [create one](../../group/index.md#create-a-group). - You must have at least the Maintainer role in that group. - You must be the Owner of that project. - The group to which the project is being transferred to must allow creation of new projects. @@ -423,15 +426,15 @@ Prerequisites: To transfer a project: -1. Navigate to your project's **Settings > General**. -1. Under **Advanced**, click **Expand**. -1. Under "Transfer project", choose the namespace you want to transfer the - project to. -1. Confirm the transfer by typing the project's path as instructed. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Advanced**. +1. Under **Transfer project**, choose the namespace to transfer the project to. +1. Select **Transfer project**. +1. Enter the project's name and select **Confirm**. -Once done, you are redirected to the new project's namespace. At this point, -read what happens with the -[redirects from the old project to the new one](../repository/index.md#what-happens-when-a-repository-path-changes). +You are redirected to the project's new URL. Read what happens with the +[redirects from the old URL to the new one](../repository/index.md#what-happens-when-a-repository-path-changes). NOTE: GitLab administrators can use the [administration interface](../../admin_area/index.md#administering-projects) @@ -520,7 +523,8 @@ If you want to use the fork for yourself and don't need to send you can safely remove the fork relationship. WARNING: -Once removed, the fork relationship cannot be restored. You can't send merge requests to the source, and if anyone has forked your project, their fork also loses the relationship. +Once removed, you can't send merge requests to the source, and if anyone has forked your project, their fork also loses the relationship. +To restore the fork relationship, [use the API](../../../api/projects.md#create-a-forked-fromto-relation-between-existing-projects). To do so: @@ -556,10 +560,6 @@ Automatically [create](../../../operations/incident_management/incidents.md#crea Configure Error Tracking to discover and view [Sentry errors within GitLab](../../../operations/error_tracking.md). -### Jaeger tracing - -Add the URL of a Jaeger server to allow your users to [easily access the Jaeger UI from within GitLab](../../../operations/tracing.md). - ### Status Page **(ULTIMATE)** [Add Storage credentials](../../../operations/incident_management/status_page.md#sync-incidents-to-the-status-page) diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index b66913b7223..e332b74f908 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -25,7 +25,7 @@ Use a project access token to authenticate: Project access tokens are similar to [group access tokens](../../group/settings/group_access_tokens.md) and [personal access tokens](../../profile/personal_access_tokens.md). -In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) as personal access tokens if the limit is set. +In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. You can use project access tokens: @@ -48,7 +48,7 @@ To create a project access token: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the project. -1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. +1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-project-access-token). diff --git a/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png b/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png Binary files differdeleted file mode 100644 index 380d96f1db9..00000000000 --- a/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png b/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png Binary files differdeleted file mode 100644 index 89864858ed3..00000000000 --- a/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png b/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png Binary files differdeleted file mode 100644 index 52776c6a290..00000000000 --- a/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_3.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index 220623d0372..343482757f5 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -2,35 +2,15 @@ stage: Create group: Editor info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: reference, how-to -description: "The static site editor enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture or Git commands." +remove_date: '2022-08-03' +redirect_to: '../web_ide/index.md' --- -# Static Site Editor **(FREE)** +# Static Site Editor (removed) **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. -> - WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214559) in GitLab 13.0. -> - Non-Markdown content blocks not editable on the WYSIWYG mode [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216836) in GitLab 13.3. -> - Formatting Markdown [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49052) in GitLab 13.7. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) in GitLab 14.7. - -WARNING: -This feature is in its end-of-life process. It is -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) -in GitLab 14.7, and is planned for -[removal](https://gitlab.com/groups/gitlab-org/-/epics/7351) in GitLab 14.10. -Users should instead use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md). [Removal instructions](#remove-the-static-site-editor) for existing projects are included on this page. - -Static Site Editor (SSE) enables users to edit content on static websites without -prior knowledge of the underlying templating language, site architecture, or -Git commands. A contributor to your project can quickly edit a Markdown page -and submit the changes for review. For example: - -- Non-technical collaborators can edit a page directly from the browser. - They don't need to know Git and the details of your project to contribute. -- Recently hired team members can quickly edit content. -- Temporary collaborators can jump from project to project and quickly edit pages instead - of having to clone or fork every single project they need to submit changes to. +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352505) in 15.0. +Use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md) instead. ## Remove the Static Site Editor @@ -68,235 +48,3 @@ from an existing project, remove links that point back to the editor: `/helpers/custom_helpers.rb` entirely. 1. Clean up any extraneous configuration files. 1. Commit and push your changes. - -## Requirements - -- In order use the Static Site Editor feature, your project needs to be - pre-configured with the [Static Site Editor Middleman template](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman). -- You need to be logged into GitLab and be a member of the - project (with Developer or higher permission levels). - -## How it works - -The Static Site Editor is in an early stage of development and only supports -Middleman sites for now. You have to use a specific site template to start -using it. The project template is configured to deploy a [Middleman](https://middlemanapp.com/) -static website with [GitLab Pages](../pages/index.md). - -Once your website is up and running, an **Edit this page** button displays on -the bottom-left corner of its pages: - - - -When you click it, GitLab opens up an editor window from which the content -can be directly edited. When you're ready, you can submit your changes in a -click of a button: - - - -When an editor submits their changes, these are the following actions that GitLab -performs automatically in the background: - -1. Creates a new branch. -1. Commits their changes. - 1. Fixes formatting according to the [Handbook Markdown Style Guide](https://about.gitlab.com/handbook/markdown-guide/) - style guide and add them through another commit. -1. Opens a merge request against the default branch. - -The editor can then navigate to the merge request to assign it to a colleague for review. - -## Set up your project - -First, set up the project. Once done, you can use the Static Site Editor to -[edit your content](#edit-content). - -1. To get started, create a new project from the [Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) - template. You can either [fork it](../repository/forking_workflow.md#creating-a-fork) - or [create a new project from a template](../working_with_projects.md#create-a-project-from-a-built-in-template). -1. Edit the [`data/config.yml`](#static-site-generator-configuration) configuration file - to replace `<username>` and `<project-name>` with the proper values for - your project's path. -1. Optional. Edit the [`.gitlab/static-site-editor.yml`](#static-site-editor-configuration-file) file - to customize the behavior of the Static Site Editor. -1. When you submit your changes, GitLab triggers a CI/CD pipeline to deploy your project with GitLab Pages. -1. When the pipeline finishes, from your project's left-side menu, go to **Settings > Pages** to find the URL of your new website. -1. Visit your website and look at the bottom-left corner of the screen to see the new **Edit this page** button. - -Anyone satisfying the [requirements](#requirements) can edit the -content of the pages without prior knowledge of Git or of your site's -codebase. - -## Edit content - -> - Support for modifying the default merge request title and description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216861) in GitLab 13.5. -> - Support for selecting a merge request template [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263252) in GitLab 13.6. - -After setting up your project, you can start editing content directly from the Static Site Editor. - -To edit a file: - -1. Visit the page you want to edit. -1. Select **Edit this page**. -1. The file is opened in the Static Site Editor in **WYSIWYG** mode. If you - wish to edit the raw Markdown instead, you can toggle the **Markdown** mode - in the bottom-right corner. -1. When you're done, click **Submit changes...**. -1. Optional. Adjust the default title and description of the merge request, to submit - with your changes. Alternatively, select a [merge request template](../../../user/project/description_templates.md#create-a-merge-request-template) - from the dropdown menu and edit it accordingly. -1. Select **Submit changes**. -1. A new merge request is automatically created and you can assign a colleague for review. - -### Text - -> Support for `*.md.erb` files [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223171) in GitLab 13.2. - -The Static Site Editors supports Markdown files (`.md`, `.md.erb`) for editing text. - -### Images - -> - Support for adding images through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216640) in GitLab 13.1. -> - Support for uploading images via the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218529) in GitLab 13.6. - -#### Upload an image - -You can upload image files via the WYSIWYG editor directly to the repository to default upload directory -`source/images`. To do so: - -1. Select the image icon (**{doc-image}**). -1. Select the **Upload file** tab. -1. To select a file from your computer, select **Choose file**. -1. Optional. Add a description to the image for SEO and accessibility ([ALT text](https://moz.com/learn/seo/alt-text)). -1. Select **Insert image**. - -The selected file can be any supported image file (`.png`, `.jpg`, `.jpeg`, `.gif`). The editor renders -thumbnail previews so you can verify the correct image is included and there aren't any references to -missing images. - -#### Link to an image - -You can also link to an image if you'd like: - -1. Select the image icon (**{doc-image}**). -1. Select the **Link to an image** tab. -1. Add the link to the image into the **Image URL** field (use the full path; relative paths are not supported yet). -1. Optional. Add a description to the image for SEO and accessibility ([ALT text](https://moz.com/learn/seo/alt-text)). -1. Select **Insert image**. - -The link can reference images already hosted in your project, an asset hosted -externally on a content delivery network, or any other external URL. The editor renders thumbnail previews -so you can verify the correct image is included and there aren't any references to missing images. - -### Videos - -> - Support for embedding YouTube videos through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216642) in GitLab 13.5. - -You can embed YouTube videos on the WYSIWYG mode by clicking the video icon (**{live-preview}**). -The following URL/ID formats are supported: - -- **YouTube watch URLs**: `https://www.youtube.com/watch?v=0t1DgySidms` -- **YouTube embed URLs**: `https://www.youtube.com/embed/0t1DgySidms` -- **YouTube video IDs**: `0t1DgySidms` - -### Front matter - -> - Markdown front matter hidden on the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216834) in GitLab 13.1. -> - Ability to edit page front matter [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235921) in GitLab 13.5. - -Front matter is a flexible and convenient way to define page-specific variables in data files -intended to be parsed by a static site generator. Use it to set a page's -title, layout template, or author. You can also pass any kind of metadata to the -generator as the page renders out to HTML. Included at the very top of each data file, the -front matter is often formatted as YAML or JSON, and requires consistent and accurate syntax. - -To edit the front matter from the Static Site Editor you can use the GitLab regular file editor, -the Web IDE, or update the data directly from the WYSIWYG editor: - -1. Click the **Page settings** button on the bottom-right to reveal a web form with the data you - have on the page's front matter. The form is populated with the current data: - -  - -1. Update the values as you wish and close the panel. -1. When you're done, click **Submit changes...**. -1. Describe your changes (add a commit message). -1. Click **Submit changes**. -1. Click **View merge request** to view it. - -Adding new attributes to the page's front matter from the form is not supported. -To add new attributes: - -- Edit the file locally -- Edit the file with the GitLab regular file editor. -- Edit the file with the Web IDE. - -After adding an attribute, the form loads the new fields. - -## Configuration files - -You can customize the behavior of a project which uses the Static Site Editor with -the following configuration files: - -- The [`.gitlab/static-site-editor.yml`](#static-site-editor-configuration-file), which customizes the - behavior of the Static Site Editor. -- [Static Site Generator configuration files](#static-site-generator-configuration), - such as `data/config.yml`, which configures the Static Site Generator itself. - It also controls the **Edit this page** button when the site is generated. - -### Static Site Editor configuration file - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4267) in GitLab 13.6. - -The `.gitlab/static-site-editor.yml` configuration file contains entries you can -use to customize behavior of the Static Site Editor (SSE). If the file does not exist, -default values which support a default Middleman project configuration are used. -The [Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) project template generates a file pre-populated with these defaults. - -To customize the behavior of the SSE, edit `.gitlab/static-site-editor.yml`'s entries, -according to your project's needs. Make sure to respect YAML syntax. - -After the table, see an [example of the SSE configuration file](#gitlabstatic-site-editoryml-example). - -| Entry | GitLab version | Type | Default value | Description | -|---|---|---|---|---| -| `image_upload_path` | [13.6](https://gitlab.com/gitlab-org/gitlab/-/issues/216641) | String | `source/images` | Directory for images uploaded from the WYSIWYG editor. | - -#### `.gitlab/static-site-editor.yml` example - -```yaml -image_upload_path: 'source/images' # Relative path to the project's root. Don't include leading or trailing slashes. -``` - -### Static Site Generator configuration - -The Static Site Editor uses Middleman's configuration file, `data/config.yml` -to customize the behavior of the project itself. This file also controls the -**Edit this page** button, rendered through the file -[`layout.erb`](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/-/blob/master/source/layouts/layout.erb). - -To [configure the project template to your own project](#set-up-your-project), -you must replace the `<username>` and `<project-name>` in the `data/config.yml` -file with the proper values for your project's path. - -[Other Static Site Generators](#using-other-static-site-generators) used with -the Static Site Editor may use different configuration files or approaches. - -## Using Other Static Site Generators - -Although Middleman is the only Static Site Generator officially supported -by the Static Site Editor, you can configure your project's build and deployment -to use a different Static Site Generator. In this case, use the Middleman layout -as an example, and follow a similar approach to properly render an **Edit this page** -button in your Static Site Generator's layout. - -## Upgrade from GitLab 12.10 to 13.0 - -In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) -to the URL structure of the Static Site Editor. Follow the instructions in this -[snippet](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/snippets/1976539) -to update your project with the 13.0 changes. - -## Limitations - -- The Static Site Editor still cannot be quickly added to existing Middleman sites. - Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. diff --git a/doc/user/project/time_tracking.md b/doc/user/project/time_tracking.md index 5f747d99ce7..0891e02f3f7 100644 --- a/doc/user/project/time_tracking.md +++ b/doc/user/project/time_tracking.md @@ -114,6 +114,18 @@ so if you remove more time than already entered, GitLab ignores the subtraction. To remove all the time spent at once, use the `/remove_time_spent` [quick action](quick_actions.md). +### Delete time spent + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356796) in GitLab 15.0. + +A timelog is a single entry of time spent, either positive or negative. + +Prerequisites: + +- You must be the author of the timelog or have at least the Maintainer role for the project. + +You can [delete timelogs](../../api/graphql/reference/index.md#mutationtimelogdelete) using the GraphQL API. + ## View a time tracking report > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271409) in GitLab 13.12. diff --git a/doc/user/project/web_ide/img/command_palette_v13_6.png b/doc/user/project/web_ide/img/command_palette_v13_6.png Binary files differdeleted file mode 100644 index 54580a79ebd..00000000000 --- a/doc/user/project/web_ide/img/command_palette_v13_6.png +++ /dev/null diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 8f9486633d5..9db30ee2ab6 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -23,13 +23,8 @@ and from merge requests: 1. Select **Open in Web IDE** from the list to display it as the editing option. 1. Select **Open in Web IDE** to open the editor. - *When viewing a merge request* - - 1. Go to your merge request, and select the **Overview** tab. - 1. Scroll to the widgets section, after the merge request description. - 1. Select **Open in Web IDE** if it is visible. - 1. If **Open in Web IDE** is not visible: - 1. Select the **(angle-down)** next to **Open in Gitpod**. - 1. Select **Open in Web IDE** from the list to display it as the editing option. - 1. Select **Open in Web IDE** to open the editor. + 1. Go to your merge request. + 1. In the upper right corner, select **Code**, then select **Open in Gitpod**. ## File finder @@ -52,7 +47,8 @@ Some commands have a keyboard shortcut assigned to them. The command palette displays this shortcut next to each command. You can use this shortcut to invoke the command without having to select it in the command palette. - +For a full list of keyboard shortcuts in the Web IDE, refer to the +[Keyboard shortcuts](../../shortcuts.md#web-ide) list. ## Syntax highlighting diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 37f2ef8fc6a..dc448fed970 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -1,8 +1,7 @@ --- stage: Create group: Editor -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: reference, how-to +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Group wikis **(PREMIUM)** @@ -61,6 +60,24 @@ available, you have to: All files in the wiki are available in this Git repository. +## Configure group wiki visibility + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208412) in GitLab 15.0. + +Wikis are enabled by default in GitLab. Group [administrators](../../permissions.md) +can enable or disable a group wiki through the group settings. + +To open group settings: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > General**. +1. Expand **Permissions and group features**. +1. Scroll to **Wiki** and select one of these options: + - **Enabled**: Everyone who can access the group can access the wiki. + - **Private**: Only group members can access the wiki. + - **Disabled**: The wiki isn't accessible, and cannot be downloaded. +1. Select **Save changes**. + ## Related topics - [Wiki settings for administrators](../../../administration/wikis/index.md) diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 7d155ea9b06..fe6c7ae62b8 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -275,7 +275,7 @@ can enable or disable a project wiki by following the instructions in Administrators for self-managed GitLab installs can [configure additional wiki settings](../../../administration/wikis/index.md). -You can't disable [group wikis](group.md) from the GitLab user interface. +You can disable group wikis from the [group settings](group.md#configure-group-wiki-visibility) ## Link an external wiki diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 03530b59e9b..bfc83aa22f5 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -275,6 +275,18 @@ To add a star to a project: - Number of open merge requests. - Number of open issues. +## View personal projects + +Personal projects are projects created under your personal namespace. + +For example, if you create an account with the username `alex`, and create a project +called `my-project` under your username, the project is created at `https://gitlab.example.com/alex/my-project`. + +To view your personal projects: + +1. On the top bar, select **Menu > Projects > Your Projects**. +1. Under **Your projects**, select **Personal**. + ## Delete a project After you delete a project, projects in personal namespaces are deleted immediately. To delay deletion of projects in a group diff --git a/doc/user/report_abuse.md b/doc/user/report_abuse.md index c0fb29b435a..93d9a1e773e 100644 --- a/doc/user/report_abuse.md +++ b/doc/user/report_abuse.md @@ -53,10 +53,8 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Report abuse from a merge request -1. On the merge request, in the top right corner, either: - - Select **Report abuse**. This option is displayed if you do not have permission to close the merge request. - - Next to **Mark as draft**, select the down arrow (**{chevron-down}**) and then select **Report abuse**. - This option is displayed if you have permission to close the merge request. +1. On the merge request, in the top right corner, select the vertical ellipsis (**{ellipsis_v}**). +1. Select **Report abuse**. 1. Submit an abuse report. 1. Select **Send report**. diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 45c5f53e33c..3cf379243e3 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -63,7 +63,6 @@ Currently, the following names are reserved as top level groups: - `503.html` - `admin` - `api` -- `apple-touch-icon-precomposed.png` - `apple-touch-icon.png` - `assets` - `dashboard` diff --git a/doc/user/search/global_search/advanced_search_syntax.md b/doc/user/search/global_search/advanced_search_syntax.md index 962aa00eea8..3aa016f0bff 100644 --- a/doc/user/search/global_search/advanced_search_syntax.md +++ b/doc/user/search/global_search/advanced_search_syntax.md @@ -1,49 +1,51 @@ --- stage: Enablement group: Global Search -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: reference +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Advanced Search syntax **(PREMIUM)** With [Advanced Search](../advanced_search.md), you can perform a thorough -search through your entire GitLab instance. +search of your entire GitLab instance. The Advanced Search syntax supports fuzzy or exact search queries with prefixes, -boolean operators, and much more. Advanced Search uses +boolean operators, and more. Advanced Search uses [Elasticsearch's syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html#simple-query-string-syntax). WARNING: -Advanced Search searches projects' default branches only. - -See query examples on the tables below and their respective expected output. -The examples link to a search on GitLab.com to help you visualize the output. +Advanced Search searches default project branches only. ## General search -| Query example | Expected output | -|---|---| -[`“display bug”`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=%22display+bug%22&group_id=9970&project_id=278964) | Returns the **exact phrase** _display bug_ (stemming still applies). | -[`bug -display`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+-display&group_id=9970&project_id=278964) | Results include _bug_, and **exclude** _display_. | -[<code>bug | display</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+banner&group_id=9970&project_id=278964) | Results include _bug_ **or** _display_. | -[<code>bug | (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Results include _bug_ **or** _display_ **and** _banner_. | -| [`bug error 50*`](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+error+50*&group_id=9970&project_id=278964) | `*` finds **partial matches**. Results include _bug_, _error_, and the partial _50_ (looking for any 500 errors, for example). | -| [`bug \-display`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=argument+%5C-last&group_id=9970&project_id=278964) | `\` **scapes symbols**. Results include _bug_ **and** _-display_. | +<!-- markdownlint-disable --> + +| Use | Description | Example | +|-----|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------| +| `"` | Exact search | [`"gem sidekiq"`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=%22gem+sidekiq%22) | +| <code>|</code> | Or | [<code>display | banner</code>](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+%7C+banner) | +| `+` | And | [`display +banner`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=display+%2Bbanner&snippets=) | +| `-` | Exclude | [`display -banner`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=display+-banner) | +| `*` | Partial | [`bug error 50*`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=bug+error+50%2A&snippets=) | +| `\` | Escape | [`\*md`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=%5C*md&group_id=9970&project_id=278964) | + +## Code search -## Code Search +| Use | Description | Example | +|--------------|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------| +| `filename:` | File name | [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) | +| `path:` | Repository location | [`path:spec/workers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fworkers&snippets=) | +| `extension:` | File extension, without the `.` | [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) | +| `blob:` | Git object ID | [`blob:998707*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707*&group_id=9970) | -| Query example | Expected output | Notes | -|---|---|---| -| [`filename:*spec.rb`](https://gitlab.com/search?snippets=&scope=blobs&repository_ref=&search=filename%3A*spec.rb&group_id=9970&project_id=278964) | Returns the specified filename. | Use `*` for fuzzy matching. | -| [`path:spec/controllers/`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=path%3Aspec%2Fcontrollers%2F&snippets=) | Returns the specified path location of the repository. | Use `*` for fuzzy matching. | -| [`extension:js`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=extension%3Ajs&snippets=) | Returns the specified file extension. | **Do not** include a leading dot. This only works with exact matches for the extension. | -| [`blob:998707b421c89b*`](https://gitlab.com/search?snippets=false&scope=blobs&repository_ref=&search=blob%3A998707b421c89b*&group_id=9970) | Returns the specified Git object ID. | This only works with exact matches. | +`extension` and `blob` return exact matches only. -## Excluding filters +## Examples -Filters can also be inverted to filter out results from the result set by prefixing the filter name with a `-` (hyphen) character. +| Example | Description | +|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------| +| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Show _rails_ in all files except the _`gemfile.lock`_ file. | +| [`RSpec.describe Resolvers -*builder`](https://gitlab.com/search?group_id=9970&project_id=278964&scope=blobs&search=RSpec.describe+Resolvers+-*builder) | Show all _RSpec.describe Resolvers_ that don't start with _builder_. | +| [<code>bug | (display +banner)</code>](https://gitlab.com/search?snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) | Show _bug_ **or** _display_ **and** _banner_. | -| Query example | Expected output | -|---|---| -| [`rails -filename:gemfile.lock`](https://gitlab.com/search?group_id=9970&project_id=278964&repository_ref=&scope=blobs&search=rails+-filename%3Agemfile.lock&snippets=) | Results include _`rails`_ in all files except the _`gemfile.lock`_ file. | +<!-- markdownlint-enable --> diff --git a/doc/user/search/img/issue_search_by_id.png b/doc/user/search/img/issue_search_by_id.png Binary files differdeleted file mode 100644 index 96c0b5c31e1..00000000000 --- a/doc/user/search/img/issue_search_by_id.png +++ /dev/null diff --git a/doc/user/search/img/issue_search_by_id_v15_0.png b/doc/user/search/img/issue_search_by_id_v15_0.png Binary files differnew file mode 100644 index 00000000000..411cebc0ccb --- /dev/null +++ b/doc/user/search/img/issue_search_by_id_v15_0.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index de5f469498e..171d8a63d2d 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -108,7 +108,7 @@ You can add this URL to your feed reader. You can filter the **Issues** list to individual instances by their ID. For example, enter filter `#10` to return only issue 10. The same applies to the **Merge requests** list. Enter filter `#30` to return only merge request 30. - + ### Filtering merge requests by approvers **(PREMIUM)** diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index e5285d63cf4..2d753fa042a 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -25,21 +25,23 @@ explained in each section. These shortcuts are available in most areas of GitLab: -| Keyboard shortcut | Description | -|---------------------------------|-------------| -| <kbd>?</kbd> | Show or hide the shortcut reference sheet. | -| <kbd>Shift</kbd> + <kbd>p</kbd> | Go to your Projects page. | -| <kbd>Shift</kbd> + <kbd>g</kbd> | Go to your Groups page. | -| <kbd>Shift</kbd> + <kbd>a</kbd> | Go to your Activity page. | -| <kbd>Shift</kbd> + <kbd>l</kbd> | Go to your Milestones page. | -| <kbd>Shift</kbd> + <kbd>s</kbd> | Go to your Snippets page. | -| <kbd>s</kbd> / <kbd>/</kbd> | Put cursor in the search bar. | -| <kbd>Shift</kbd> + <kbd>i</kbd> | Go to your Issues page. | -| <kbd>Shift</kbd> + <kbd>m</kbd> | Go to your [Merge requests](project/merge_requests/index.md) page. | -| <kbd>Shift</kbd> + <kbd>t</kbd> | Go to your To-Do List page. | -| <kbd>p</kbd> then <kbd>b</kbd> | Show or hide the Performance Bar. | -| <kbd>g</kbd> then <kbd>x</kbd> | Toggle between [GitLab](https://gitlab.com/) and [GitLab Next](https://next.gitlab.com/) (GitLab SaaS only). | -| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | +| Keyboard shortcut | Description | +|------------------------------------|-------------| +| <kbd>?</kbd> | Show or hide the shortcut reference sheet. | +| <kbd>Shift</kbd> + <kbd>p</kbd> | Go to your Projects page. | +| <kbd>Shift</kbd> + <kbd>g</kbd> | Go to your Groups page. | +| <kbd>Shift</kbd> + <kbd>a</kbd> | Go to your Activity page. | +| <kbd>Shift</kbd> + <kbd>l</kbd> | Go to your Milestones page. | +| <kbd>Shift</kbd> + <kbd>s</kbd> | Go to your Snippets page. | +| <kbd>s</kbd> / <kbd>/</kbd> | Put cursor in the search bar. | +| <kbd>f</kbd> | Put cursor in the filter bar. | +| <kbd>Shift</kbd> + <kbd>i</kbd> | Go to your Issues page. | +| <kbd>Shift</kbd> + <kbd>m</kbd> | Go to your [Merge requests](project/merge_requests/index.md) page. | +| <kbd>Shift</kbd> + <kbd>t</kbd> | Go to your To-Do List page. | +| <kbd>p</kbd>, then <kbd>b</kbd> | Show or hide the Performance Bar. | +| <kbd>Escape</kbd> | Hide tooltips or popovers. | +| <kbd>g</kbd>, then <kbd>x</kbd> | Toggle between [GitLab](https://gitlab.com/) and [GitLab Next](https://next.gitlab.com/) (GitLab SaaS only). | +| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | Additionally, the following shortcuts are available when editing text in text fields (for example, comments, replies, issue descriptions, and merge request @@ -51,7 +53,7 @@ descriptions): | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | Toggle Markdown preview when editing text in a text field that has **Write** and **Preview** tabs at the top. | | <kbd>Command</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>b</kbd> | Bold the selected text (surround it with `**`). | | <kbd>Command</kbd> + <kbd>i</kbd> | <kbd>Control</kbd> + <kbd>i</kbd> | Italicize the selected text (surround it with `_`). | -| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | Strike through the selected text (surround it with `~~`). | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | Strike through the selected text (surround it with `~~`). | | <kbd>Command</kbd> + <kbd>k</kbd> | <kbd>Control</kbd> + <kbd>k</kbd> | Add a link (surround the selected text with `[]()`). | The shortcuts for editing in text fields are always enabled, even if other @@ -79,13 +81,14 @@ relatively quickly to work, and they take you to another page in the project. | <kbd>g</kbd> + <kbd>j</kbd> | Go to the CI/CD jobs list (**CI/CD > Jobs**). | | <kbd>g</kbd> + <kbd>l</kbd> | Go to the project metrics (**Monitor > Metrics**). | | <kbd>g</kbd> + <kbd>e</kbd> | Go to the project environments (**Deployments > Environments**). | -| <kbd>g</kbd> + <kbd>k</kbd> | Go to the project Kubernetes cluster integration page (**Infrastructure > Kubernetes clusters**). Note that you must have at least [`maintainer` permissions](permissions.md) to access this page. | +| <kbd>g</kbd> + <kbd>k</kbd> | Go to the project Kubernetes cluster integration page (**Infrastructure > Kubernetes clusters**). You must have at least [`maintainer` permissions](permissions.md) to access this page. | | <kbd>g</kbd> + <kbd>s</kbd> | Go to the project snippets list (**Snippets**). | | <kbd>g</kbd> + <kbd>w</kbd> | Go to the [project wiki](project/wiki/index.md) (**Wiki**), if enabled. | +| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | -### Issues and merge requests +### Issues -These shortcuts are available when viewing issues and [merge requests](project/merge_requests/index.md): +These shortcuts are available when viewing issues: | Keyboard shortcut | Description | |------------------------------|-------------| @@ -94,18 +97,26 @@ These shortcuts are available when viewing issues and [merge requests](project/m | <kbd>m</kbd> | Change milestone. | | <kbd>l</kbd> | Change label. | | <kbd>r</kbd> | Start writing a comment. Pre-selected text is quoted in the comment. Can't be used to reply in a thread. | -| <kbd>n</kbd> | Move to next unresolved discussion (merge requests only). | -| <kbd>p</kbd> | Move to previous unresolved discussion (merge requests only). | -| <kbd>]</kbd> or <kbd>j</kbd> | Move to next file (merge requests only). | -| <kbd>[</kbd> or <kbd>k</kbd> | Move to previous file (merge requests only). | -| <kbd>b</kbd> | Copy source branch name (merge requests only). | | <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | - -Merge requests additionally support the following shortcuts: - -| macOS shortcut | Windows shortcut | Description | -|---------------------------------|---------------------|-------------| -| <kbd>Command</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>p</kbd> | Search for, and then jump to a file for review. | +| <kbd>→</kbd> | Go to the next design. | +| <kbd>←</kbd> | Go to the previous design. | +| <kbd>Escape</kbd> | Close the design. | + +### Merge requests + +These shortcuts are available when viewing [merge requests](project/merge_requests/index.md): + +| macOS shortcut | Windows shortcut | Description | +|-----------------------------------|---------------------|-------------| +| <kbd>]</kbd> or <kbd>j</kbd> | | Move to next file. | +| <kbd>[</kbd> or <kbd>k</kbd> | | Move to previous file. | +| <kbd>Command</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>p</kbd> | Search for, and then jump to a file for review. | +| <kbd>n</kbd> | | Move to next unresolved discussion. | +| <kbd>p</kbd> | | Move to previous unresolved discussion. | +| <kbd>b</kbd> | | Copy source branch name. | +| <kbd>r</kbd> | | Start writing a comment. Pre-selected text is quoted in the comment. Can't be used to reply in a thread. | +| <kbd>c</kbd> | | Move to next commit. | +| <kbd>x</kbd> | | Move to previous commit. | ### Project files @@ -119,15 +130,88 @@ These shortcuts are available when browsing the files in a project (go to | <kbd>Enter</kbd> | Open selection. | | <kbd>Escape</kbd> | Go back to file list screen (only while searching for files, **Repository > Files**, then select **Find File**). | | <kbd>y</kbd> | Go to file permalink (only while viewing a file). | -| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | +| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | ### Web IDE These shortcuts are available when editing a file with the [Web IDE](project/web_ide/index.md): -| macOS shortcut | Windows shortcut | Description | +| macOS shortcut | Windows/Linux shortcut | Description | |---------------------------------|---------------------|-------------| -| <kbd>Command</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>p</kbd> | Search for, and then open another file for editing. | +| <kbd>Option</kbd> + <kbd>Command</kbd> + <kbd>↑</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>↑</kbd> | Add cursor above | +| <kbd>Option</kbd> + <kbd>Command</kbd> + <kbd>↓</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>↓</kbd> | Add cursor below | +| <kbd>Shift</kbd> + <kbd>Option</kbd> + <kbd>I</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>I</kbd> | Add cursors to line ends | +| <kbd>Command</kbd> + <kbd>K</kbd>, <kbd>Command</kbd> + <kbd>C</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, <kbd>Control</kbd> + <kbd>C</kbd> _or_ <kbd>Control</kbd> + <kbd>/</kbd> | Add line comment | +| <kbd>Command</kbd> + <kbd>D</kbd> | <kbd>Control</kbd> + <kbd>D</kbd> | Add selection to next find match | +| <kbd>Command</kbd> + <kbd>F2</kbd> | <kbd>Control</kbd> + <kbd>F2</kbd> | Change all occurrences | +| <kbd>F1</kbd> | <kbd>F1</kbd> | Command palette | +| <kbd>Shift</kbd> + <kbd>Option</kbd> + <kbd>↓</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>↓</kbd> | Copy line down | +| <kbd>Shift</kbd> + <kbd>Option</kbd> + <kbd>↑</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>↑</kbd> | Copy line up [(Linux note)](#linux-shortcuts) | +| <kbd>Command</kbd> + <kbd>U</kbd> | <kbd>Control</kbd> + <kbd>U</kbd> | Cursor undo | +| <kbd>Command</kbd> + <kbd>Backspace</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Backspace</kbd> | Delete all left | +| <kbd>Control</kbd> + <kbd>K</kbd> | | Delete all right | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>K</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>K</kbd> | Delete line | +| | <kbd>Control</kbd> + <kbd>Backspace</kbd> | Delete word | +| <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>→</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>→</kbd> | Expand selection | +| <kbd>Command</kbd> + <kbd>P</kbd> | <kbd>Control</kbd> + <kbd>P</kbd> | File finder | +| <kbd>Command</kbd> + <kbd>F</kbd> | <kbd>Control</kbd> + <kbd>F</kbd> | Find | +| <kbd>Enter</kbd> | <kbd>Enter</kbd> or <kbd>F3</kbd> | Find next | +| <kbd>Command</kbd> + <kbd>F3</kbd> | <kbd>F3</kbd> | Find next selection [(Linux note)](#linux-shortcuts) | +| <kbd>Shift</kbd> + <kbd>Enter</kbd> + <kbd>F3</kbd> | <kbd>Shift</kbd> + <kbd>F3</kbd> | Find previous | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>F3</kbd> | <kbd>Shift</kbd> + <kbd>F3</kbd> | Find previous selection | +| <kbd>Command</kbd> + <kbd>E</kbd> | | Find with selection | +| <kbd>Option</kbd> + <kbd>Command</kbd> + <kbd>[</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>[</kbd> | Fold | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>O</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>O</kbd> | Fold all | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>/</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>/</kbd> | Fold all block comments | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>8</kbd> | Fold all regions | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>-</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>-</kbd> | Fold all regions except selected | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>1</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>1</kbd> | Fold level 1 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>2</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>2</kbd> | Fold level 2 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>3</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>3</kbd> | Fold level 3 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>4</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>4</kbd> | Fold level 4 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>5</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>5</kbd> | Fold level 5 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>6</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>6</kbd> | Fold level 6 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>7</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>7</kbd> | Fold level 7 | +| <kbd>Command</kbd> + <kbd>K</kbd> , then <kbd>Command</kbd> + <kbd>[</kbd> | <kbd>Control</kbd> + <kbd>K</kbd> , then <kbd>Control</kbd> + <kbd>[</kbd> | Fold recursively | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>\</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>\</kbd> | Go to bracket | +| <kbd>Control</kbd> + <kbd>G</kbd> | <kbd>Control</kbd> + <kbd>G</kbd> | Go to line or column | +| <kbd>Option</kbd> + <kbd>F8</kbd> | <kbd>Alt</kbd> + <kbd>F8</kbd> | Go to next problem (error, warning, information) | +| <kbd>F8</kbd> | <kbd>F8</kbd> | Go to next problem in files (error, warning, information) | +| <kbd>Shift</kbd> + <kbd>Option</kbd> + <kbd>F8</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>F8</kbd> | Go to previous problem (error, warning, information) | +| <kbd>Shift</kbd> + <kbd>F8</kbd> | <kbd>Shift</kbd> + <kbd>F8</kbd> | Go to previous problem in files (error, warning, information) | +| <kbd>Command</kbd> + <kbd>]</kbd> | <kbd>Control</kbd> + <kbd>]</kbd> | Indent line | +| <kbd>Shift</kbd> + <kbd>Command</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Enter</kbd> | Insert line above | +| <kbd>Command</kbd> + <kbd>Enter</kbd> | <kbd>Control</kbd> + <kbd>Enter</kbd> | Insert line below | +| <kbd>Control</kbd> + <kbd>J</kbd> | <kbd>Control</kbd> + <kbd>J</kbd> | Join lines [(Linux note)](#linux-shortcuts) | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>D</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>D</kbd> | Move last selection to next find match | +| <kbd>Option</kbd> + <kbd>↓</kbd> | <kbd>Alt</kbd> + <kbd>↓</kbd> | Move line down | +| <kbd>Option</kbd> + <kbd>↑</kbd> | <kbd>Alt</kbd> + <kbd>↑</kbd> | Move line up | +| <kbd>Command</kbd> + <kbd>[</kbd> | <kbd>Control</kbd> + <kbd>[</kbd> | Outdent line | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>P</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd> | Preview Markdown [(Linux note)](#linux-shortcuts) | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>U</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>U</kbd> or <kbd>Control</kbd> + <kbd>/</kbd> | Remove line comment | +| <kbd>Option</kbd> + <kbd>Command</kbd> + <kbd>F</kbd> | <kbd>Control</kbd> + <kbd>F</kbd> | Replace | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>.</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>.</kbd> | Replace with next value | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>,</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>,</kbd> | Replace with previous value | +| <kbd>Command</kbd> + <kbd>S</kbd> | <kbd>Control</kbd> + <kbd>S</kbd> | Save files | +| <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>L</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>L</kbd> | Select all occurrences of find match | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>B</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>B</kbd> | Set selection anchor | +| <kbd>Option</kbd> + <kbd>F1</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>F1</kbd> | Show accessibility help | +| <kbd>Shift</kbd> + <kbd>F10</kbd> | <kbd>Shift</kbd> + <kbd>F10</kbd> | Show editor context menu | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>I</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>I</kbd> | Show hover | +| <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>Command</kbd> + <kbd>←</kbd> | <kbd>Shift</kbd> + <kbd>Alt</kbd> + <kbd>←</kbd> | Shrink selection | +| <kbd>Shift</kbd> + <kbd>Option</kbd> + <kbd>A</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>A</kbd> | Toggle block comment | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>L</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>L</kbd> | Toggle fold | +| <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>M</kbd> | <kbd>Control</kbd> + <kbd>M</kbd> | Toggle Tab key moves focus | +| <kbd>Command</kbd> + <kbd>/</kbd> | <kbd>Control</kbd> + <kbd>/</kbd> | Toggle line comment | +| <kbd>Control</kbd> + <kbd>T</kbd> | | Transpose letters | +| <kbd>Control</kbd> + <kbd>Space</kbd> | <kbd>Control</kbd> + <kbd>Space</kbd> | Trigger Suggest | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>X</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>X</kbd> | Trim trailing whitespace | +| <kbd>Option</kbd> + <kbd>Command</kbd> + <kbd>]</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>]</kbd> | Unfold | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>J</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>J</kbd> | Unfold all | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>9</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>9</kbd> | Unfold all regions | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>=</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>=</kbd> | Unfold all regions except selected | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>]</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>]</kbd> | Unfold recursively | +| <kbd>Command</kbd> + <kbd>K</kbd>, then <kbd>Command</kbd> + <kbd>X</kbd> | <kbd>Control</kbd> + <kbd>K</kbd>, then <kbd>Control</kbd> + <kbd>X</kbd> | Trim trailing whitespace | | <kbd>Command</kbd> + <kbd>Enter</kbd> | <kbd>Control</kbd> + <kbd>Enter</kbd> | Commit (when editing the commit message). | ### Repository graph @@ -148,8 +232,8 @@ page (go to **Repository > Graph**): This shortcut is available when viewing a [wiki page](project/wiki/index.md): -| Keyboard shortcut | Description | -|-------------------|-------------| +| Keyboard shortcut | Description | +|-------------------|-----------------| | <kbd>e</kbd> | Edit wiki page. | ### Content editor @@ -183,7 +267,7 @@ These shortcuts are available when editing a file with the | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | Apply heading style 5 | | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | Apply heading style 6 | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | Ordered list | -| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | Bullet list | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | Unordered list | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | Task list | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | Blockquote | | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>c</kbd> | Code block | @@ -194,13 +278,13 @@ These shortcuts are available when editing a file with the #### Text selection -| macOS shortcut | Windows shortcut | Description | -|----------------|------------------|-------------| -| <kbd>Command</kbd> + <kbd>a</kbd> | <kbd>Control</kbd> + <kbd>a</kbd> | Select all | -| <kbd>Shift</kbd> + <kbd>←</kbd> | <kbd>Shift</kbd> + <kbd>←</kbd> | Extend selection one character to left | -| <kbd>Shift</kbd> + <kbd>→</kbd> | <kbd>Shift</kbd> + <kbd>→</kbd> | Extend selection one character to right | -| <kbd>Shift</kbd> + <kbd>↑</kbd> | <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection one line up | -| <kbd>Shift</kbd> + <kbd>↓</kbd> | <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection one line down | +| macOS shortcut | Windows shortcut | Description | +|-----------------------------------|-----------------------------------|-------------| +| <kbd>Command</kbd> + <kbd>a</kbd> | <kbd>Control</kbd> + <kbd>a</kbd> | Select all | +| <kbd>Shift</kbd> + <kbd>←</kbd> | <kbd>Shift</kbd> + <kbd>←</kbd> | Extend selection one character to left | +| <kbd>Shift</kbd> + <kbd>→</kbd> | <kbd>Shift</kbd> + <kbd>→</kbd> | Extend selection one character to right | +| <kbd>Shift</kbd> + <kbd>↑</kbd> | <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection one line up | +| <kbd>Shift</kbd> + <kbd>↓</kbd> | <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection one line down | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection to the beginning of the document | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection to the end of the document | @@ -217,11 +301,23 @@ These shortcuts are available when using a [filtered search input](search/index. These shortcuts are available when viewing [epics](group/epics/index.md): -| Keyboard shortcut | Description | -|-------------------|-------------| +| Keyboard shortcut | Description | +|-------------------|-------------------| | <kbd>r</kbd> | Start writing a comment. Pre-selected text is quoted in the comment. Can't be used to reply in a thread. | | <kbd>e</kbd> | Edit description. | -| <kbd>l</kbd> | Change label. | +| <kbd>l</kbd> | Change label. | + +## Metrics + +These shortcuts are available when using metrics: + +| Keyboard shortcut | Description | +|-------------------|---------------------| +| <kbd>e</kbd> | Expand panel. | +| <kbd>l</kbd> | View logs. | +| <kbd>d</kbd> | Download CSV. | +| <kbd>c</kbd> | Copy link to chart. | +| <kbd>a</kbd> | Alerts. | ## Disable keyboard shortcuts @@ -232,3 +328,10 @@ To disable keyboard shortcuts: 1. While viewing a page that supports keyboard shortcuts, and outside a text box, press <kbd>?</kbd> to display the list of shortcuts. 1. Select **Toggle shortcuts**. + +## Troubleshooting + +### Linux shortcuts + +Linux users may encounter GitLab keyboard shortcuts that are overridden by +their operating system, or their browser. diff --git a/doc/user/ssh.md b/doc/user/ssh.md index 54d4722ee2b..27bb7124afe 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -1,8 +1,7 @@ --- stage: Manage group: Authentication and Authorization -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" -type: howto, reference +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Use SSH keys to communicate with GitLab **(FREE)** @@ -41,7 +40,7 @@ Administrators can [restrict which keys are permitted and their minimum lengths] The book [Practical Cryptography With Go](https://leanpub.com/gocrypto/read#leanpub-auto-chapter-5-digital-signatures) suggests that [ED25519](https://ed25519.cr.yp.to/) keys are more secure and performant than RSA keys. -OpenSSH 6.5 introduced ED25519 SSH keys in 2014 and they should be available on most +OpenSSH 6.5 introduced ED25519 SSH keys in 2014, and they should be available on most operating systems. ### ED25519_SK SSH keys @@ -60,7 +59,7 @@ must have [OpenSSH 8.2](https://www.openssh.com/releasenotes.html#8.2) or later ### RSA SSH keys -Available documentation suggests that ED25519 is more secure than RSA. +Available documentation suggests ED25519 is more secure than RSA. If you use an RSA key, the US National Institute of Science and Technology in [Publication 800-57 Part 3 (PDF)](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-57Pt3r1.pdf) @@ -71,27 +70,27 @@ Review the `man` page for your installed `ssh-keygen` command for details. Before you create a key pair, see if a key pair already exists. -1. On Windows, Linux, or macOS, go to your home directory. +1. Go to your home directory. 1. Go to the `.ssh/` subdirectory. If the `.ssh/` subdirectory doesn't exist, you are either not in the home directory, or you haven't used `ssh` before. In the latter case, you need to [generate an SSH key pair](#generate-an-ssh-key-pair). 1. See if a file with one of the following formats exists: - | Algorithm | Public key | Private key | - | --------- | ---------- | ----------- | + | Algorithm | Public key | Private key | + |-----------------------|------------|-------------| | ED25519 (preferred) | `id_ed25519.pub` | `id_ed25519` | - | ED25519_SK | `id_ed25519_sk.pub` | `id_ed25519_sk` | - | ECDSA_SK | `id_ecdsa_sk.pub` | `id_ecdsa_sk` | - | RSA (at least 2048-bit key size) | `id_rsa.pub` | `id_rsa` | - | DSA (deprecated) | `id_dsa.pub` | `id_dsa` | - | ECDSA | `id_ecdsa.pub` | `id_ecdsa` | + | ED25519_SK | `id_ed25519_sk.pub` | `id_ed25519_sk` | + | ECDSA_SK | `id_ecdsa_sk.pub` | `id_ecdsa_sk` | + | RSA (at least 2048-bit key size) | `id_rsa.pub` | `id_rsa` | + | DSA (deprecated) | `id_dsa.pub` | `id_dsa` | + | ECDSA | `id_ecdsa.pub` | `id_ecdsa` | ## Generate an SSH key pair -If you do not have an existing SSH key pair, generate a new one. +If you do not have an existing SSH key pair, generate a new one: 1. Open a terminal. -1. Type `ssh-keygen -t` followed by the key type and an optional comment. +1. Run `ssh-keygen -t` followed by the key type and an optional comment. This comment is included in the `.pub` file that's created. You may want to use an email address for the comment. @@ -107,7 +106,7 @@ If you do not have an existing SSH key pair, generate a new one. ssh-keygen -t rsa -b 2048 -C "<comment>" ``` -1. Press Enter. Output similar to the following is displayed: +1. Press <kbd>Enter</kbd>. Output similar to the following is displayed: ```plaintext Generating public/private ed25519 key pair. @@ -126,11 +125,10 @@ If you do not have an existing SSH key pair, generate a new one. Enter same passphrase again: ``` -1. A confirmation is displayed, including information about where your files are stored. + A confirmation is displayed, including information about where your files are stored. -A public and private key are generated. -[Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account) and keep -the private key secure. +A public and private key are generated. [Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account) +and keep the private key secure. ### Configure SSH to point to a different directory @@ -158,7 +156,7 @@ configure your SSH client to point to the directory where the private key is sto IdentityFile ~/.ssh/example_com_rsa ``` - For more information on these settings, see the [`man ssh_config`](https://man.openbsd.org/ssh_config) page in the SSH configuration manual. +For more information on these settings, see the [`man ssh_config`](https://man.openbsd.org/ssh_config) page in the SSH configuration manual. Public SSH keys must be unique to GitLab because they bind to your account. Your SSH key is the only identifier you have when you push code with SSH. @@ -166,7 +164,7 @@ It must uniquely map to a single user. ### Update your SSH key passphrase -You can update the passphrase for your SSH key. +You can update the passphrase for your SSH key: 1. Open a terminal and run this command: @@ -174,34 +172,32 @@ You can update the passphrase for your SSH key. ssh-keygen -p -f /path/to/ssh_key ``` -1. At the prompts, type the passphrase and press Enter. +1. At the prompts, enter the passphrase and then press <kbd>Enter</kbd>. ### Upgrade your RSA key pair to a more secure format -If your version of OpenSSH is between 6.5 and 7.8, -you can save your private RSA SSH keys in a more secure -OpenSSH format. - -1. Open a terminal and run this command: +If your version of OpenSSH is between 6.5 and 7.8, you can save your private +RSA SSH keys in a more secure OpenSSH format by opening a terminal and running +this command: - ```shell - ssh-keygen -o -f ~/.ssh/id_rsa - ``` +```shell +ssh-keygen -o -f ~/.ssh/id_rsa +``` - Alternatively, you can generate a new RSA key with the more secure encryption format with - the following command: +Alternatively, you can generate a new RSA key with the more secure encryption format with +the following command: - ```shell - ssh-keygen -o -t rsa -b 4096 -C "<comment>" - ``` +```shell +ssh-keygen -o -t rsa -b 4096 -C "<comment>" +``` ## Generate an SSH key pair for a FIDO/U2F hardware security key -To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later. +To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later: 1. Insert a hardware security key into your computer. 1. Open a terminal. -1. Type `ssh-keygen -t` followed by the key type and an optional comment. +1. Run `ssh-keygen -t` followed by the key type and an optional comment. This comment is included in the `.pub` file that's created. You may want to use an email address for the comment. @@ -229,7 +225,7 @@ To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later. from the security key by [`ssh-add -K`](https://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/ssh-add.1#K) or [`ssh-keygen -K`](https://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/ssh-keygen#K). -1. Select Enter. Output similar to the following is displayed: +1. Press <kbd>Enter</kbd>. Output similar to the following is displayed: ```plaintext Generating public/private ed25519-sk key pair. @@ -251,31 +247,31 @@ To generate ED25519_SK or ECDSA_SK SSH keys, you must use OpenSSH 8.2 or later. Enter same passphrase again: ``` -1. A confirmation is displayed, including information about where your files are stored. + A confirmation is displayed, including information about where your files are stored. A public and private key are generated. [Add the public SSH key to your GitLab account](#add-an-ssh-key-to-your-gitlab-account). ## Add an SSH key to your GitLab account -To use SSH with GitLab, copy your public key to your GitLab account. +To use SSH with GitLab, copy your public key to your GitLab account: 1. Copy the contents of your public key file. You can do this manually or use a script. For example, to copy an ED25519 key to the clipboard: - **macOS:** + **macOS** ```shell tr -d '\n' < ~/.ssh/id_ed25519.pub | pbcopy ``` - **Linux** (requires the `xclip` package): + **Linux** (requires the `xclip` package) ```shell xclip -sel clip < ~/.ssh/id_ed25519.pub ``` - **Git Bash on Windows:** + **Git Bash on Windows** ```shell cat ~/.ssh/id_ed25519.pub | clip @@ -298,8 +294,6 @@ To use SSH with GitLab, copy your public key to your GitLab account. - GitLab 13.12 and earlier, the expiration date is informational only. It doesn't prevent you from using the key. Administrators can view expiration dates and use them for guidance when [deleting keys](admin_area/credentials_inventory.md#delete-a-users-ssh-key). - - GitLab 14.0 and later, the expiration date is enforced. Administrators can - [allow expired keys to be used](admin_area/settings/account_and_limit_settings.md#allow-expired-ssh-keys-to-be-used-deprecated). - GitLab checks all SSH keys at 02:00 AM UTC every day. It emails an expiration notice for all SSH keys that expire on the current date. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) - GitLab checks all SSH keys at 01:00 AM UTC every day. It emails an expiration notice for all SSH keys that are scheduled to expire seven days from now. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) 1. Select **Add key**. @@ -308,9 +302,12 @@ To use SSH with GitLab, copy your public key to your GitLab account. Verify that your SSH key was added correctly. +The following commands use the example hostname `gitlab.example.com`. Replace this example hostname with your GitLab instance's hostname, for example, `git@gitlab.com`. + 1. For GitLab.com, to ensure you're connecting to the correct server, confirm the [SSH host keys fingerprints](gitlab_com/index.md#ssh-host-keys-fingerprints). -1. Open a terminal and run this command, replacing `gitlab.example.com` with your GitLab instance URL: +1. Open a terminal and run this command, replacing `gitlab.example.com` with your + GitLab instance URL: ```shell ssh -T git@gitlab.example.com @@ -326,7 +323,7 @@ Verify that your SSH key was added correctly. Warning: Permanently added 'gitlab.example.com' (ECDSA) to the list of known hosts. ``` - Type `yes` and press Enter. + Type `yes` and press <kbd>Enter</kbd>. 1. Run the `ssh -T git@gitlab.example.com` command again. You should receive a _Welcome to GitLab, `@username`!_ message. @@ -352,10 +349,10 @@ on `ssh` command options, see the `man` pages for both `ssh` and `ssh_config`. ## Use different accounts on a single GitLab instance -You can use multiple accounts to connect to a single instance of GitLab. -You can do this by using the command in the [previous topic](#use-different-keys-for-different-repositories). -However, even if you set `IdentitiesOnly` to `yes`, you cannot sign in if an `IdentityFile` exists -outside of a `Host` block. +You can use multiple accounts to connect to a single instance of GitLab. You +can do this by using the command in the [previous topic](#use-different-keys-for-different-repositories). +However, even if you set `IdentitiesOnly` to `yes`, you cannot sign in if an +`IdentityFile` exists outside of a `Host` block. Instead, you can assign aliases to hosts in the `~.ssh/config` file. diff --git a/doc/user/todos.md b/doc/user/todos.md index 5cea619c830..c261d719da0 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -84,28 +84,25 @@ You can manually add an item to your To-Do List.  -## Create a to-do item by directly addressing someone +## Create a to-do item by mentioning someone -You can create a to-do item by directly addressing someone at the start of a line. -For example, in the following comment: +You can create a to-do item by mentioning someone anywhere except for a code block. Mentioning a user many times in one message only creates one to-do item. -```markdown +For example, from the following comment, everyone except `frank` gets a to-do item created for them: + +````markdown @alice What do you think? cc: @bob - @carol can you please have a look? > @dan what do you think? -@erin @frank thank you! -``` - -The people who receive to-do items are `@alice`, `@erin`, and -`@frank`. +Hey @erin, this is what they said: -To view to-do items where a user was directly addressed, go to the To-Do List and -from the **Action** filter, select **Directly addressed**. - -Mentioning a user many times only creates one to-do item. +``` +Hi, please message @frank :incoming_envelope: +``` +```` ## Actions that mark a to-do item as done diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 84a2449f481..21aa93d3f8b 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -48,6 +48,19 @@ The following storage usage statistics are available to a maintainer: - Total excess storage used: Total amount of storage used that exceeds their allocated storage. - Purchased storage available: Total storage that has been purchased but is not yet used. +## Manage your storage usage + +You can use several methods to manage and reduce your usage for some storage types. + +For more information, see the following pages: + +- [Reduce package registry storage](packages/package_registry/reduce_package_registry_storage.md) +- [Reduce dependency proxy storage](packages/dependency_proxy/reduce_dependency_proxy_storage.md) +- [Reduce repository size](project/repository/reducing_the_repo_size_using_git.md) +- [Reduce container registry storage](packages/container_registry/reduce_container_registry_storage.md) +- [Reduce container registry data transfers](packages/container_registry/reduce_container_registry_data_transfer.md) +- [Reduce wiki repository size](../administration/wikis/index.md#reduce-wiki-repository-size) + ## Excess storage usage Excess storage usage is the amount that a project's repository exceeds the free storage quota. If no |
