diff options
Diffstat (limited to 'doc/user')
208 files changed, 3663 insertions, 1665 deletions
diff --git a/doc/user/admin_area/activating_deactivating_users.md b/doc/user/admin_area/activating_deactivating_users.md index 448c65038c2..29f162616bf 100644 --- a/doc/user/admin_area/activating_deactivating_users.md +++ b/doc/user/admin_area/activating_deactivating_users.md @@ -44,7 +44,7 @@ Please note that for the deactivation option to be visible to an admin, the user Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). NOTE: **Note:** -A deactivated user does not consume a [seat](../../subscriptions/index.md#choosing-the-number-of-users). +A deactivated user does not consume a [seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). ## Activating a user @@ -63,7 +63,7 @@ Users can also be activated using the [GitLab API](../../api/users.md#activate-u NOTE: **Note:** Activating a user will change the user's state to active and it consumes a -[seat](../../subscriptions/index.md#choosing-the-number-of-users). +[seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). TIP: **Tip:** A deactivated user can also activate their account themselves by simply logging back in via the UI. diff --git a/doc/user/admin_area/analytics/convdev.md b/doc/user/admin_area/analytics/convdev.md new file mode 100644 index 00000000000..3ffda3f4400 --- /dev/null +++ b/doc/user/admin_area/analytics/convdev.md @@ -0,0 +1,5 @@ +--- +redirect_to: 'dev_ops_report.md' +--- + +This document was moved to [another location](dev_ops_report.md). diff --git a/doc/user/instance_statistics/dev_ops_score.md b/doc/user/admin_area/analytics/dev_ops_report.md index 35dcbd01916..f04bd69b76b 100644 --- a/doc/user/instance_statistics/dev_ops_score.md +++ b/doc/user/admin_area/analytics/dev_ops_report.md @@ -1,23 +1,25 @@ -# DevOps Score +# DevOps Report > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30469) in GitLab 9.3. > - [Renamed from Conversational Development Index](https://gitlab.com/gitlab-org/gitlab/-/issues/20976) in GitLab 12.6. NOTE: **Note:** -Your GitLab instance's [usage ping](../admin_area/settings/usage_statistics.md#usage-ping-core-only) must be activated in order to use this feature. +Your GitLab instance's [usage ping](../settings/usage_statistics.md#usage-ping) must be activated in order to use this feature. -The DevOps Score gives you an overview of your entire instance's adoption of +The DevOps Report gives you an overview of your entire instance's adoption of [Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/) from planning to monitoring. -This displays the usage of these GitLab features over +## DevOps Score + +DevOps Score displays the usage of GitLab's major features on your instance over the last 30 days, averaged over the number of active users in that time period. It also provides a Lead score per feature, which is calculated based on GitLab's analysis -of top-performing instances based on [usage ping data](../admin_area/settings/usage_statistics.md#usage-ping-core-only) that GitLab has +of top-performing instances based on [usage ping data](../settings/usage_statistics.md#usage-ping) that GitLab has collected. Your score is compared to the lead score of each feature and then expressed as a percentage at the bottom of said feature. -Your overall **index score** is an average of your feature scores. You can use this score to compare your DevOps status to other organizations. +Your overall **DevOps Score** is an average of your feature scores. You can use this score to compare your DevOps status to other organizations. - + The page also provides helpful links to articles and GitLab docs, to help you improve your scores. diff --git a/doc/user/admin_area/analytics/img/cohorts_v13_4.png b/doc/user/admin_area/analytics/img/cohorts_v13_4.png Binary files differnew file mode 100644 index 00000000000..4af1841a033 --- /dev/null +++ b/doc/user/admin_area/analytics/img/cohorts_v13_4.png diff --git a/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png b/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png Binary files differnew file mode 100644 index 00000000000..1fa070a6915 --- /dev/null +++ b/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png diff --git a/doc/user/admin_area/analytics/index.md b/doc/user/admin_area/analytics/index.md new file mode 100644 index 00000000000..b3336b471f8 --- /dev/null +++ b/doc/user/admin_area/analytics/index.md @@ -0,0 +1,10 @@ +# Instance-level analytics + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41416) in GitLab 11.2. + +Administrators have access to instance-wide analytics, as shown in **Admin Area > Analytics**. + +There are two kinds of statistics: + +- [DevOps Report](dev_ops_report.md): Provides an overview of your entire instance's feature usage. +- [User Cohorts](user_cohorts.md): Display the monthly cohorts of new users and their activities over time. diff --git a/doc/user/instance_statistics/user_cohorts.md b/doc/user/admin_area/analytics/user_cohorts.md index 8da2d57d474..faf8caa7e00 100644 --- a/doc/user/instance_statistics/user_cohorts.md +++ b/doc/user/admin_area/analytics/user_cohorts.md @@ -2,7 +2,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/23361) in GitLab 9.1. -As a benefit of having the [usage ping active](../admin_area/settings/usage_statistics.md), +As a benefit of having the [usage ping active](../settings/usage_statistics.md), GitLab lets you analyze the users' activities over time of your GitLab installation. ## Overview @@ -10,12 +10,12 @@ GitLab lets you analyze the users' activities over time of your GitLab installat How do we read the user cohorts table? Let's take an example with the following user cohorts. - + -For the cohort of Jan 2018, 15 users have been added on this server and have -been active since this month. One month later, in Feb 2018, all 15 users are -still active. 6 months later (Month 6, July), we can see 10 users from this cohort -are active, or 66% of the original cohort of 15 that joined in January. +For the cohort of March 2020, three users have been added on this server and have +been active since this month. One month later, in April 2020, two users are +still active. Five months later (August), we can see that one user from this cohort +is active, or 33% of the original cohort of three that joined in March. The Inactive users column shows the number of users who have been added during the month, but who have never actually had any activity in the instance. diff --git a/doc/user/admin_area/blocking_unblocking_users.md b/doc/user/admin_area/blocking_unblocking_users.md index 2f98709a089..d8dde317d38 100644 --- a/doc/user/admin_area/blocking_unblocking_users.md +++ b/doc/user/admin_area/blocking_unblocking_users.md @@ -33,7 +33,7 @@ Personal projects, and group and user history of the blocked user will be left i Users can also be blocked using the [GitLab API](../../api/users.md#block-user). NOTE: **Note:** -A blocked user does not consume a [seat](../../subscriptions/index.md#choosing-the-number-of-users). +A blocked user does not consume a [seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). ## Unblocking a user @@ -48,4 +48,4 @@ Users can also be unblocked using the [GitLab API](../../api/users.md#unblock-us NOTE: **Note:** Unblocking a user will change the user's state to active and it consumes a -[seat](../../subscriptions/index.md#choosing-the-number-of-users). +[seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 9259c93cfa3..7f2d49dafea 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -13,7 +13,7 @@ type: howto GitLab administrators are responsible for the overall security of their instance. To assist, GitLab provides a Credentials inventory to keep track of all the credentials that can be used to access their self-managed instance. -Using Credentials inventory, GitLab administrators can see all the personal access tokens and SSH keys that exist in their instance and: +Using Credentials inventory, you can see all the personal access tokens (PAT) and SSH keys that exist in your GitLab instance. In addition, you can [revoke them](#revoke-a-users-personal-access-token) and see: - Who they belong to. - Their access scope. @@ -25,4 +25,19 @@ To access the Credentials inventory, navigate to **Admin Area > Credentials**. The following is an example of the Credentials inventory page: - + + +## Revoke a user's personal access token + +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214811) in GitLab 13.4. + +If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: + +| Token state | [Token expiry enforced?](settings/account_and_limit_settings.md#optional-enforcement-of-personal-access-token-expiry) | Show Revoke button? | Comments | +|-------------|------------------------|--------------------|----------------------------------------------------------------------------| +| Active | Yes | Yes | Allows administrators to revoke the PAT, such as for a compromised account | +| Active | No | Yes | Allows administrators to revoke the PAT, such as for a compromised account | +| Expired | Yes | No | PAT expires automatically | +| Expired | No | Yes | The administrator may revoke the PAT to prevent indefinite use | +| Revoked | Yes | No | Not applicable; token is already revoked | +| Revoked | No | No | Not applicable; token is already revoked | diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md index d17e0f7430c..28738ed52f3 100644 --- a/doc/user/admin_area/geo_nodes.md +++ b/doc/user/admin_area/geo_nodes.md @@ -8,7 +8,7 @@ type: howto # Geo nodes Admin Area **(PREMIUM ONLY)** You can configure various settings for GitLab Geo nodes. For more information, see -[Geo documentation](../../administration/geo/replication/index.md). +[Geo documentation](../../administration/geo/index.md). On the primary node, go to **Admin Area > Geo**. On secondary nodes, go to **Admin Area > Geo > Nodes**. diff --git a/doc/user/admin_area/img/credentials_inventory_v13_2.png b/doc/user/admin_area/img/credentials_inventory_v13_2.png Binary files differdeleted file mode 100644 index 5b56422a0a3..00000000000 --- a/doc/user/admin_area/img/credentials_inventory_v13_2.png +++ /dev/null diff --git a/doc/user/admin_area/img/credentials_inventory_v13_4.png b/doc/user/admin_area/img/credentials_inventory_v13_4.png Binary files differnew file mode 100644 index 00000000000..06925ea2f6f --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_v13_4.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 8aa50bb0496..58430ab615b 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -20,8 +20,8 @@ The Admin Area is made up of the following sections: | Section | Description | |:-----------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **{overview}** [Overview](#overview-section) | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administering-projects), [users](#administering-users), [groups](#administering-groups), [jobs](#administering-jobs), [Runners](#administering-runners), and [Gitaly servers](#administering-gitaly-servers). | -| **{monitor}** Monitoring | View GitLab [system information](#system-info), and information on [background jobs](#background-jobs), [logs](#logs), [health checks](monitoring/health_check.md), [requests profiles](#requests-profiles), and [audit logs](#audit-log-premium-only). | +| **{overview}** [Overview](#overview-section) | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administering-projects), [users](#administering-users), [groups](#administering-groups), [jobs](#administering-jobs), [runners](#administering-runners), and [Gitaly servers](#administering-gitaly-servers). | +| **{monitor}** Monitoring | View GitLab [system information](#system-info), and information on [background jobs](#background-jobs), [logs](#logs), [health checks](monitoring/health_check.md), [requests profiles](#requests-profiles), and [audit logs](#audit-log). | | **{messages}** Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | **{hook}** System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | | **{applications}** Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | @@ -147,7 +147,7 @@ The following totals are also included: GitLab billing is based on the number of **Active users**, calculated as **Total users** - **Blocked users**. For details of active users, see -[Choosing the number of users](../../subscriptions/index.md#choosing-the-number-of-users). +[Choosing the number of users](../../subscriptions/self_managed/index.md#choose-the-number-of-users). NOTE: **Note:** Users statistics are calculated daily, so user changes made since the last update won't be @@ -196,51 +196,51 @@ For each job, the following details are listed: | Timing | Duration of the job, and how long ago the job completed. | | Coverage | Percentage of tests coverage. | -### Administering Runners +### Administering runners -You can administer all Runners in the GitLab instance from the Admin Area's **Runners** page. See -[GitLab Runner](https://docs.gitlab.com/runner/) for more information on Runner itself. +You can administer all runners in the GitLab instance from the Admin Area's **Runners** page. See +[GitLab Runner](https://docs.gitlab.com/runner/) for more information. To access the **Runners** page, go to **Admin Area > Overview > Runners**. The **Runners** page features: -- A description of Runners, and their possible states. -- Instructions on installing a Runner. -- A list of all registered Runners. +- A description of runners and their possible states. +- Instructions on installing a runner. +- A list of all registered runners. Runners are listed in descending order by the date they were created, by default. You can change the sort order to *Last Contacted* from the dropdown beside the search field. -To search Runners' descriptions: +To search runners' descriptions: -1. In the **Search or filter results...** field, type the description of the Runner you want to +1. In the **Search or filter results...** field, type the description of the runner you want to find. 1. Press Enter. -You can also filter Runners by status, type, and tag. To filter: +You can also filter runners by status, type, and tag. To filter: 1. Click in the **Search or filter results...** field. 1. Select **status:**, **type:**, or **tag:**. 1. Select or enter your search criteria. - + -For each Runner, the following attributes are listed: +For each runner, the following attributes are listed: | Attribute | Description | | ------------ | ----------- | | Type | One or more of the following states: shared, group, specific, locked, or paused | -| Runner token | Token used to identify the Runner, and which the Runner uses to communicate with the GitLab instance | -| Description | Description given to the Runner when it was created | +| Runner token | Token used to identify the runner, and which the runner uses to communicate with the GitLab instance | +| Description | Description given to the runner when it was created | | Version | GitLab Runner version | -| IP address | IP address of the host on which the Runner is registered | -| Projects | Projects to which the Runner is assigned | -| Jobs | Total of jobs run by the Runner | -| Tags | Tags associated with the Runner | -| Last contact | Timestamp indicating when the GitLab instance last contacted the Runner | +| IP address | IP address of the host on which the runner is registered | +| Projects | Projects to which the runner is assigned | +| Jobs | Total of jobs run by the runner | +| Tags | Tags associated with the runner | +| Last contact | Timestamp indicating when the GitLab instance last contacted the runner | -You can also edit, pause, or remove each Runner. +You can also edit, pause, or remove each runner. ### Administering Gitaly servers diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 2c849db66b1..ecbc615f56a 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -125,7 +125,7 @@ before uploading your license. GitLab.com users cannot upload and use a self-managed license. If you wish to use paid features on GitLab.com, a separate subscription may be -[purchased](../../subscriptions/index.md#subscribe-to-gitlabcom). +[purchased](../../subscriptions/gitlab_com/index.md). ### Users exceed license limit upon renewal diff --git a/doc/user/admin_area/monitoring/convdev.md b/doc/user/admin_area/monitoring/convdev.md index 2ba28d4bc1c..d9c3950f2e4 100644 --- a/doc/user/admin_area/monitoring/convdev.md +++ b/doc/user/admin_area/monitoring/convdev.md @@ -1,5 +1,5 @@ --- -redirect_to: '../../instance_statistics/dev_ops_score.md' +redirect_to: '../analytics/dev_ops_report.md' --- -Conversational Development Index was renamed to [DevOps Score](../../instance_statistics/dev_ops_score.md) in GitLab 12.6. +Conversational Development Index was renamed to [DevOps Report](../analytics/dev_ops_report.md) in GitLab 12.6. diff --git a/doc/user/admin_area/monitoring/dev_ops_report.md b/doc/user/admin_area/monitoring/dev_ops_report.md new file mode 100644 index 00000000000..9ad9830ed59 --- /dev/null +++ b/doc/user/admin_area/monitoring/dev_ops_report.md @@ -0,0 +1,5 @@ +--- +redirect_to: '../analytics/dev_ops_report.md' +--- + +This document was moved to [another location](../analytics/dev_ops_report.md). diff --git a/doc/user/admin_area/monitoring/dev_ops_score.md b/doc/user/admin_area/monitoring/dev_ops_score.md deleted file mode 100644 index f8b66531f2f..00000000000 --- a/doc/user/admin_area/monitoring/dev_ops_score.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -redirect_to: '../../instance_statistics/dev_ops_score.md' ---- - -This document was moved to [another location](../../instance_statistics/dev_ops_score.md). diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 329b6ff5bb0..2a38ccb31f0 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -153,7 +153,7 @@ https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` NOTE: **Note:** -In case the database or Redis service are unaccessible, the probe endpoints response is not guaranteed to be correct. +In case the database or Redis service are inaccessible, the probe endpoints response is not guaranteed to be correct. You should switch to [IP whitelist](#ip-whitelist) from deprecated access token to avoid it. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 4651a548ff9..957f84206e9 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -81,7 +81,7 @@ The repository size limit includes repository files and LFS, and does not includ For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). NOTE: **Note:** -GitLab.com repository size [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). +GitLab.com repository size [is set by GitLab](../../gitlab_com/index.md#account-and-limit-settings). ## Troubleshooting @@ -137,7 +137,7 @@ Once a lifetime for personal access tokens is set, GitLab will: > - It is deployed behind a feature flag, disabled by default. > - It is disabled on GitLab.com. > - It is not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-optional-enforcement-of-personal-access-token-expiry-feature-core-only). **(CORE ONLY)** +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-optional-enforcement-of-personal-access-token-expiry-feature). **(CORE ONLY)** GitLab administrators can choose to prevent personal access tokens from expiring automatically. The tokens will be usable after the expiry date, unless they are revoked explicitly. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 0479da7fb52..b4867d33644 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -7,7 +7,7 @@ type: reference # Continuous Integration and Deployment Admin settings **(CORE ONLY)** -In this area, you will find settings for Auto DevOps, Runners, and job artifacts. +In this area, you will find settings for Auto DevOps, runners, and job artifacts. You can find it in the **Admin Area > Settings > CI/CD**.  @@ -86,13 +86,13 @@ be updated for artifacts created before this setting was changed. The administrator may need to manually search for and expire previously-created artifacts, as described in the [troubleshooting documentation](../../../administration/troubleshooting/gitlab_rails_cheat_sheet.md#remove-artifacts-more-than-a-week-old). -## Shared Runners pipeline minutes quota **(STARTER ONLY)** +## Shared runners pipeline minutes quota **(STARTER ONLY)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1078) in GitLab Starter 8.16. -If you have enabled shared Runners for your GitLab instance, you can limit their +If you have enabled shared runners for your GitLab instance, you can limit their usage by setting a maximum number of pipeline minutes that a group can use on -shared Runners per month. Setting this to `0` (default value) will grant +shared runners per month. Setting this to `0` (default value) will grant unlimited pipeline minutes. While build limits are stored as minutes, the counting is done in seconds. Usage resets on the first day of each month. On GitLab.com, the quota is calculated based on your @@ -116,7 +116,7 @@ also change each group's pipeline minutes quota to override the global value. 1. Click **Save changes** for the changes to take effect. Once saved, you can see the build quota in the group admin view. -The quota can also be viewed in the project admin view if shared Runners +The quota can also be viewed in the project admin view if shared runners are enabled.  @@ -196,7 +196,9 @@ To set required pipeline configuration:  -## Package Registry configuration **(PREMIUM ONLY)** +## Package Registry configuration + +### NPM Forwarding **(PREMIUM ONLY)** GitLab administrators can disable the forwarding of NPM requests to [npmjs.com](https://www.npmjs.com/). @@ -208,3 +210,15 @@ To disable it: 1. Click **Save changes**.  + +### Package file size limits + +GitLab administrators can adjust the maximum allowed file size for each package type. + +To set the maximum file size: + +1. Go to **Admin Area > Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Find the package type you would like to adjust. +1. Enter the maximum file size, in bytes. +1. Click **Save size limits**. diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index c5c5f08aea1..0b250e07412 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -22,11 +22,11 @@ known response, the result is cached for 6 hours. If the external authorization is enabled, GitLab will further block pages and functionality that render cross-project data. That includes: -- most pages under Dashboard (Activity, Milestones, Snippets, Assigned merge - requests, Assigned issues, Todos) -- under a specific group (Activity, Contribution analytics, Issues, Issue boards, - Labels, Milestones, Merge requests) -- Global and Group search will be disabled +- Most pages under Dashboard (Activity, Milestones, Snippets, Assigned merge + requests, Assigned issues, To-Do List). +- Under a specific group (Activity, Contribution analytics, Issues, Issue boards, + Labels, Milestones, Merge requests). +- Global and Group search will be disabled. This is to prevent performing to many requests at once to the external authorization service. diff --git a/doc/user/admin_area/settings/img/domain_blacklist.png b/doc/user/admin_area/settings/img/domain_denylist.png Binary files differindex a7e972b7c0a..a7e972b7c0a 100644 --- a/doc/user/admin_area/settings/img/domain_blacklist.png +++ b/doc/user/admin_area/settings/img/domain_denylist.png diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index db6dbb7f38b..ba9bccbf3e7 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -42,7 +42,7 @@ Access the default page for admin area settings by navigating to **Admin Area > | Option | Description | | ------ | ----------- | -| [Repository's custom initial branch name](../../project/repository/branches/index.md#custom-initial-branch-name-core-only) | Set a custom branch name rather than master for all the new repositories created within your instance. | +| [Repository's custom initial branch name](../../project/repository/branches/index.md#custom-initial-branch-name) | Set a custom branch name rather than master for all the new repositories created within your instance. | | [Repository mirror](visibility_and_access_controls.md#allow-mirrors-to-be-set-up-for-projects) | Configure repository mirroring. | | [Repository storage](../../../administration/repository_storage_types.md) | Configure storage path settings. | | Repository maintenance | ([Repository checks](../../../administration/repository_checks.md) and [Housekeeping](../../../administration/housekeeping.md)). Configure automatic Git checks and housekeeping on repositories. | @@ -60,8 +60,8 @@ Access the default page for admin area settings by navigating to **Admin Area > | Option | Description | | ------ | ----------- | | [Continuous Integration and Deployment](continuous_integration.md) | Auto DevOps, runners and job artifacts. | -| [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration-premium-only) **(PREMIUM ONLY)** | Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/README.md). This pipeline configuration will be run after the project's own configuration. | -| [Package Registry](continuous_integration.md#package-registry-configuration-premium-only) | Settings related to the use and experience of using GitLab's Package Registry. Note there are [risks involved](./../../packages/container_registry/index.md#use-with-external-container-registries) in enabling some of these settings. | +| [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration) **(PREMIUM ONLY)** | Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/README.md). This pipeline configuration will be run after the project's own configuration. | +| [Package Registry](continuous_integration.md#package-registry-configuration) | Settings related to the use and experience of using GitLab's Package Registry. Note there are [risks involved](./../../packages/container_registry/index.md#use-with-external-container-registries) in enabling some of these settings. | ## Reporting @@ -106,7 +106,7 @@ Access the default page for admin area settings by navigating to **Admin Area > | [Pages](../../../administration/pages/index.md#custom-domain-verification) | Size and domain settings for static websites | | [Real-time features](../../../administration/polling.md) | Change this value to influence how frequently the GitLab UI polls for updates. | | [Gitaly timeouts](gitaly_timeouts.md) | Configure Gitaly timeouts. | -| Localization | [Default first day of the week](../../profile/preferences.md) and [Time tracking](../../project/time_tracking.md#limit-displayed-units-to-hours-core-only). | +| Localization | [Default first day of the week](../../profile/preferences.md) and [Time tracking](../../project/time_tracking.md#limit-displayed-units-to-hours). | NOTE: **Note:** You can change the [Default first day of the week](../../profile/preferences.md) for the entire GitLab instance diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 8ef5ac8dc8f..80092102091 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -6,14 +6,12 @@ type: reference You can use sign-up restrictions to: -- Disable new signups. +- Disable new sign-ups. - Require user email confirmation. -- Blacklist or whitelist email addresses belonging to specific domains. +- Denylist or allowlist email addresses belonging to specific domains. NOTE: **Note:** -These restrictions are only applied during sign-up from an external user. An admin is -able to add a user through the admin panel with a disallowed domain. Also -note that the users can change their email addresses after signup to +These restrictions are only applied during sign-up from an external user. An admin can add a user through the admin panel with a disallowed domain. Also, note that the users can change their email addresses after sign-up to disallowed domains. ## Disable new signups @@ -26,12 +24,12 @@ You can restrict new users from signing up by themselves for an account in your ### Recommendations -For customers running public facing GitLab instances, we highly recommend that you -consider disabling new signups if you do not expect public users to sign up for an +For customers running public-facing GitLab instances, we highly recommend that you +consider disabling new sign-ups if you do not expect public users to sign up for an account. Alternatively, you could also consider setting up a -[whitelist](#whitelist-email-domains) or [blacklist](#blacklist-email-domains) on +[allowlist](#allowlist-email-domains) or [denylist](#denylist-email-domains) on email domains to prevent malicious users from creating accounts. ## Require email confirmation @@ -48,14 +46,14 @@ their email address before they are allowed to sign in. You can [change](../../../security/password_length_limits.md#modify-minimum-password-length-using-gitlab-ui) the minimum number of characters a user must have in their password using the GitLab UI. -## Whitelist email domains +## Allowlist email domains > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/598) in GitLab 7.11.0 -You can restrict users to only sign up using email addresses matching the given +You can restrict users only to sign up using email addresses matching the given domains list. -## Blacklist email domains +## Denylist email domains > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5259) in GitLab 8.10. @@ -71,17 +69,17 @@ To access this feature: 1. Navigate to the **Admin Area > Settings > General**. 1. Expand the **Sign-up restrictions** section. -For the blacklist, you can enter the list manually or upload a `.txt` file that +For the denylist, you can enter the list manually or upload a `.txt` file that contains list entries. -For the whitelist, you must enter the list manually. +For the allowlist, you must enter the list manually. -Both the whitelist and blacklist accept wildcards. For example, you can use +Both the allowlist and denylist accept wildcards. For example, you can use `*.company.com` to accept every `company.com` subdomain, or `*.io` to block all domains ending in `.io`. Domains should be separated by a whitespace, semicolon, comma, or a new line. - + <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index f3eb094887e..55bbcfbe1d8 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -60,14 +60,11 @@ sequenceDiagram See [Usage Ping guide](../../../development/telemetry/usage_ping.md). -## Instance statistics visibility **(CORE ONLY)** +## Instance-level statistics **(CORE ONLY)** Once usage ping is enabled, GitLab will gather data from other instances and -will be able to show [usage statistics](../../instance_statistics/index.md) -of your instance to your users. - -To make this visible only to admins, go to **Admin Area > Settings > Metrics and profiling**, expand -**Usage statistics**, and set the **Instance Statistics visibility** option to **Only admins**. +will be able to show [usage statistics](../analytics/index.md) +of your instance to your admins in **Admin Area > Analytics**. <!-- ## Troubleshooting diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index e5c7947399d..95a87378e18 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -78,7 +78,7 @@ CAUTION: **Warning:** The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to [Immediate deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) in GitLab 13.2. -Projects within a group can be deleted after a delayed period, by [configuring in Group Settings](../../group/index.md#enabling-delayed-project-removal-premium). +Projects within a group can be deleted after a delayed period, by [configuring in Group Settings](../../group/index.md#enabling-delayed-project-removal). The default period is 7 days, and can be changed. Setting this period to 0 will enable immediate removal of projects or groups. @@ -92,7 +92,7 @@ To change this period: Alternatively, projects that are marked for removal can be deleted immediately. To do so: -1. [Restore the project](../../project/settings/#restore-a-project-premium). +1. [Restore the project](../../project/settings/#restore-a-project). 1. Delete the project as described in the [Administering Projects page](../../admin_area/#administering-projects). ## Default project visibility diff --git a/doc/user/admin_area/user_cohorts.md b/doc/user/admin_area/user_cohorts.md index 21e61e2ec44..ec0153ac3b6 100644 --- a/doc/user/admin_area/user_cohorts.md +++ b/doc/user/admin_area/user_cohorts.md @@ -1,5 +1,5 @@ --- -redirect_to: '../instance_statistics/user_cohorts.md' +redirect_to: 'analytics/user_cohorts.md' --- -This document was moved to [another location](../instance_statistics/user_cohorts.md). +This document was moved to [another location](analytics/user_cohorts.md). diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index 8c4c54153bb..89acb430a9f 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -10,40 +10,45 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/38062) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.7. -Code Review Analytics makes it easy to view the longest-running reviews among open merge requests, -enabling you to take action on individual merge requests and reduce overall cycle time. +Code Review Analytics makes it easy to view the longest-running reviews among open merge requests and +enables you to: + +1. Take action on individual merge requests. +1. Reduce overall cycle time. NOTE: **Note:** -Initially, no data will appear. Data is populated as users comment on open merge requests. +Initially, no data appears. Data is populated as users comment on open merge requests. ## Overview Code Review Analytics displays a table of open merge requests that have at least one non-author comment. The review time is measured from the time the first non-author comment was submitted. -The code review period for a merge request is automatically identified as the time since the first non-author comment. -To access Code Review Analytics, from your project's menu, go to **{chart}** **Project Analytics > Code Review**. +To access Code Review Analytics, from your project's menu, go to **Project Analytics > Code Review**. + +You can filter the list of merge requests by milestone and label.  -- The table is sorted by review duration, helping you quickly find the longest-running reviews which may need intervention or to be broken down into smaller parts. -- You can filter the list of MRs by milestone and label. -- Columns to display the author, approvers, comment count, and line change (-/+) counts. +The table is sorted by: + +- **Review time**: Helping you to quickly find the longest-running reviews which may need intervention + or to be broken down into smaller parts. +- Other columns: Display the author, approvers, comment count, and line change (-/+) counts. ## Use cases This feature is designed for [development team leaders](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) -and others who want to understand broad code review dynamics, and identify patterns to help explain them. - -You can use Code Review Analytics to expose your team's unique challenges with code review, and -identify improvements that might substantially accelerate your development cycle. +and others who want to understand broad code review dynamics, and identify patterns to explain them. -Code Review Analytics can be used when: +You can use Code Review Analytics to: +- Expose your team's unique challenges with code review. +- Identify improvements that might substantially accelerate your development cycle. - Your team agrees that code review is moving too slow. - The [Value Stream Analytics feature](value_stream_analytics.md) shows that reviews are your team's most time-consuming step. +- Analyze the patterns and trends of different types of work that are moving slow. -You can use Code Review Analytics to see the types of work that are currently moving the slowest, and analyze the patterns -and trends between them. For example: +For example: - Lots of comments or commits? Maybe the code is too complex. - A particular author is involved? Maybe more training is required. diff --git a/doc/user/analytics/img/delete_value_stream_v13.4.png b/doc/user/analytics/img/delete_value_stream_v13.4.png Binary files differnew file mode 100644 index 00000000000..c97fcb76343 --- /dev/null +++ b/doc/user/analytics/img/delete_value_stream_v13.4.png diff --git a/doc/user/analytics/img/merge_request_analytics_v13_3.png b/doc/user/analytics/img/merge_request_analytics_v13_3.png Binary files differdeleted file mode 100644 index f90f3625a51..00000000000 --- a/doc/user/analytics/img/merge_request_analytics_v13_3.png +++ /dev/null diff --git a/doc/user/analytics/img/mr_throughput_chart_v13_3.png b/doc/user/analytics/img/mr_throughput_chart_v13_3.png Binary files differnew file mode 100644 index 00000000000..04fa54f323c --- /dev/null +++ b/doc/user/analytics/img/mr_throughput_chart_v13_3.png diff --git a/doc/user/analytics/img/mr_throughput_table_v13_3.png b/doc/user/analytics/img/mr_throughput_table_v13_3.png Binary files differnew file mode 100644 index 00000000000..63ffb9389f4 --- /dev/null +++ b/doc/user/analytics/img/mr_throughput_table_v13_3.png diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index 47852cb0498..044b9eb3e64 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -6,15 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Analytics -## Analytics workspace +## Instance-level analytics > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12077) in GitLab 12.2. -The Analytics workspace will make it possible to aggregate analytics across +Instance-level analytics make it possible to aggregate analytics across GitLab, so that users can view information across multiple projects and groups in one place. -To access the Analytics workspace, click on **More > Analytics** in the top navigation bar. +[Learn more about instance-level analytics](../admin_area/analytics/index.md). ## Group-level analytics @@ -38,7 +38,8 @@ The following analytics features are available at the project level: - [Code Review](code_review_analytics.md). **(STARTER)** - [Insights](../group/insights/index.md). **(ULTIMATE)** - [Issue](../group/issues_analytics/index.md). **(PREMIUM)** -- [Merge Request](merge_request_analytics.md). **(STARTER)** +- [Merge Request](merge_request_analytics.md), enabled with the `project_merge_request_analytics` + [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(STARTER)** - [Repository](repository_analytics.md). - [Value Stream](value_stream_analytics.md), enabled with the `cycle_analytics` [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(STARTER)** diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index 01295ae888b..6a18d46fd1a 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -5,7 +5,6 @@ group: Analytics info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- - # Merge Request Analytics **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229045) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.3. @@ -14,16 +13,14 @@ Merge Request Analytics helps you understand the efficiency of your code review ## Overview -Merge Request Analytics displays information about all accepted merge requests. +Merge Request Analytics displays information that will help you evaluate the efficiency and productivity of your merge request process. -The Throughput chart shows the number of completed merge requests, by month. Merge request throughput is +The Throughput chart shows the number of merge requests merged, by month. Merge request throughput is a common measure of productivity in software engineering. Although imperfect, the average throughput can be a meaningful benchmark of your team's overall productivity. To access Merge Request Analytics, from your project's menu, go to **Analytics > Merge Request**. - - ## Use cases This feature is designed for [development team leaders](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) @@ -37,6 +34,57 @@ Merge Request Analytics could be used when: - You want to know if you were more productive this month than last month, or 12 months ago. - You want to drill into low- or high-productivity months to understand the work that took place. +## Visualizations and data + +The following visualizations and data are available, representing all merge requests that were merged in the past 12 months. + +### Throughput chart + +The throughput chart shows the number of merge requests merged per month. + + + +### Throughput table + +Data table displaying a maximum of the 100 most recent merge requests merged for the time period. + + + +## Filter the data + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229266) in GitLab 13.4 + +You can filter the data that is presented on the page based on the following parameters: + +- Author +- Assignees +- Labels +- Milestones + +To filter results: + +1. Click on the filter bar. +1. Select a parameter to filter by. +1. Select a value from the autocompleted results, or enter search text to refine the results. + ## Permissions -- On [Starter or Bronze tier](https://about.gitlab.com/pricing/) and above. +The **Merge Request Analytics** feature can be accessed only: + +- On [Starter](https://about.gitlab.com/pricing/) and above. +- By users with [Reporter access](../permissions.md) and above. + +## Enable and disable related feature flags + +Merge Request Analytics is disabled by default but can be enabled using the following +[feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development): + +- `project_merge_request_analytics` + +A GitLab administrator can: + +- Enable this feature by running the following command in a Rails console: + + ```ruby + Feature.enable(:project_merge_request_analytics) + ``` diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 9114b6de6bc..14012d4a28d 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -45,8 +45,6 @@ There are seven stages that are tracked as part of the Value Stream Analytics ca - Time spent on code review - **Staging** (Continuous Deployment) - Time between merging and deploying to production -- **Total** (Total) - - Total lifecycle time. That is, the velocity of the project or team. [Previously known](https://gitlab.com/gitlab-org/gitlab/-/issues/38317) as **Production**. ## Filter the analytics data @@ -95,7 +93,7 @@ Note: A commit is associated with an issue by [crosslinking](../project/issues/c ## How the stages are measured Value Stream Analytics records stage time and data based on the project issues with the -exception of the staging and total stages, where only data deployed to +exception of the staging stage, where only data deployed to production are measured. Specifically, if your CI is not set up and you have not defined a `production` @@ -112,7 +110,6 @@ Each stage of Value Stream Analytics is further described in the table below. | Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. | | Review | Measures the median time taken to review the merge request that has a closing issue pattern, between its creation and until it's merged. | | Staging | Measures the median time between merging the merge request with a closing issue pattern until the very first deployment to production. It's tracked by the environment set to `production` or matching `production/*` (case-sensitive, `Production` won't work) in your GitLab CI/CD configuration. If there isn't a production environment, this is not tracked. | -| Total | The sum of all time (medians) taken to run the entire process, from issue creation to deploying the code to production. [Previously known](https://gitlab.com/gitlab-org/gitlab/-/issues/38317) as **Production**. | How this works, behind the scenes: @@ -131,7 +128,7 @@ Value Stream Analytics dashboard will not present any data for: - Merge requests that do not close an issue. - Issues not labeled with a label present in the Issue Board or for issues not assigned a milestone. -- Staging and production stages, if the project has no `production` or `production/*` +- Staging stage, if the project has no `production` or `production/*` environment. ## Example workflow @@ -158,9 +155,6 @@ environments is configured. request at 19:00. (stop of **Review** stage / start of **Staging** stage). 1. Now that the merge request is merged, a deployment to the `production` environment starts and finishes at 19:30 (stop of **Staging** stage). -1. The cycle completes and the sum of the median times of the previous stages - is recorded to the **Total** stage. That is the time between creating an - issue and deploying its relevant merge request to production. From the above example you can conclude the time it took each stage to complete as long as their total time: @@ -171,10 +165,6 @@ as long as their total time: - **Test**: 5min - **Review**: 5h (19:00 - 14:00) - **Staging**: 30min (19:30 - 19:00) -- **Total**: Since this stage measures the sum of median time of all - previous stages, we cannot calculate it if we don't know the status of the - stages before. In case this is the very first cycle that is run in the project, - then the **Total** time is 10h 30min (19:30 - 09:00) A few notes: @@ -267,7 +257,7 @@ Once a custom stage has been added, you can "drag and drop" stages to rearrange The pre-defined start and end events can cover many use cases involving both issues and merge requests. For supporting more complex workflows, use stages based on group labels. These events are based on -labels being added or removed. In particular, [scoped labels](../project/labels.md#scoped-labels-premium) +labels being added or removed. In particular, [scoped labels](../project/labels.md#scoped-labels) are useful for complex workflows. In this example, we'd like to measure more accurate code review times. The workflow is the following: @@ -313,6 +303,19 @@ To create a value stream:  +### Deleting a value stream + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221205) in GitLab 13.4. + +To delete a custom value stream: + +1. Navigate to your group's **Analytics > Value Stream**. +1. Click the Value stream dropdown and select the value stream you would like to delete. +1. Click the **Delete (name of value stream)**. +1. Click the **Delete** button to confirm. + + + ### Disabling custom value streams Custom value streams are enabled by default. If you have a self-managed instance, an @@ -324,7 +327,8 @@ Feature.disable(:value_stream_analytics_create_multiple_value_streams) ## Days to completion chart -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21631) in GitLab 12.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21631) in GitLab 12.6. +> - [Chart median line removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235455) in GitLab 13.4. This chart visually depicts the total number of days it takes for cycles to be completed. @@ -332,15 +336,6 @@ This chart uses the global page filters for displaying data based on the selecte group, projects, and timeframe. In addition, specific stages can be selected from within the chart itself. -### Chart median line - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36675) in GitLab 12.7. - -The median line on the chart displays data that is offset by the number of days selected. -For example, if 30 days worth of data has been selected (for example, 2019-12-16 to 2020-01-15) the -median line will represent the previous 30 days worth of data (2019-11-16 to 2019-12-16) -as a metric to compare against. - ### Disabling chart This chart is enabled by default. If you have a self-managed instance, an @@ -350,18 +345,9 @@ administrator can open a Rails console and disable it with the following command Feature.disable(:cycle_analytics_scatterplot_enabled) ``` -### Disabling chart median line - -This chart's median line is enabled by default. If you have a self-managed instance, an -administrator can open a Rails console and disable it with the following command: - -```ruby -Feature.disable(:cycle_analytics_scatterplot_median_enabled) -``` - -## Type of work - Tasks by type chart +## Type of work - Tasks by type chart **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32421) in GitLab 12.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32421) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. This chart shows a cumulative count of issues and merge requests per day. @@ -380,7 +366,7 @@ The current permissions on the Project Value Stream Analytics dashboard are: - Internal projects - any authenticated user can access. - Private projects - any member Guest and above can access. -You can [read more about permissions](../../ci/yaml/README.md) in general. +You can [read more about permissions](../../user/permissions.md) in general. For Value Stream Analytics functionality introduced in GitLab 12.3 and later: diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md new file mode 100644 index 00000000000..ae22655e30b --- /dev/null +++ b/doc/user/application_security/api_fuzzing/index.md @@ -0,0 +1,739 @@ +--- +stage: Secure +group: Fuzz Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Web API Fuzz Testing **(ULTIMATE)** + +You can add web API fuzzing to your [GitLab CI/CD](../../../ci/README.md) +pipelines. This helps you discover bugs and potential security issues that other QA processes may miss. +API fuzzing performs fuzz testing of API operation parameters. +Fuzz testing sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API backend. + +We recommend that you use fuzz testing in addition to [GitLab Secure](../index.md)'s +other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +you can run fuzz tests as part your CI/CD workflow. + +## Requirements + +- One of the following web API types: + - REST API + - SOAP + - GraphQL + - Form bodies, JSON, or XML +- An OpenAPI definition, or HTTP Archive (HAR) of requests to test + +## When fuzzing scans run + +When using the `API-Fuzzing.gitlab-ci.yml` template, the `fuzz` job runs last, as shown here. To +ensure API fuzzing scans the latest code, your CI pipeline should deploy changes to a test +environment in one of the jobs preceding the `fuzz` job: + +```yaml +stages: + - build + - test + - deploy + - fuzz +``` + +Note that if your pipeline is configured to deploy to the same web server on each run, running a +pipeline while another is still running could cause a race condition in which one pipeline +overwrites the code from another. The API to scan should be excluded from changes for the duration +of a fuzzing scan. The only changes to the API should be from the fuzzing scanner. Be aware that +any changes made to the API (for example, by users, scheduled tasks, database changes, code +changes, other pipelines, or other scanners) during a scan could cause inaccurate results. + +## Configuration + +There are two ways to perform scans. See the configuration section for the one you wish to use: + +- [OpenAPI v2 specification](#openapi-specification) +- [HTTP Archive (HAR)](#http-archive-har) + +Examples of both configurations can be found here: + +- [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/openapi) +- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har) + +### OpenAPI Specification + +The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an +API description format for REST APIs. This section shows you how to configure API fuzzing by using +an OpenAPI specification to provide information about the target API to test. OpenAPI specifications +are provided as a filesystem resource or URL. + +Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: + +1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + ``` + +1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. + +1. The [configuration file](#configuration-files) has several testing profiles defined with varying + amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this + profile completes quickly, allowing for easier configuration validation. + + Provide the profile by adding the `FUZZAPI_PROFILE` variable to your `.gitlab-ci.yml` file, + substituting `Quick-10` for the profile you choose: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + ``` + +1. Provide the location of the OpenAPI v2 specification. You can provide the specification as a file + or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + ``` + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `FUZZAPI_TARGET_URL`: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +DANGER: **Danger:** +**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run fuzzing against a test server. + +### HTTP Archive (HAR) + +The [HTTP Archive format (HAR)](http://www.softwareishard.com/blog/har-12-spec/) +is an archive file format for logging HTTP transactions. When used with GitLab's API fuzzer, HAR +must contain records of calling the web API to test. The API fuzzer extracts all the requests and +uses them to perform testing. + +You can use various tools to generate HAR files: + +- [Fiddler](https://www.telerik.com/fiddler): Web debugging proxy +- [Insomnia Core](https://insomnia.rest/): API client +- [Chrome](https://www.google.com/chrome): Browser +- [Firefox](https://www.mozilla.org/en-US/firefox/): Browser + +DANGER: **Warning:** +HAR files may contain sensitive information such as authentication tokens, API keys, and session +cookies. We recommend that you review the HAR file contents before adding them to a repository. + +Follow these steps to configure API fuzzing to use a HAR file that provides information about the +target API to test: + +1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + ``` + +1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. + +1. The [configuration file](#configuration-files) has several testing profiles defined with varying + amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this + profile completes quickly, allowing for easier configuration validation. + + Provide the profile by adding the `FUZZAPI_PROFILE` variable to your `.gitlab-ci.yml` file, + substituting `Quick-10` for the profile you choose: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + ``` + +1. Add the `FUZZAPI_HAR` variable and set it to the HAR file's location: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + ``` + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `FUZZAPI_TARGET_URL`: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + FUZZAPI_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +DANGER: **Danger:** +**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run fuzzing against a test server. + +### Authentication + +Authentication is handled by providing the authentication token as a header or cookie. You can +provide a script that performs an authentication flow or calculates the token. + +#### HTTP Basic Authentication + +[HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) +is an authentication method built into the HTTP protocol and used in-conjunction with +[transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). +To use HTTP basic authentication, two variables are added to your `.gitlab-ci.yml` file: + +- `FUZZAPI_HTTP_USERNAME`: The username for authentication. +- `FUZZAPI_HTTP_PASSWORD`: The password for authentication. + +For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui) +(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the +GitLab projects page at **Settings > CI/CD**, in the **Variables** section. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_HTTP_USERNAME: testuser + FUZZAPI_HTTP_PASSWORD: $TEST_API_PASSWORD + +``` + +#### Bearer Tokens + +Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web +Tokens (JWT). Bearer tokens are transmitted using the `Authorization` HTTP header. To use bearer +tokens with API fuzzing, you need one of the following: + +- A token that doesn't expire +- A way to generate a token that lasts the length of testing +- A Python script that API fuzzing can call to generate the token + +##### Token doesn't expire + +If the bearer token doesn't expire, you can provide it using the `FUZZAPI_OVERRIDES_ENV` variable. +The `FUZZAPI_OVERRIDES_ENV` content is a JSON snippet that provides headers and cookies that should +be added to outgoing HTTP requests made by API fuzzing. + +Create a CI/CD variable, for example `TEST_API_BEARERAUTH`, with the value +`{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can +create CI/CD variables from the GitLab projects page at **Settings > CI/CD** in the **Variables** +section. + +Set `FUZZAPI_OVERRIDES_ENV` in your `.gitlab-ci.yml` file: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: $TEST_API_BEARERAUTH +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +##### Token generated at test-runtime + +If the bearer token must be generated, and the resulting token doesn't expire during testing, you +can provide to API fuzzing a file containing the token. This file can be generated by a prior stage +and job, or as part of the API fuzzing job. + +API fuzzing expects to receive a JSON file with the following structure: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +This file can be generated by a prior stage and provided to API fuzzing through the +`FUZZAPI_OVERRIDES_FILE` variable. + +Set `FUZZAPI_OVERRIDES_FILE` in your `.gitlab-ci.yml` file: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +##### Token has short expiration + +If the bearer token must be generated and expires prior to the scan's completion, you can provide a +program or script for the API fuzzer to execute on a provided interval. The provided script runs in +an Alpine Linux container that has Python 3 and Bash installed. If the Python script requires +additional packages, it must detect this and install the packages at runtime. + +The script must create a JSON file containing the bearer token in a specific format: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +You must provide three variables, each set for correct operation: + +- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. +- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. +- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_CMD: renew_token.py + FUZZAPI_OVERRIDES_INTERVAL: 300 +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +### Configuration files + +To get started quickly, GitLab provides you with the configuration file +[`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml). +This file has several testing profiles that perform various amounts of testing. The run time of each +increases as the numbers go up. To use a configuration file, add it to your repository's root as +`.gitlab-api-fuzzing.yml`. + +| Profile | Scan Type | +|:---------|:-----------| +|Quick-10 |Fuzzing 10 times per parameter | +|Medium-20 |Fuzzing 20 times per parameter | +|Medium-50 |Fuzzing 50 times per parameter | +|Long-100 |Fuzzing 100 times per parameter | + +### Available variables + +| Environment variable | Description | +|-----------------------------|--------------------| +| `FUZZAPI_VERSION` |Specify API Fuzzing container version. Defaults to `latest`. | +| `FUZZAPI_TARGET_URL` |Base URL of API testing target. | +|[`FUZZAPI_CONFIG`](#configuration-files)|API Fuzzing configuration file. Defaults to `.gitlab-apifuzzer.yml`. | +|[`FUZZAPI_PROFILE`](#configuration-files)|Configuration profile to use during testing. Defaults to `Quick`. | +| `FUZZAPI_REPORT` |Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. | +|[`FUZZAPI_OPENAPI`](#openapi-specification)|OpenAPI specification file or URL. | +|[`FUZZAPI_HAR`](#http-archive-har)|HTTP Archive (HAR) file. | +|[`FUZZAPI_OVERRIDES_FILE`](#overrides) |Path to a JSON file containing overrides. | +|[`FUZZAPI_OVERRIDES_ENV`](#overrides) |JSON string containing headers to override. | +|[`FUZZAPI_OVERRIDES_CMD`](#overrides) |Overrides command. | +|[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) |How often to run overrides command in seconds. Defaults to `0` (once). | +|[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) |Username for HTTP authentication. | +|[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) |Password for HTTP authentication. | + +<!--|[`FUZZAPI_D_TARGET_IMAGE`](#target-container) |API target docker image | +|[`FUZZAPI_D_TARGET_ENV`](#target-container) |Docker environment options | +|[`FUZZAPI_D_TARGET_VOLUME`](#target-container)|Docker volume options | +|[`FUZZAPI_D_TARGET_PORTS`](#target-container) |Docker port options | +| `FUZZAPI_D_WORKER_IMAGE` |Custom worker docker image | +| `FUZZAPI_D_WORKER_ENV` |Custom worker docker environment options | +| `FUZZAPI_D_WORKER_VOLUME` |Custom worker docker volume options | +| `FUZZAPI_D_WORKER_PORTS` |Custom worker docker port options | +| `FUZZAPI_D_NETWORK` |Name of docker network, defaults to "testing-net"| +| `FUZZAPI_D_PRE_SCRIPT` |Pre script runs after docker login and docker network create, but before starting the scanning image container.| +| `FUZZAPI_D_POST_SCRIPT` |Post script runs after scanning image container is started. This is the place to start your target(s) and kick off scanning when using an advanced configuration.| --> + +### Overrides + +API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests +made. You can use this to inject semver headers, authentication, and so on. The +[authentication section](#authentication) includes examples of using overrides for that purpose. + +Overrides uses a JSON document to define the headers and cookies: + +```json +{ + "headers": { + "header1": "value", + "header2": "value" + }, + "cookies": { + "cookie1": "value", + "cookie2": "value" + } +} +``` + +Example usage for setting a single header: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +Example usage for setting both a header and cookie: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + }, + "cookies": { + "flags": "677" + } +} +``` + +You can provide this JSON document as a file or environment variable. You may also provide a command +to generate the JSON document. The command can run at intervals to support values that expire. + +#### Using a file + +To provide the overrides JSON as a file, the `FUZZAPI_OVERRIDES_FILE` environment variable is set. The path is relative to the job current working directory. + +Example `.gitlab-ci.yml`: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json +``` + +#### Using an environment variable + +To provide the overrides JSON as an environment variable, use the `FUZZAPI_OVERRIDES_ENV` variable. +This allows you to place the JSON as CI/CD variables that can be masked and protected. + +In this example `.gitlab-ci.yml`, the JSON is provided directly: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: '{"headers":{"X-API-Version":"2"}}' +``` + +In this example `.gitlab-ci.yml`, the CI/CD variable `SECRET_OVERRIDES` provides the JSON. This is a +[group or instance level environment variable defined in the UI](../../../ci/variables/README.md#instance-level-cicd-environment-variables): + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: $SECRET_OVERRIDES +``` + +#### Using a command + +If the value must be generated or regenerated on expiration, you can provide a program or script for +the API fuzzer to execute on a specified interval. The provided script runs in an Alpine Linux +container that has Python 3 and Bash installed. If the Python script requires additional packages, +it must detect this and install the packages at runtime. The script creates the overrides JSON file +as defined above. + +You must provide three variables, each set for correct operation: + +- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. +- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. +- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_CMD: renew_token.py + FUZZAPI_OVERRIDES_INTERVAL: 300 +``` + +## Running your first scan + +When configured correctly, a CI/CD pipeline contains a `Fuzz` stage and a `apifuzzer_fuzz` job. The +job only fails when an invalid configuration is provided. During normal operation, the job always +succeeds even if faults are identified during fuzz testing. + +Faults are displayed on the **Tests** pipeline tab with the suite name **API-Fuzzing**. The **Name** +field on the **Tests** page includes the fuzz-tested operation and parameter. The **Trace** field +contains a writeup of the identified fault. This writeup contains information on what the fuzzer +tested and how it detected something wrong. + +To prevent an excessive number of reported faults, the API fuzzing scanner limits the number of +faults it reports to one per parameter. + +### Fault Writeup + +The faults that API fuzzing finds aren't associated with a specific vulnerability type. They require +investigation to determine what type of issue they are and if they should be fixed. See +[handling false positives](#handling-false-positives) for information about configuration changes +you can make to limit the number of false positives reported. + +This table contains a description of fields in an API fuzzing fault writeup. + +| Writeup Item | Description | +|:-------------|:------------| +| Operation | The operation tested. | +| Parameter | The field modified. This can be a path segment, header, query string, or body element. | +| Endpoint | The endpoint being tested. | +| Check | Check module producing the test. Checks can be turned on and off. | +| Assert | Assert module that detected a failure. Assertions can be configured and turned on and off. | +| CWE | Fuzzing faults always have the same CWE. | +| OWASP | Fuzzing faults always have the same OWASP ID. | +| Exploitability | Fuzzing faults always have an `unknown` exploitability. | +| Impact | Fuzzing faults always have an `unknown` risk impact. | +| Description | Verbose description of what the check did. Includes the original parameter value and the modified (mutated) value. | +| Detection | Why a failure was detected and reported. This is related to the Assert that was used. | +| Original Request | The original, unmodified HTTP request. Useful when reviewing the actual request to see what changes were made. | +| Actual Request | The request that produced the failure. This request has been modified in some way by the Check logic. | +| Actual Response | The response to the actual request. | +| Recorded Request | An unmodified request. | +| Recorded Response | The response to the unmodified request. You can compare this with the actual request when triaging this fault. | + +## Handling False Positives + +False positives can be handled in two ways: + +- Turn off the Check producing the false positive. This prevents the check from generating any + faults. Example checks are the JSON Fuzzing Check, and Form Body Fuzzing Check. +- Fuzzing checks have several methods of detecting when a fault is identified, called _Asserts_. + Asserts can also be turned off and configured. For example, the API fuzzer by default uses HTTP + status codes to help identify when something is a real issue. If an API returns a 500 error during + testing, this creates a fault. This isn't always desired, as some frameworks return 500 errors + often. + +### Turn off a Check + +Checks perform testing of a specific type and can be turned on and off for specific configuration +profiles. The provided [configuration files](#configuration-files) define several profiles that you +can use. The profile definition in the configuration file lists all the checks that are active +during a scan. To turn off a specific check, simply remove it from the profile definition in the +configuration file. The profiles are defined in the `Profiles` section of the configuration file. + +Example profile definition: + +```yaml +Profiles: + - Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +To turn off the General Fuzzing Check you can remove these lines: + +```yaml +- Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +This results in the following YAML: + +```yaml +- Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +### Turn off an Assertion for a Check + +Assertions detect faults in tests produced by checks. Many checks support multiple Assertions such +as Log Analysis, Response Analysis, and Status Code. When a fault is found, the Assertion used is +provided. To identify which Assertions are on by default, see the Checks default configuration in +the configuration file. The section is called `Checks`. + +This example shows the FormBody Fuzzing Check: + +```yaml +Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 30 + UnicodeFuzzing: true + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: StatusCodeAssertion +``` + +Here you can see three Assertions are on by default. A common source of false positives is +`StatusCodeAssertion`. To turn it off, modify its configuration in the `Profiles` section. This +example provides only the other two Assertions (`LogAnalysisAssertion`, +`ResponseAnalysisAssertion`). This prevents `FormBodyFuzzingCheck` from using `StatusCodeAssertion`: + +```yaml +Profiles: + - Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlInjectionCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +<!-- +### Target Container + +The API Fuzzing template supports launching a docker container containing an API target using docker-in-docker. + +TODO +--> + +## Glossary + +- Assert: Assertions are detection modules used by checks to trigger a fault. Many assertions have + configurations. A check can use multiple Assertions. For example, Log Analysis, Response Analysis, + and Status Code are common Assertions used together by checks. Checks with multiple Assertions + allow them to be turned on and off. +- Check: Performs a specific type of test, or performed a check for a type of vulnerability. For + example, the JSON Fuzzing Check performs fuzz testing of JSON payloads. The API fuzzer is + comprised of several checks. Checks can be turned on and off in a profile. +- Fault: During fuzzing, a failure identified by an Assert is called a fault. Faults are + investigated to determine if they are a security vulnerability, a non-security issue, or a false + positive. Faults don't have a known vulnerability type until they are investigated. Example + vulnerability types are SQL Injection and Denial of Service. +- Profile: A configuration file has one or more testing profiles, or sub-configurations. You may + have a profile for feature branches and another with extra testing for a main branch. diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 1195d07d7b7..a6ad701360e 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -7,23 +7,41 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Security Configuration **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. +> - SAST configuration was [enabled](https://gitlab.com/groups/gitlab-org/-/epics/3659) in 13.3 and [improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in 13.4. +> - DAST Profiles feature was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40474) in 13.4. -The Security Configuration page displays the configuration state of each security feature in the -current project. The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines/index.md) -to determine each feature's configuration state. If a job with the expected security report artifact -exists in the pipeline, the feature is considered enabled. +The Security Configuration page displays the configuration state of each security control in the +current project. -You can only enable SAST from the Security Configuration page. Documentation links are included for -the other features. For details about configuring SAST, see [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). +To view a project's security configuration, go to the project's home page, +then in the left sidebar go to **Security & Compliance > Configuration**. + +For each security control the page displays: + +- **Status** - Status of the security control: enabled, not enabled, or available. +- **Manage** - A management option or a link to the documentation. + +## Status + +The status of each security control is determined by the project's latest default branch +[CI pipeline](../../../ci/pipelines/index.md). +If a job with the expected security report artifact exists in the pipeline, the feature's status is +_enabled_. + +For SAST, click **View history** to see the `.gitlab-ci.yml` file’s history. NOTE: **Note:** If the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), all security features are configured by default. -## View Security Configuration +## Manage -To view a project's security configuration: +You can configure the following security controls: -1. Go to the project's home page. -1. In the left sidebar, go to **Security & Configuration** > **Configuration**. +- Auto DevOps + - Click **Enable Auto DevOps** to enable it for the current project. For more details, see [Auto DevOps](../../../topics/autodevops/index.md). +- SAST + - Click either **Enable** or **Configure** to use SAST for the current project. For more details, see [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). +- DAST Profiles + - Click **Manage** to manage the available DAST profiles used for on-demand scans. For more details, see [DAST on-demand scans](../dast/index.md#on-demand-scans). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 6b7086ddc71..880e5a3875a 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -26,7 +26,7 @@ To integrate security scanners other than Clair and Klar into GitLab, see You can enable container scanning by doing one of the following: - [Include the CI job](#configuration) in your existing `.gitlab-ci.yml` file. -- Implicitly use [Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning-ultimate) +- Implicitly use [Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning) provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab compares the found vulnerabilities between the source and target branches, and shows the @@ -40,12 +40,12 @@ information directly in the merge request. ## Requirements -To enable Container Scanning in your pipeline, you need the following: +To enable container scanning in your pipeline, you need the following: -- [GitLab Runner](https://docs.gitlab.com/runner/) with the [Docker](https://docs.gitlab.com/runner/executors/docker.html) - or [Kubernetes](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -- Docker `18.09.03` or higher installed on the same computer as the Runner. If you're using the - shared Runners on GitLab.com, then this is already the case. +- [GitLab Runner](https://docs.gitlab.com/runner/) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) + or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +- Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the + shared runners on GitLab.com, then this is already the case. - [Build and push](../../packages/container_registry/index.md#container-registry-examples-with-gitlab-cicd) your Docker image to your project's container registry. The name of the Docker image should use the following [predefined environment variables](../../../ci/variables/predefined_variables.md): @@ -72,7 +72,7 @@ To enable Container Scanning in your pipeline, you need the following: ## Configuration -How you enable Container Scanning depends on your GitLab version: +How you enable container scanning depends on your GitLab version: - GitLab 11.9 and later: [Include](../../../ci/yaml/README.md#includetemplate) the [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) @@ -91,16 +91,16 @@ include: The included template: - Creates a `container_scanning` job in your CI/CD pipeline. -- Pulls the built Docker image from your project's [Container Registry](../../packages/container_registry/index.md) +- Pulls the built Docker image from your project's [container registry](../../packages/container_registry/index.md) (see [requirements](#requirements)) and scans it for possible vulnerabilities. GitLab saves the results as a -[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) +[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning) that you can download and analyze later. When downloading, you always receive the most-recent artifact. -The following is a sample `.gitlab-ci.yml` that builds your Docker image, pushes it to the Container -Registry, and scans the containers: +The following is a sample `.gitlab-ci.yml` that builds your Docker image, pushes it to the container +registry, and scans the containers: ```yaml variables: @@ -127,7 +127,7 @@ include: - template: Container-Scanning.gitlab-ci.yml ``` -### Customizing the Container Scanning settings +### Customizing the container scanning settings There may be cases where you want to customize how GitLab scans your containers. For example, you may want to enable more verbose output from Clair or Klar, access a Docker registry that requires @@ -136,7 +136,7 @@ parameter in your `.gitlab-ci.yml` to set [environment variables](#available-var The environment variables you set in your `.gitlab-ci.yml` overwrite those in `Container-Scanning.gitlab-ci.yml`. -This example [includes](../../../ci/yaml/README.md#include) the Container Scanning template and +This example [includes](../../../ci/yaml/README.md#include) the container scanning template and enables verbose output from Clair by setting the `CLAIR_OUTPUT` environment variable to `High`: ```yaml @@ -153,30 +153,30 @@ variables: #### Available variables -Container Scanning can be [configured](#customizing-the-container-scanning-settings) -using environment variables. - -| Environment Variable | Default | Description | -| -------------------- | ----------- | ------- | -| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | -| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from klar. | -| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the clair server process. | -| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | -| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | -| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | -| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | -| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | -| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | -| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | -| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | -| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | -| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | -| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | -| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | -| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. | -| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | - -### Overriding the Container Scanning template +You can [configure](#customizing-the-container-scanning-settings) container +scanning by using the following environment variables: + +| Environment Variable | Default | Description | +| ------------------------------ | ------------- | ----------- | +| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. | +| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | +| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | +| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | +| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | +| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the clair server process. | +| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. | +| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | +| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | +| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | +| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | +| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | +| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | +| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from klar. | +| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | +| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | +| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | + +### Overriding the container scanning template If you want to override the job definition (for example, to change properties like `variables`), you must declare a `container_scanning` job after the template inclusion, and then @@ -201,35 +201,35 @@ instead. To allowlist specific vulnerabilities, follow these steps: 1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in - [overriding the Container Scanning template](#overriding-the-container-scanning-template). + [overriding the container scanning template](#overriding-the-container-scanning-template). 1. Define the allowlisted vulnerabilities in a YAML file named `vulnerability-allowlist.yml`. This must use the format described in the [allowlist example file](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/raw/master/testdata/vulnerability-allowlist.yml). 1. Add the `vulnerability-allowlist.yml` file to your project's Git repository. -### Running Container Scanning in an offline environment +### Running container scanning in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for the Container Scanning job to +to external resources through the internet, some adjustments are required for the container scanning job to successfully run. For more information, see [Offline environments](../offline_deployments/index.md). -#### Requirements for offline Container Scanning +#### Requirements for offline container Scanning -To use Container Scanning in an offline environment, you need: +To use container scanning in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- To configure a local Docker Container Registry with copies of the Container Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [Container Scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). +- To configure a local Docker container registry with copies of the container scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [container scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. -#### Make GitLab Container Scanning analyzer images available inside your Docker registry +#### Make GitLab container scanning analyzer images available inside your Docker registry -For Container Scanning, import the following default images from `registry.gitlab.com` into your +For container scanning, import the following default images from `registry.gitlab.com` into your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext @@ -249,7 +249,7 @@ For details on saving and transporting Docker images as a file, see Docker's doc [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). -#### Set Container Scanning CI job variables to use local Container Scanner analyzers +#### Set container scanning CI job variables to use local container scanner analyzers 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: @@ -267,10 +267,10 @@ For details on saving and transporting Docker images as a file, see Docker's doc self-signed certificate, then you must set `DOCKER_INSECURE: "true"` in the above `container_scanning` section of your `.gitlab-ci.yml`. -#### Automating Container Scanning vulnerability database updates with a pipeline +#### Automating container scanning vulnerability database updates with a pipeline It can be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to -automatically build a new version of the vulnerabilities database on a preset schedule. Automating +build a new version of the vulnerabilities database on a preset schedule. Automating this with a pipeline means you won't have to do it manually each time. You can use the following `.gitlab-yml.ci` as a template: @@ -293,9 +293,9 @@ build_latest_vulnerabilities: The above template works for a GitLab Docker registry running on a local installation, however, if you're using a non-GitLab Docker registry, you'll need to change the `$CI_REGISTRY` value and the `docker login` credentials to match the details of your local registry. -## Running the standalone Container Scanning Tool +## Running the standalone container scanning tool -It's possible to run the [GitLab Container Scanning Tool](https://gitlab.com/gitlab-org/security-products/analyzers/klar) +It's possible to run the [GitLab container scanning tool](https://gitlab.com/gitlab-org/security-products/analyzers/klar) against a Docker container without needing to run it within the context of a CI job. To scan an image directly, follow these steps: @@ -329,10 +329,10 @@ The results are stored in `gl-container-scanning-report.json`. ## Reports JSON format -The Container Scanning tool emits a JSON report file. For more information, see the +The container scanning tool emits a JSON report file. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/container-scanning-report-format.json). -Here's an example Container Scanning report: +Here's an example container scanning report: ```json-doc { @@ -401,7 +401,7 @@ For more information about the vulnerabilities database update, check the ## Interacting with the vulnerabilities -Once a vulnerability is found, you can [interact with it](../index.md#interacting-with-the-vulnerabilities). +After a vulnerability is found, you can [interact with it](../index.md#interacting-with-the-vulnerabilities). ## Solutions for vulnerabilities (auto-remediation) @@ -413,7 +413,7 @@ the [`DOCKERFILE_PATH`](#available-variables) environment variable. To ensure th has access to this file, it's necessary to set [`GIT_STRATEGY: fetch`](../../../ci/yaml/README.md#git-strategy) in your `.gitlab-ci.yml` file by following the instructions described in this document's -[overriding the Container Scanning template](#overriding-the-container-scanning-template) section. +[overriding the container scanning template](#overriding-the-container-scanning-template) section. Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). @@ -421,8 +421,8 @@ Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vu ### `docker: Error response from daemon: failed to copy xattrs` -When the GitLab Runner uses the Docker executor and NFS is used -(for example, `/var/lib/docker` is on an NFS mount), Container Scanning might fail with +When the runner uses the `docker` executor and NFS is used +(for example, `/var/lib/docker` is on an NFS mount), container scanning might fail with an error like the following: ```plaintext @@ -430,6 +430,6 @@ docker: Error response from daemon: failed to copy xattrs: failed to set xattr " ``` This is a result of a bug in Docker which is now [fixed](https://github.com/containerd/continuity/pull/138 "fs: add WithAllowXAttrErrors CopyOpt"). -To prevent the error, ensure the Docker version that the Runner is using is +To prevent the error, ensure the Docker version that the runner is using is `18.09.03` or higher. For more information, see [issue #10241](https://gitlab.com/gitlab-org/gitlab/-/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 1672e9fbb25..dff71cb9445 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -25,9 +25,11 @@ Docker image with the fuzz engine to run your app. | Language | Fuzzing Engine | Example | |----------|----------------|---------| -| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | [c-cpp-example](https://gitlab.com/gitlab-org/security-products/demos/c-cpp-fuzzing-example) | -| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example) | -| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | | +| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | [c-cpp-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/c-cpp-fuzzing-example) | +| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example) | +| Swift | [libfuzzer](https://github.com/apple/swift/blob/master/docs/libFuzzerIntegration.md) | [swift-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/swift-fuzzing-example) | +| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | [rust-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/rust-fuzzing-example) | +| Java | [JQF](https://github.com/rohanpadhye/JQF) | [java-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/java-fuzzing-example) | ## Configuration diff --git a/doc/user/application_security/cve_id_request.md b/doc/user/application_security/cve_id_request.md new file mode 100644 index 00000000000..94cacf2882f --- /dev/null +++ b/doc/user/application_security/cve_id_request.md @@ -0,0 +1,69 @@ +--- +type: tutorial +stage: Secure +group: Vulnerability Research +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# CVE ID Requests + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41203) in GitLab 13.4, only for public projects on GitLab.com. + +As part of [GitLab's role as a CVE Numbering Authority](https://about.gitlab.com/security/cve) +([CNA](https://cve.mitre.org/cve/cna.html)), you may request +[CVE](https://cve.mitre.org/index.html) identifiers from GitLab to track +vulnerabilities found within your project. + +## Overview + +CVE identifiers track specific vulnerabilities within projects. Having a CVE assigned to a +vulnerability in your project helps your users stay secure and informed. For example, +[dependency scanning tools](../application_security/dependency_scanning/index.md) +can detect when vulnerable versions of your project are used as a dependency. + +## Conditions + +If the following conditions are met, a **Request CVE ID** button appears in your issue sidebar: + +- The project is hosted in GitLab.com. +- The project is public. +- You are a maintainer of the project. +- The issue is confidential. + +## Submitting a CVE ID Request + +Clicking the **Request CVE ID** button in the issue sidebar takes you to the new issue page for +[GitLab's CVE project](https://gitlab.com/gitlab-org/cves). + + + +Creating the confidential issue starts the CVE request process. + + + +You are required to fill in the issue description, which includes: + +- A description of the vulnerability +- The project's vendor and name +- Impacted versions +- Fixed versions +- The vulnerability type (a [CWE](https://cwe.mitre.org/data/index.html) identifier) +- A [CVSS v3 vector](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator) + +## CVE Assignment + +GitLab triages your submitted CVE ID request and communicates with you throughout the CVE validation +and assignment process. + + + +Once a CVE identifier is assigned, you may use and reference it as you see fit. + +Details of the vulnerability submitted in the CVE ID request are published according to your +schedule. It's common to request a CVE for an unpatched vulnerability, reference the assigned CVE +identifier in release notes, and later publish the vulnerability's details after the fix is +released. + +Separate communications notify you when different stages of the publication process are complete. + + diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index b2020d48d38..73a8e727389 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -26,7 +26,7 @@ If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your runn for known vulnerabilities using Dynamic Application Security Testing (DAST). You can take advantage of DAST by either [including the CI job](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate), +[Auto DAST](../../../topics/autodevops/stages.md#auto-dast), provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the DAST report, compares the found vulnerabilities between the source and target @@ -106,7 +106,7 @@ The included template creates a `dast` job in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast) that you can later download and analyze. Due to implementation limitations we always take the latest DAST artifact available. Behind the scenes, the [GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast) @@ -177,13 +177,13 @@ include: variables: DAST_WEBSITE: https://example.com DAST_AUTH_URL: https://example.com/sign-in - DAST_USERNAME_FIELD: session[user] # the name of username field at the sign-in HTML form - DAST_PASSWORD_FIELD: session[password] # the name of password field at the sign-in HTML form - DAST_AUTH_EXCLUDE_URLS: http://example.com/sign-out,http://example.com/sign-out-2 # optional, URLs to skip during the authenticated scan; comma-separated, no spaces in between + DAST_USERNAME_FIELD: session[user] # the name of username field at the sign-in HTML form + DAST_PASSWORD_FIELD: session[password] # the name of password field at the sign-in HTML form + DAST_AUTH_EXCLUDE_URLS: http://example.com/sign-out,http://example.com/sign-out-2 # optional, URLs to skip during the authenticated scan; comma-separated, no spaces in between ``` The results are saved as a -[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast) that you can later download and analyze. Due to implementation limitations, we always take the latest DAST artifact available. @@ -206,6 +206,9 @@ variables: DAST_FULL_SCAN_ENABLED: "true" ``` +NOTE: **Note:** +If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). + #### Domain validation The DAST job can be run anywhere, which means you can accidentally hit live web servers @@ -364,7 +367,7 @@ dast: DAST_API_SPECIFICATION: api-specification.yml ``` -#### Full scan +#### Full API scan API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` environment variable. Domain validation is not supported for full API scans. @@ -395,7 +398,11 @@ variables: DAST_API_HOST_OVERRIDE: api-test.host.com ``` -Note that `DAST_API_HOST_OVERRIDE` is only applied to specifications imported by URL. +NOTE: **Note:** +Using a host override is ONLY supported when importing the API +specification from a URL. It does not work and will be ignored when importing +the specification from a file. This is due to a limitation in the ZAP OpenAPI +extension. #### Authentication using headers @@ -412,6 +419,31 @@ variables: DAST_REQUEST_HEADERS: "Authorization: Bearer my.token" ``` +### URL scan + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.4. + +A URL scan allows you to specify which parts of a website are scanned by DAST. + +#### Define the URLs to scan + +To specify the paths to be scanned, add a comma-separated list of the paths to the `DAST_PATHS` environment variable. Note that you can only scan paths of a single host. + +```yaml +include: + - template: DAST.gitlab-ci.yml + +variables: + DAST_PATHS=/page1.html,/category1/page1.html,/page3.html +``` + +NOTE: **Note:** +`DAST_AUTH_EXCLUDE_URLS` are ignored when `DAST_PATHS` is set. + +#### Full Scan + +To perform a [full scan](#full-scan) on the listed paths, use the `DAST_FULL_SCAN_ENABLED` environment variable. + ### Customizing the DAST settings CAUTION: **Deprecation:** @@ -451,12 +483,12 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_USERNAME_FIELD` | string | The name of username field at the sign-in HTML form. | | `DAST_PASSWORD_FIELD` | string | The name of password field at the sign-in HTML form. | | `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | -| `DAST_AUTH_EXCLUDE_URLS` | URLs | The URLs to skip during the authenticated scan; comma-separated, no spaces in between. Not supported for API scans. | +| `DAST_AUTH_EXCLUDE_URLS` | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | | `DAST_FULL_SCAN_ENABLED` | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | | `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | Set to `true` to require [domain validation](#domain-validation) when running DAST full scans. Not supported for API scans. Default: `false` | | `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | -| `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Example: `example.com:8080` | -| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were supressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` | +| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | | `DAST_REQUEST_HEADERS` | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | | `DAST_DEBUG` | boolean | Enable debug message output. Default: `false` | | `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. | @@ -465,6 +497,7 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. | | `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false` | | `DAST_USE_AJAX_SPIDER` | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false` | +| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html` | | `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. | | `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` | @@ -484,8 +517,8 @@ dast: ``` You must then overwrite the `script` command to pass in the appropriate -argument. For example, passive scanning can be delayed using option `-D`. The following -configuration delays passive scanning by five minutes: +argument. For example, vulnerability definitions in alpha can be included with +`-a`. The following configuration includes those definitions: ```yaml include: @@ -494,7 +527,7 @@ include: dast: script: - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -D 300 -t $DAST_WEBSITE + - /analyze -a -t $DAST_WEBSITE ``` ### Custom ZAProxy configuration @@ -559,8 +592,8 @@ To use DAST in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. @@ -600,110 +633,171 @@ security reports without requiring internet access. Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. -## On-Demand Scans +## Site profile -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. -> - It's deployed behind a feature flag, enabled by default. -> - It's enabled on GitLab.com. -> - It's able to be enabled or disabled per-project. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans). +A site profile describes the attributes of a web site to scan on demand with DAST. A site profile is +required for an on-demand DAST scan. -You can run a passive DAST scan against a target website, outside the DevOps lifecycle. These scans -are always associated with the default branch of your project and the results are available in the -project dashboard. +A site profile contains the following: -### Site profile +- **Profile name**: A name you assign to the site to be scanned. +- **Target URL**: The URL that DAST runs against. -An on-demand scan requires a site profile, which includes a profile name and target URL. The profile -name allows you to describe the site to be scanned. The target URL specifies the URL against which -the DAST scan is run. +### Create a site profile -### Run an on-demand scan +To create a site profile: -NOTE: **Note:** -You must have permission to run an on-demand DAST scan against a protected branch. -The default branch is automatically protected. For more details, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **New Profile > Site Profile**. +1. Type in a unique **Profile name** and **Target URL** then click **Save profile**. -Running an on-demand scan requires an existing site profile. If a site profile for the target URL -doesn't exist, first [create a site profile](#create-a-site-profile). An on-demand DAST scan has -a fixed timeout of 60 seconds. +### Edit a site profile -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Select a site profile from the profiles dropdown. -- Click **Run scan**. +To edit an existing site profile: -#### Create a site profile +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **Edit** in the row of the profile to edit. +1. Edit the **Profile name** and **Target URL**, then click **Save profile**. -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Click **New Site Profile**. -- Type in a unique **Profile name** and **Target URL** then click **Save profile**. +### Delete a site profile -#### Delete a site profile +To delete an existing site profile: -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Click **Delete** in the matching site profile's row. +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **{remove}** in the row of the profile to delete. -### Enable or disable On-demand Scans and site profiles +## Scanner profile -On-demand Scans with site profiles is enabled by default. You can disable On-demand Scans -instance-wide, or disable it for specific projects if you prefer. DAST site profiles are not -available if the On-demand Scans feature is disabled. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4. +> - [Deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - Enabled on GitLab.com. +> - Can be enabled or disabled per-project. +> - Recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can [disable this feature](#enable-or-disable-dast-scanner-profiles). -Use of On-demand Scans with site profiles requires **both** the following feature flags enabled: +A scanner profile defines the scanner settings used to run an on-demand scan: -- security_on_demand_scans_feature_flag -- security_on_demand_scans_site_profiles_feature_flag +- **Profile name:** A name you give the scanner profile. For example, "Spider_15". +- **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site. +- **Target timeout:** The maximum number of seconds DAST waits for the site to be available before + starting the scan. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable or enable the feature flags. +### Create a scanner profile + +To create a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **New Profile > Scanner Profile**. +1. Enter a unique **Profile name**, the desired **Spider timeout**, and the **Target timeout**. +1. Click **Save profile**. + +### Edit a scanner profile + +To edit a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **Edit** in the scanner profile's row. -#### Enable or disable On-demand Scans +### Delete a scanner profile -To disable On-demand Scans: +To delete a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **{remove}** in the scanner profile's row. + +### Enable or disable DAST scanner profiles + +The scanner profile feature is ready for production use. It's deployed behind a feature flag that +is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to disable it. + +To disable it: ```ruby # Instance-wide -Feature.disable(:security_on_demand_scans_feature_flag) +Feature.disable(:security_on_demand_scans_scanner_profiles) # or by project -Feature.disable(:security_on_demand_scans_feature_flag, Project.find(<project id>)) +Feature.disable(:security_on_demand_scans_scanner_profiles, Project.find(<project id>)) ``` -To enable On-demand Scans: +To enable it: ```ruby # Instance-wide -Feature.enable(:security_on_demand_scans_feature_flag) +Feature.enable(:security_on_demand_scans_scanner_profiles) # or by project -Feature.enable(:security_on_demand_scans_feature_flag, Project.find(<project id>)) +Feature.enable(:security_on_demand_scans_scanner_profiles, Project.find(<project ID>)) ``` -#### Enable or disable site profiles +## On-demand scans + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. +> - It's deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It's able to be enabled or disabled per-project. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans). + +An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger +the scan. You must start it manually. + +An on-demand DAST scan: + +- Uses settings in the site profile and scanner profile you select when you run the scan, + instead of those in the `.gitlab-ci.yml` file. +- Is associated with your project's default branch. + +### Run an on-demand DAST scan + +NOTE: **Note:** +You must have permission to run an on-demand DAST scan against a protected branch. +The default branch is automatically protected. For more details, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). + +To run an on-demand DAST scan, you need: + +- A [scanner profile](#create-a-scanner-profile). +- A [site profile](#create-a-site-profile). + +1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left sidebar. +1. Click **Create new DAST scan**. +1. In **Scanner settings**, select a scanner profile from the dropdown. +1. In **Site profiles**, select a site profile from the dropdown. +1. Click **Run scan**. + +The on-demand DAST scan runs and the project's dashboard shows the results. + +### Enable or disable On-demand Scans + +The On-demand DAST Scans feature is enabled by default. You can disable on-demand scans +instance-wide, or disable it for specific projects if you prefer. + +To run on-demand DAST scans, an administrator must enable the +`security_on_demand_scans_feature_flag` feature flag. -The Site Profiles feature is enabled instance-wide by default. You can disable it instance-wide, or disable it -for specific projects if you prefer. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable or enable the feature flag. +can disable or enable the feature flags. -To disable Site Profiles: +To disable On-demand DAST Scans: ```ruby # Instance-wide -Feature.disable(:security_on_demand_scans_site_profiles_feature_flag) +Feature.disable(:security_on_demand_scans_feature_flag) # or by project -Feature.disable(:security_on_demand_scans_site_profiles_feature_flag, Project.find(<project id>)) +Feature.disable(:security_on_demand_scans_feature_flag, Project.find(<project id>)) ``` -To enable Site Profiles: +To enable On-demand DAST Scans: ```ruby # Instance-wide -Feature.enable(:security_on_demand_scans_site_profiles_feature_flag) +Feature.enable(:security_on_demand_scans_feature_flag) # or by project -Feature.enable(:security_on_demand_scans_site_profiles_feature_flag, Project.find(<project id>)) +Feature.enable(:security_on_demand_scans_feature_flag, Project.find(<project ID>)) ``` ## Reports @@ -825,6 +919,10 @@ variables: Here, DAST is being allocated 3072 MB. Change the number after `-Xmx` to the required memory amount. +### DAST job exceeding the job timeout + +If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index d41f9441464..40189235e64 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -90,32 +90,7 @@ That's needed when one totally relies on [custom analyzers](#custom-analyzers). ## Custom analyzers -### Custom analyzers with Docker-in-Docker - -When Docker-in-Docker for Dependency Scanning is enabled, -you can provide your own analyzers as a comma-separated list of Docker images. -Here's how to add `analyzers/nuget` and `analyzers/perl` to the default images. -In `.gitlab-ci.yml` define: - -```yaml -include: - template: Dependency-Scanning.gitlab-ci.yml - -variables: - DS_ANALYZER_IMAGES: "my-docker-registry/analyzers/nuget,amy-docker-registry/analyzers/perl" -``` - -The values must be the full path to the container registry images, -like what you would feed to the `docker pull` command. - -NOTE: **Note:** -This configuration doesn't benefit from the integrated detection step. Dependency -Scanning has to fetch and spawn each Docker image to establish whether the -custom analyzer can scan the source code. - -### Custom analyzers without Docker-in-Docker - -When Docker-in-Docker for Dependency Scanning is disabled, you can provide your own analyzers by +You can provide your own analyzers by defining CI jobs in your CI configuration. For consistency, you should suffix your custom Dependency Scanning jobs with `-dependency_scanning`. Here's how to add a scanning job that's based on the Docker image `my-docker-registry/analyzers/nuget` and generates a Dependency Scanning report diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 6b14f93735b..5cce336d04c 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -20,7 +20,7 @@ vulnerabilities using Dependency Scanning. All dependencies are scanned, including the transitive dependencies (also known as nested dependencies). You can take advantage of Dependency Scanning by either [including the Dependency Scanning template](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -the [Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) +the [Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning) provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the Dependency Scanning report, compares the found vulnerabilities @@ -43,14 +43,12 @@ The results are sorted by the severity of the vulnerability: To run Dependency Scanning jobs, by default, you need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared Runners on GitLab.com, this is enabled by default. +If you're using the shared runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** -If you use your own Runners, make sure your installed version of Docker +If you use your own runners, make sure your installed version of Docker is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. -Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for Dependency Scanning](#enabling-docker-in-docker). - ## Supported languages and package managers GitLab relies on [`rules`](../../../ci/yaml/README.md#rules) to start relevant analyzers depending on the languages detected in the repository. @@ -61,6 +59,7 @@ The following languages and dependency managers are supported: | Language (package managers) | Supported files | Scan tool(s) | |----------------------------- | --------------- | ------------ | | C# .NET ([NuGet](https://www.nuget.org/) 4.9+) | [`packages.lock.json`](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#enabling-lock-file) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| C/C++ ([Conan](https://conan.io/)) | [`conan.lock`](https://docs.conan.io/en/latest/versioning/lockfiles.html) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Java ([Gradle](https://gradle.org/), [Maven](https://maven.apache.org/)) | `build.gradle`, `build.gradle.kts`, `pom.xml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | | Go ([Golang](https://golang.org/)) | `go.sum` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | @@ -99,7 +98,7 @@ include: The included template creates Dependency Scanning jobs in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[Dependency Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning-ultimate) +[Dependency Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning) that you can later download and analyze. Due to implementation limitations, we always take the latest Dependency Scanning artifact available. @@ -153,24 +152,10 @@ The following variables allow configuration of global dependency scanning settin | --------------------------------------- |------------ | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | -| `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | -| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. | +| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"` | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info` | -#### Configuring Docker-in-Docker orchestrator - -The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). - -| Environment variable | Default | Description | -| --------------------------------------- | ----------- | ----------- | -| `DS_ANALYZER_IMAGES` | | Comma-separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | -| `DS_ANALYZER_IMAGE_TAG` | | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | -| `DS_PULL_ANALYZER_IMAGES` | | Pull the images from the Docker registry (set to `0` to disable). | -| `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, or `h`. For example, `300ms`, `1.5h`, or `2h45m`. | -| `DS_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling an analyzer's image. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, or `h`. For example, `300ms`, `1.5h`, or `2h45m`. | -| `DS_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, or `h`. For example, `300ms`, `1.5h`, or `2h45m`. | - #### Configuring specific analyzers used by Dependency Scanning The following variables are used for configuring specific analyzers (used for a specific language/framework). @@ -205,27 +190,6 @@ you can use the `MAVEN_CLI_OPTS` environment variable. Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Enabling Docker-in-Docker - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12487) in GitLab Ultimate 12.5. - -If needed, you can enable Docker-in-Docker to restore the Dependency Scanning behavior that existed -prior to GitLab 13.0. Follow these steps to do so: - -1. Configure GitLab Runner with Docker-in-Docker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). -1. Set the `DS_DISABLE_DIND` variable to `false`: - - ```yaml - include: - - template: Dependency-Scanning.gitlab-ci.yml - - variables: - DS_DISABLE_DIND: "false" - ``` - -This creates a single `dependency_scanning` job in your CI/CD pipeline instead of multiple -`<analyzer-name>-dependency_scanning` jobs. - ## Interacting with the vulnerabilities Once a vulnerability is found, you can interact with it. Read more on how to @@ -388,7 +352,6 @@ jobs to run successfully. For more information, see [Offline environments](../of Here are the requirements for using Dependency Scanning in an offline environment: -- Keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - Docker Container Registry with locally available copies of Dependency Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/). @@ -399,8 +362,8 @@ Here are the requirements for using Dependency Scanning in an offline environmen NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. @@ -443,7 +406,6 @@ include: variables: SECURE_ANALYZERS_PREFIX: "docker-registry.example.com/analyzers" GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" - GIT_SSL_NO_VERIFY: "true" ``` See explanations of the variables above in the [configuration section](#configuration). diff --git a/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png b/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png Binary files differdeleted file mode 100644 index cb8911b14b1..00000000000 --- a/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/img/adding_a_dismissal_reason_v13_4.png b/doc/user/application_security/img/adding_a_dismissal_reason_v13_4.png Binary files differnew file mode 100644 index 00000000000..8e7bcf09428 --- /dev/null +++ b/doc/user/application_security/img/adding_a_dismissal_reason_v13_4.png diff --git a/doc/user/application_security/img/create_issue_from_vulnerability_v13_3.png b/doc/user/application_security/img/create_issue_from_vulnerability_v13_3.png Binary files differnew file mode 100644 index 00000000000..b792fbc9af1 --- /dev/null +++ b/doc/user/application_security/img/create_issue_from_vulnerability_v13_3.png diff --git a/doc/user/application_security/img/create_issue_with_list_hover.png b/doc/user/application_security/img/create_issue_with_list_hover.png Binary files differdeleted file mode 100644 index 4c38862e68f..00000000000 --- a/doc/user/application_security/img/create_issue_with_list_hover.png +++ /dev/null diff --git a/doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png b/doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png Binary files differnew file mode 100644 index 00000000000..a914c2996f7 --- /dev/null +++ b/doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png diff --git a/doc/user/application_security/img/cve_id_request_button.png b/doc/user/application_security/img/cve_id_request_button.png Binary files differnew file mode 100644 index 00000000000..15707ba9eb2 --- /dev/null +++ b/doc/user/application_security/img/cve_id_request_button.png diff --git a/doc/user/application_security/img/cve_request_communication.png b/doc/user/application_security/img/cve_request_communication.png Binary files differnew file mode 100644 index 00000000000..0766b371c11 --- /dev/null +++ b/doc/user/application_security/img/cve_request_communication.png diff --git a/doc/user/application_security/img/cve_request_communication_publication.png b/doc/user/application_security/img/cve_request_communication_publication.png Binary files differnew file mode 100644 index 00000000000..9e34c217e13 --- /dev/null +++ b/doc/user/application_security/img/cve_request_communication_publication.png diff --git a/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png b/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png Binary files differdeleted file mode 100644 index 19d47712f9e..00000000000 --- a/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/img/interacting_with_vulnerability_v13_3.png b/doc/user/application_security/img/interacting_with_vulnerability_v13_3.png Binary files differnew file mode 100644 index 00000000000..db698995469 --- /dev/null +++ b/doc/user/application_security/img/interacting_with_vulnerability_v13_3.png diff --git a/doc/user/application_security/img/new_cve_request_issue.png b/doc/user/application_security/img/new_cve_request_issue.png Binary files differnew file mode 100644 index 00000000000..a342c73992e --- /dev/null +++ b/doc/user/application_security/img/new_cve_request_issue.png diff --git a/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png b/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png Binary files differnew file mode 100644 index 00000000000..f497b0fbc4e --- /dev/null +++ b/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png diff --git a/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png b/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png Binary files differnew file mode 100644 index 00000000000..fc847b578f5 --- /dev/null +++ b/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png diff --git a/doc/user/application_security/img/vulnerability-check_v13_0.png b/doc/user/application_security/img/vulnerability-check_v13_0.png Binary files differdeleted file mode 100644 index 9f0bd0f759b..00000000000 --- a/doc/user/application_security/img/vulnerability-check_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/img/vulnerability-check_v13_4.png b/doc/user/application_security/img/vulnerability-check_v13_4.png Binary files differnew file mode 100644 index 00000000000..e0b53059b45 --- /dev/null +++ b/doc/user/application_security/img/vulnerability-check_v13_4.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index c003b512808..d509176f2b2 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -67,6 +67,7 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | +| [API fuzzing](api_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities in web APIs with fuzzing. | | [Secret Detection](secret_detection/index.md) **(ULTIMATE)** | Analyze Git history for leaked secrets. | | [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) | Analyze source code for known vulnerabilities. | @@ -76,12 +77,12 @@ GitLab uses the following tools to scan and report known vulnerabilities found i When [Auto DevOps](../../topics/autodevops/) is enabled, all GitLab Security scanning tools will be configured using default settings. -- [Auto SAST](../../topics/autodevops/stages.md#auto-sast-ultimate) -- [Auto Secret Detection](../../topics/autodevops/stages.md#auto-secret-detection-ultimate) -- [Auto DAST](../../topics/autodevops/stages.md#auto-dast-ultimate) -- [Auto Dependency Scanning](../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) -- [Auto License Compliance](../../topics/autodevops/stages.md#auto-license-compliance-ultimate) -- [Auto Container Scanning](../../topics/autodevops/stages.md#auto-container-scanning-ultimate) +- [Auto SAST](../../topics/autodevops/stages.md#auto-sast) +- [Auto Secret Detection](../../topics/autodevops/stages.md#auto-secret-detection) +- [Auto DAST](../../topics/autodevops/stages.md#auto-dast) +- [Auto Dependency Scanning](../../topics/autodevops/stages.md#auto-dependency-scanning) +- [Auto License Compliance](../../topics/autodevops/stages.md#auto-license-compliance) +- [Auto Container Scanning](../../topics/autodevops/stages.md#auto-container-scanning) While you cannot directly customize Auto DevOps, you can [include the Auto DevOps template in your project's `.gitlab-ci.yml` file](../../topics/autodevops/customize.md#customizing-gitlab-ciyml). @@ -121,7 +122,7 @@ information with several options: - [Solution](#solutions-for-vulnerabilities-auto-remediation): For some vulnerabilities, a solution is provided for how to fix the vulnerability. - + ### View details of a DAST vulnerability @@ -165,7 +166,8 @@ reports. You can specify the list of all headers to be masked. For details, see ### Dismissing a vulnerability -To dismiss a vulnerability, you must set its status to Dismissed. Follow these steps to do so: +To dismiss a vulnerability, you must set its status to Dismissed. This dismisses the vulnerability +for the entire project. Follow these steps to do so: 1. Select the vulnerability in the Security Dashboard. 1. Select **Dismissed** from the **Status** selector menu at the top-right. @@ -181,7 +183,7 @@ vulnerability's status to Dismissed, a text box appears for you to add a comment dismissal. Once added, you can edit or delete it. This allows you to add and update context for a vulnerability as you learn more over time. - + #### Dismissing multiple vulnerabilities @@ -196,22 +198,6 @@ Pressing the "Dismiss Selected" button will dismiss all the selected vulnerabili  -### Creating an issue for a vulnerability - -You can create an issue for a vulnerability by selecting the **Create issue** -button from within the vulnerability modal, or by using the action buttons to the right of -a vulnerability row in the group security dashboard. - -This creates a [confidential issue](../project/issues/confidential_issues.md) in the project the -vulnerability came from, and pre-populates it with some useful information taken from the vulnerability -report. Once the issue is created, you are redirected to it so you can edit, assign, or comment on -it. - -Upon returning to the group security dashboard, the vulnerability now has an associated issue next -to the name. - - - ### Solutions for vulnerabilities (auto-remediation) > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5656) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.7. @@ -246,10 +232,29 @@ vulnerability. Any vulnerability that has a [solution](#solutions-for-vulnerabilities-auto-remediation) can have a merge request created to automatically solve the issue. -If this action is available, the vulnerability modal contains a **Create merge request** button. +If this action is available, the vulnerability page or modal contains a **Create merge request** button. Click this button to create a merge request to apply the solution onto the source branch. - + + +### Creating an issue for a vulnerability + +You can create an issue for a vulnerability by visiting the vulnerability's page and clicking +**Create issue**, which you can find in the **Related issues** section. + + + +This creates a [confidential issue](../project/issues/confidential_issues.md) in the project the +vulnerability came from, and pre-populates it with some useful information taken from the vulnerability +report. Once the issue is created, you are redirected to it so you can edit, assign, or comment on +it. CVE identifiers can be requested from GitLab by clicking the +[_CVE ID Request_ button](cve_id_request.md) that is enabled for maintainers of +public projects on GitLab.com + +Upon returning to the group security dashboard, the vulnerability now has an associated issue next +to the name. + + ### Managing related issues for a vulnerability @@ -307,15 +312,29 @@ rating. ### Enabling Security Approvals within a project -To enable Security Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#adding--editing-a-default-approval-rule) -must be created with the case-sensitive name `Vulnerability-Check`. This approval group must be set -with the number of approvals required greater than zero. You must have Maintainer or Owner [permissions](../permissions.md#project-members-permissions) to manage approval rules. +To enable the `Vulnerability-Check` or `License-Check` Security Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#adding--editing-a-default-approval-rule) +must be created. A [security scanner job](#security-scanning-tools) must be enabled for +`Vulnerability-Check`, and a [license scanning](../compliance/license_compliance/index.md#configuration) +job must be enabled for `License-Check`. When the proper jobs aren't configured, the following +appears: + + + +If at least one security scanner is enabled, you will be able to enable the `Vulnerability-Check` approval rule. If a license scanning job is enabled, you will be able to enable the `License-Check` rule. + + + +For this approval group, you must set the number of approvals required to greater than zero. You +must have Maintainer or Owner [permissions](../permissions.md#project-members-permissions) +to manage approval rules. + +Follow these steps to enable `Vulnerability-Check`: 1. Navigate to your project's **Settings > General** and expand **Merge request approvals**. -1. Click **Add approval rule**, or **Edit**. - - Add or change the **Rule name** to `Vulnerability-Check` (case sensitive). +1. Click **Enable**, or **Edit**. +1. Add or change the **Rule name** to `Vulnerability-Check` (case sensitive). - + Once this group is added to your project, the approval rule is enabled for all merge requests. @@ -332,32 +351,14 @@ An approval is optional when the security report: - Contains no new vulnerabilities when compared to the target branch. - Contains only new vulnerabilities of `low` or `medium` severity. -## Enabling License Approvals within a project +### Enabling License Approvals within a project > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -`License-Check` is an approval rule you can enable to allow an individual or group to approve a -merge request that contains a `denied` license. - -You can enable `License-Check` one of two ways: - -- Create a [project approval rule](../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) - with the case-sensitive name `License-Check`. -- Create an approval group in the [project policies section for License Compliance](../compliance/license_compliance/index.md#policies). - You must set this approval group's number of approvals required to greater than zero. Once you - enable this group in your project, the approval rule is enabled for all merge requests. - -Any code changes cause the approvals required to reset. - -An approval is required when a license report: - -- Contains a dependency that includes a software license that is `denied`. -- Is not generated during pipeline execution. - -An approval is optional when a license report: - -- Contains no software license violations. -- Contains only new licenses that are `allowed` or unknown. +`License-Check` is a [security approval rule](#enabling-security-approvals-within-a-project) +you can enable to allow an individual or group to approve a merge request that contains a `denied` +license. For instructions on enabling this rule, see +[Enabling license approvals within a project](../compliance/license_compliance/index.md#enabling-license-approvals-within-a-project). ## Working in an offline environment diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index a5cf93f9448..3a7c0148388 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -64,10 +64,10 @@ Once a vulnerability is found, you can interact with it. Read more on how to Please note that in some cases the reported vulnerabilities provide metadata that can contain external links exposed in the UI. These links might not be accessible within an offline environment. -### Suggested Solutions for vulnerabilities +### Automatic remediation for vulnerabilities -The [suggested solutions](../index.md#solutions-for-vulnerabilities-auto-remediation) feature -(auto-remediation) is available for Dependency Scanning and Container Scanning, but may not work +The [automatic remediation for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation) feature +(auto-remediation) is available for offline Dependency Scanning and Container Scanning, but may not work depending on your instance's configuration. We can only suggest solutions, which are generally more current versions that have been patched, when we are able to access up-to-date registry services hosting the latest versions of that dependency or image. @@ -96,7 +96,7 @@ above. You can find more information at each of the pages below: To use many GitLab features, including [security scans](../index.md#working-in-an-offline-environment) -and [Auto DevOps](../../../topics/autodevops/index.md), the GitLab Runner must be able to fetch the +and [Auto DevOps](../../../topics/autodevops/index.md), the runner must be able to fetch the relevant Docker images. The process for making these images available without direct access to the public internet @@ -124,7 +124,7 @@ The pipeline downloads the Docker images needed for the Security Scanners and sa [job artifacts](../../../ci/pipelines/job_artifacts.md) or pushes them to the [Container Registry](../../packages/container_registry/index.md) of the project where the pipeline is executed. These archives can be transferred to another location and [loaded](https://docs.docker.com/engine/reference/commandline/load/) in a Docker daemon. -This method requires a GitLab Runner with access to both `gitlab.com` (including +This method requires a runner with access to both `gitlab.com` (including `registry.gitlab.com`) and the local offline instance. This runner must run in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) to be able to use the `docker` command inside the jobs. This runner can be installed in a DMZ or on diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 214044ad783..727f077aa09 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -28,7 +28,6 @@ SAST supports the following official analyzers: - [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) (NodeJsScan) - [`phpcs-security-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP CS security-audit) - [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) (PMD (Apex only)) -- [`secrets`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) (Secrets (Gitleaks & TruffleHog secret detectors)) - [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) (Security Code Scan (.NET)) - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) @@ -97,32 +96,7 @@ That's needed when one totally relies on [custom analyzers](#custom-analyzers). ## Custom Analyzers -### Custom analyzers with Docker-in-Docker - -When Docker-in-Docker for SAST is enabled, -you can provide your own analyzers as a comma-separated list of Docker images. -Here's how to add `analyzers/csharp` and `analyzers/perl` to the default images: -In `.gitlab-ci.yml` define: - -```yaml -include: - - template: SAST.gitlab-ci.yml - -variables: - SAST_ANALYZER_IMAGES: "my-docker-registry/analyzers/csharp,amy-docker-registry/analyzers/perl" -``` - -The values must be the full path to the container registry images, -like what you would feed to the `docker pull` command. - -NOTE: **Note:** -This configuration doesn't benefit from the integrated detection step. -SAST has to fetch and spawn each Docker image to establish whether the -custom analyzer can scan the source code. - -### Custom analyzers without Docker-in-Docker - -When Docker-in-Docker for SAST is disabled, you can provide your own analyzers by +You can provide your own analyzers by defining CI jobs in your CI configuration. For consistency, you should suffix your custom SAST jobs with `-sast`. Here's how to add a scanning job that's based on the Docker image `my-docker-registry/analyzers/csharp` and generates a SAST report @@ -146,7 +120,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) | Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | | --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | -| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | | Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | | Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | | File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -159,7 +133,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) | Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | | Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | x | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | | Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index fd331020719..a4fc3c9e638 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -43,29 +43,28 @@ A pipeline consists of multiple jobs, including SAST and DAST scanning. If any j ## Requirements -To run SAST jobs, by default, you need a GitLab Runner with the +To run SAST jobs, by default, you need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared Runners on GitLab.com, this is enabled by default. - -Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker-ultimate). +If you're using the shared runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** Our SAST jobs require a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** -If you use your own Runners, make sure the Docker version installed +If you use your own runners, make sure the Docker version installed is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. ## Supported languages and frameworks -The following table shows which languages, package managers and frameworks are supported and which tools are used. +GitLab SAST supports a variety of languages, package managers, and frameworks. Our SAST security scanners also feature automatic language detection which works even for mixed-language projects. If any supported language is detected in project source code we will automatically run the appropriate SAST analyzers. + +You can also [view our language roadmap](https://about.gitlab.com/direction/secure/static-analysis/sast/#language-support) and [request other language support by opening an issue](https://gitlab.com/groups/gitlab-org/-/epics/297). | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |--------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| | .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | | .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11., [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | | Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | | C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | | Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | @@ -94,9 +93,6 @@ All open source (OSS) analyzers have been moved to the GitLab Core tier. Progres tracked in the corresponding [epic](https://gitlab.com/groups/gitlab-org/-/epics/2098). -Please note that support for [Docker-in-Docker](#enabling-docker-in-docker-ultimate) -will not be extended to the GitLab Core tier. - #### Summary of features per tier Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), @@ -108,8 +104,9 @@ as shown in the following table: | [Customize SAST Settings](#customizing-the-sast-settings) | **{check-circle}** | **{check-circle}** | | View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | | [Presentation of JSON Report in Merge Request](#overview) | **{dotted-circle}** | **{check-circle}** | -| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities-ultimate) | **{dotted-circle}** | **{check-circle}** | -| [Access to Security Dashboard](#security-dashboard-ultimate) | **{dotted-circle}** | **{check-circle}** | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) | **{dotted-circle}** | **{check-circle}** | +| [Access to Security Dashboard](#security-dashboard) | **{dotted-circle}** | **{check-circle}** | +| [Configure SAST in the UI](#configure-sast-in-the-ui) | **{dotted-circle}** | **{check-circle}** | ## Contribute your scanner @@ -119,7 +116,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) To configure SAST for a project you can: -- Use [Auto SAST](../../../topics/autodevops/stages.md#auto-sast-ultimate) provided by +- Use [Auto SAST](../../../topics/autodevops/stages.md#auto-sast) provided by [Auto DevOps](../../../topics/autodevops/index.md). - [Configure SAST manually](#configure-sast-manually). - [Configure SAST using the UI](#configure-sast-in-the-ui) (introduced in GitLab 13.3). @@ -135,30 +132,31 @@ Add the following to your `.gitlab-ci.yml` file: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml ``` The included template creates SAST jobs in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[SAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssast-ultimate) +[SAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssast) that you can later download and analyze. Due to implementation limitations, we always take the latest SAST artifact available. -### Configure SAST in the UI +### Configure SAST in the UI **(ULTIMATE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab Ultimate 13.3. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab Ultimate 13.3. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in GitLab Ultimate 13.4. -For a project that does not have a `.gitlab-ci.yml` file, you can enable SAST with a basic -configuration using the **SAST Configuration** page: +You can enable and configure SAST with a basic configuration using the **SAST Configuration** +page: -1. From the project's home page, go to **Security & Configuration** > **Configuration** in the +1. From the project's home page, go to **Security & Compliance** > **Configuration** in the left sidebar. -1. Click **Enable via Merge Request** on the Static Application Security Testing (SAST) row. -1. Enter the appropriate SAST details into the fields on the page. See [Available variables](#available-variables) - for a description of these variables. -1. Click **Create Merge Request**. +1. If the project does not have a `gitlab-ci.yml` file, click **Enable** in the Static Application Security Testing (SAST) row, otherwise click **Configure**. +1. Enter the custom SAST values, then click **Create Merge Request**. + + Custom values are stored in the `.gitlab-ci.yml` file. For variables not in the SAST Configuration page, their values are left unchanged. Default values are inherited from the GitLab SAST template. 1. Review and merge the merge request. ### Customizing the SAST settings @@ -215,25 +213,6 @@ you can use the `MAVEN_CLI_OPTS` environment variable. Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Enabling Docker-in-Docker **(ULTIMATE)** - -If needed, you can enable Docker-in-Docker to restore the SAST behavior that existed prior to GitLab -13.0. Follow these steps to do so: - -1. Configure a GitLab Runner with Docker-in-Docker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). -1. Set the variable `SAST_DISABLE_DIND` set to `false`: - - ```yaml - include: - - template: SAST.gitlab-ci.yml - - variables: - SAST_DISABLE_DIND: "false" - ``` - -This creates a single `sast` job in your CI/CD pipeline instead of multiple `<analyzer-name>-sast` -jobs. - #### Enabling Kubesec analyzer > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12752) in GitLab Ultimate 12.6. @@ -265,8 +244,8 @@ analyzer and compilation will be skipped: image: maven:3.6-jdk-8-alpine stages: - - build - - test + - build + - test include: - template: SAST.gitlab-ci.yml @@ -327,7 +306,6 @@ The following are Docker image-related variables. | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | -| `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker-ultimate). This variable is `true` by default. | #### Vulnerability filters @@ -340,23 +318,7 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre | `SAST_BANDIT_EXCLUDED_PATHS` | | Comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*, */venv/*'` | | `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | | `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | -| `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | | `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | -| `SAST_GITLEAKS_COMMIT_FROM` | | The commit a Gitleaks scan starts at. | -| `SAST_GITLEAKS_COMMIT_TO` | | The commit a Gitleaks scan ends at. | -| `SAST_GITLEAKS_HISTORIC_SCAN` | `false` | Flag to enable a historic Gitleaks scan. | - -#### Docker-in-Docker orchestrator - -The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker-ultimate). - -| Environment variable | Default value | Description | -|------------------------------------------|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). | -| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). | -| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | -| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | -| `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | #### Analyzer settings @@ -514,15 +476,14 @@ run successfully. For more information, see [Offline environments](../offline_de To use SAST in an offline environment, you need: -- To keep Docker-In-Docker disabled (default). -- A GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - A Docker Container Registry with locally available copies of SAST [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Configure certificate checking of packages (optional). NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. @@ -543,7 +504,6 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/kubesec:2 registry.gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan:2 registry.gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit:2 registry.gitlab.com/gitlab-org/security-products/analyzers/pmd-apex:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/secrets:2 registry.gitlab.com/gitlab-org/security-products/analyzers/security-code-scan:2 registry.gitlab.com/gitlab-org/security-products/analyzers/sobelow:2 registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 @@ -563,13 +523,13 @@ For details on saving and transporting Docker images as a file, see Docker's doc Add the following configuration to your `.gitlab-ci.yml` file. You must replace `SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry: - ```yaml +```yaml include: - template: SAST.gitlab-ci.yml variables: SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" - ``` +``` The SAST job should now use local copies of the SAST analyzers to scan your code and generate security reports without requiring internet access. diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 7daf2f3308b..f3e411cdc16 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -37,13 +37,13 @@ GitLab displays identified secrets visibly in a few places: To run Secret Detection jobs, by default, you need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared Runners on GitLab.com, this is enabled by default. +If you're using the shared runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** -If you use your own Runners, make sure the Docker version installed +If you use your own runners, make sure the Docker version installed is **not** `19.03.0`. See [troubleshooting information](../sast#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. ### Making Secret Detection available to all GitLab tiers @@ -83,7 +83,7 @@ variable. For example, `https://username:$password@example.com/path/to/repo` won detected, whereas `https://username:password@example.com/path/to/repo` would be detected. NOTE: **Note:** -You don't have to configure Secret Detection manually as shown in this section if you're using [Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection-ultimate) +You don't have to configure Secret Detection manually as shown in this section if you're using [Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection) provided by [Auto DevOps](../../../topics/autodevops/index.md). To enable Secret Detection for GitLab 13.1 and later, you must include the `Secret-Detection.gitlab-ci.yml` template that’s provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined in that template. @@ -99,7 +99,7 @@ The included template creates Secret Detection jobs in your CI/CD pipeline and s your project's source code for secrets. The results are saved as a -[Secret Detection report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssecret_detection-ultimate) +[Secret Detection report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssecret_detection) that you can later download and analyze. Due to implementation limitations, we always take the latest Secret Detection artifact available. diff --git a/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_3.png b/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_3.png Binary files differdeleted file mode 100644 index 7b9a48b8738..00000000000 --- a/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png b/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png Binary files differnew file mode 100644 index 00000000000..67a7bb5f368 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png Binary files differnew file mode 100644 index 00000000000..3c618090be8 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png Binary files differdeleted file mode 100644 index 77e75551bd9..00000000000 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_4.png Binary files differnew file mode 100644 index 00000000000..9ade24be16f --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png Binary files differnew file mode 100644 index 00000000000..d010adcc90c --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png Binary files differdeleted file mode 100644 index 75b5ad1d885..00000000000 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png Binary files differdeleted file mode 100644 index 591a08f4d7a..00000000000 --- a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png Binary files differnew file mode 100644 index 00000000000..c89179e739d --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_v13_3.gif b/doc/user/application_security/security_dashboard/img/pipeline_security_v13_3.gif Binary files differdeleted file mode 100644 index 29e7168b6ea..00000000000 --- a/doc/user/application_security/security_dashboard/img/pipeline_security_v13_3.gif +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png Binary files differnew file mode 100644 index 00000000000..34c64f830ba --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png diff --git a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png Binary files differdeleted file mode 100644 index 2b792727a99..00000000000 --- a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png Binary files differnew file mode 100644 index 00000000000..eb91cfc47ad --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index b8fcc513cb1..51d9b4f45cd 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -35,7 +35,7 @@ To use the instance, group, project, or pipeline security dashboard: the [supported reports](#supported-reports). 1. The configured jobs must use the [new `reports` syntax](../../../ci/pipelines/job_artifacts.md#artifactsreports). 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used. - If you're using the shared Runners on GitLab.com, this is already the case. + If you're using the shared runners on GitLab.com, this is already the case. ## Pipeline Security @@ -43,13 +43,11 @@ To use the instance, group, project, or pipeline security dashboard: At the pipeline level, the Security section displays the vulnerabilities present in the branch of the project the pipeline was run against. - + Visit the page for any pipeline that ran any of the [supported reports](#supported-reports). To view the pipeline's security findings, select the **Security** tab when viewing the pipeline. - - NOTE: **Note:** A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. @@ -63,11 +61,13 @@ to **Security & Compliance > Security Dashboard**. By default, the Security Dash detected and confirmed vulnerabilities. The Security Dashboard first displays the total number of vulnerabilities by severity (for example, -Critical, High, Medium, Low). Below this, a table displays each vulnerability's status, severity, +Critical, High, Medium, Low, Info, Unknown). Below this, a table shows each vulnerability's status, severity, and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) page to view more information about that vulnerability. -You can filter the vulnerabilities by: + + +You can filter the vulnerabilities by one or more of the following: - Status - Severity @@ -122,24 +122,28 @@ branches of all the projects you configure to display on the dashboard. It inclu [group Security Dashboard's](#group-security-dashboard) features. + + You can access the Instance Security Dashboard from the menu bar at the top of the page. Under **More**, select **Security**.  +The dashboard is empty before you add projects to it. + + + ### Adding projects to the dashboard To add projects to the dashboard: -1. Click the **Edit dashboard** button on the Instance Security Dashboard page. +1. Click **Settings** in the left navigation bar or click the **Add projects** button. 1. Search for and add one or more projects using the **Search your projects** field. 1. Click the **Add projects** button. -Once added, the Security Dashboard displays the vulnerabilities found in your chosen projects' +After you add projects, the Security Dashboard displays the vulnerabilities found in those projects' default branches. - - ## Export vulnerabilities > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. @@ -150,6 +154,8 @@ is built, the CSV report downloads to your local machine. The report contains al vulnerabilities for the projects defined in the **Security Dashboard**, as filters don't apply to the export function. + + NOTE: **Note:** It may take several minutes for the download to start if your project contains thousands of vulnerabilities. Do not close the page until the download finishes. @@ -190,7 +196,7 @@ to configure daily security scans. Each dashboard's vulnerability list contains vulnerabilities from the latest scans that were merged into the default branch. - + You can filter which vulnerabilities the Security Dashboard displays by: @@ -208,7 +214,7 @@ To create an issue associated with the vulnerability, click the **Create Issue** Once you create the issue, the vulnerability list contains a link to the issue and an icon whose color indicates the issue's status (green for open issues, blue for closed issues). - + <!-- ## Troubleshooting diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md new file mode 100644 index 00000000000..8006a49ba35 --- /dev/null +++ b/doc/user/application_security/terminology/index.md @@ -0,0 +1,171 @@ +--- +stage: Secure +group: Static Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- + +# Secure and Defend terminology + +This terminology list for GitLab Secure and Defend aims to: + +- Promote a ubiquitous language for discussing application security. +- Improve the effectiveness of communication regarding GitLab's application security features. +- Get new contributors up to speed faster. + +NOTE: **Note:** +This document defines application security terms in the specific context of GitLab's Secure and +Defend products. Terms may therefore have different meanings outside of GitLab Secure and Defend. + +## Terms + +### Analyzer + +Software that performs a scan. The scan analyzes an attack surface for vulnerabilities and produces +a report containing findings. Reports adhere to the [Secure report format](#secure-report-format). + +Analyzers integrate into GitLab using a CI job. The report produced by the analyzer is published as +an artifact once the job is complete. GitLab ingests this report, allowing users to visualize and +manage found vulnerabilities. For more information, see [Security Scanner Integration](../../../development/integrations/secure.md). + +Many GitLab analyzers follow a standard approach using Docker to run a wrapped scanner. For example, +the Docker image `bandit-sast` is an analyzer that wraps the scanner `Bandit`. You can optionally +use the [Common library](https://gitlab.com/gitlab-org/security-products/analyzers/common) +to assist in building an Analyzer. + +### Attack surface + +The different places in an application that are vulnerable to attack. Secure products discover and +search the attack surface during scans. Each product defines the attack surface differently. For +example, SAST uses files and line numbers, and DAST uses URLs. + +### CVE + +Common Vulnerabilities and Exposures (CVE®) is a list of common identifiers for publicly known +cybersecurity vulnerabilities. The list is managed by the [Mitre Corporation](https://cve.mitre.org/). + +### CVSS + +The Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing +the severity of computer system security vulnerabilities. + +### CWE + +Common Weakness Enumeration (CWE™) is a community-developed list of common software and hardware +weakness types that have security ramifications. Weaknesses are flaws, faults, bugs, +vulnerabilities, or other errors in software or hardware implementation, code, design, or +architecture. If left unaddressed, weaknesses could result in systems, networks, or hardware being +vulnerable to attack. The CWE List and associated classification taxonomy serve as a language that +you can use to identify and describe these weaknesses in terms of CWEs. + +### Duplicate finding + +A legitimate finding that is reported multiple times. This can occur when different scanners +discover the same finding, or when a single scan inadvertently reports the same finding more than +once. + +### False positive + +A finding that doesn't exist but is incorrectly reported as existing. + +### Feedback + +Feedback the user provides about a finding. Types of feedback include dismissal, creating an issue, +or creating a merge request. + +### Finding + +An asset that has the potential to be vulnerable, identified within a project by an analyzer. Assets +include but are not restricted to source code, binary packages, containers, dependencies, networks, +applications, and infrastructure. + +### Insignificant finding + +A legitimate finding that a particular customer doesn't care about. + +### Location fingerprint + +A finding's location fingerprint is a text value that's unique for each location on the attack +surface. Each Secure product defines this according to its type of attack surface. For example, SAST +incorporates file path and line number. + +### Pipeline Security tab + +A page that displays findings discovered in the associated CI pipeline. + +### Primary identifier + +A finding's primary identifier is a value unique to that finding. The external type and external ID +of the finding's [first identifier](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/v2.4.0-rc1/dist/sast-report-format.json#L228) +combine to create the value. + +Examples of primary identifiers include ZAP's `PluginID`, or `CVE` for Klar. Note that the +identifier must be stable. Subsequent scans must return the same value for the same finding, even if +the location has slightly changed. + +### Report finding + +A [finding](#finding) that only exists in a report produced by an analyzer, and is yet to be +persisted to the database. The report finding becomes a [vulnerability finding](#vulnerability-finding) +once it's imported into the database. + +### Scan type (report type) + +The type of scan. This must be one of the following: + +- `container_scanning` +- `dependency_scanning` +- `dast` +- `sast` + +### Scanner + +Software that can scan for vulnerabilities. The resulting scan report is typically not in the +[Secure report format](#secure-report-format). Examples include ESLint, Klar, and ZAP. + +### Secure product + +A group of features related to a specific area of application security with first-class support by +GitLab. Products include Container Scanning, Dependency Scanning, Dynamic Application Security +Testing (DAST), Secret Detection, Static Application Security Testing (SAST), and Fuzz Testing. Each +of these products typically include one or more analyzers. + +### Secure report format + +A standard report format that Secure products comply with when creating JSON reports. The format is described by a +[JSON schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas). + +### Security Dashboard + +Provides an overview of all the vulnerabilities for a project, group, or GitLab instance. +Vulnerabilities are only created from findings discovered on the project's default branch. + +### Vendor + +The party maintaining an analyzer. As such, a vendor is responsible for integrating a scanner into +GitLab and keeping it compatible as they evolve. A vendor isn't necessarily the author or maintainer +of the scanner, as in the case of using an open core or OSS project as a base solution of an +offering. For scanners included as part of a GitLab distribution or GitLab subscription, the vendor +is listed as GitLab. + +### Vulnerability + +A flaw that has a negative impact on the security of its environment. Vulnerabilities describe the +error or weakness, and don't describe where the error is located (see [finding](#finding)). +Each vulnerability maps to a unique finding. + +### Vulnerability finding + +When a [report finding](#report-finding) is stored to the database, it becomes a vulnerability +[finding](#finding). + +### Vulnerability tracking + +Deals with the responsibility of matching findings across scans so that a finding's life cycle can +be understood. Engineers and security teams use this information to decide whether to merge code +changes, and to see unresolved findings and when they were introduced. Vulnerabilities are tracked +by comparing the location fingerprint, primary identifier, and report type. + +### Vulnerability occurrence + +Deprecated, see [finding](#finding). diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index c916cdbfe7c..5414800b290 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -66,7 +66,7 @@ global: enabled: true metrics: enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' + - 'flow:sourceContext=namespace;destinationContext=namespace' ``` The **Container Network Policy** section displays the following information @@ -88,8 +88,9 @@ investigate it for potential threats by The **Threat Monitoring** page's **Policy** tab displays deployed network policies for all available environments. You can check a -network policy's `yaml` manifest and toggle the policy's enforcement -status. This section has the following prerequisites: +network policy's `yaml` manifest, toggle the policy's enforcement +status, and create and edit deployed policies. This section has the +following prerequisites: - Your project contains at least one [environment](../../../ci/environments/index.md) - You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd) @@ -124,3 +125,47 @@ Disabled network policies have the `podSelector` block. This narrows the scope of such a policy and as a result it doesn't affect any pods. The policy itself is still deployed to the corresponding deployment namespace. + +### Container Network Policy editor + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3403) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.4. + +The policy editor allows you to create, edit, and delete policies. To +create a new policy click the **New policy** button located in the +**Policy** tab's header. To edit an existing policy, click**Edit +policy** in the selected policy drawer. + +NOTE: **Note:** +The policy editor only supports the +[CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/policy/)specification. Regular +Kubernetes +[NetworkPolicy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#networkpolicy-v1-networking-k8s-io) +resources aren't supported. + +The policy editor has two modes: + +- The visual _Rule_ mode allows you to construct and preview policy + rules using rule blocks and related controls. +- YAML mode allows you to enter a policy definition in `.yaml` format + and is aimed at expert users and cases that the Rule mode doesn't + support. + +You can use both modes interchangeably and switch between them at any +time. If a YAML resource is incorrect, Rule mode is automatically +disabled. You must use YAML mode to fix your policy before Rule mode +is available again. + +Rule mode supports the following rule types: + +- [Labels](https://docs.cilium.io/en/v1.8/policy/language/#labels-based). +- [Entities](https://docs.cilium.io/en/v1.8/policy/language/#entities-based). +- [IP/CIDR](https://docs.cilium.io/en/v1.8/policy/language/#ip-cidr-based). Only + the `toCIDR` block without `except` is supported. +- [DNS](https://docs.cilium.io/en/v1.8/policy/language/#dns-based). +- [Level 4](https://docs.cilium.io/en/v1.8/policy/language/#layer-4-examples) + can be added to all other rules. + +Once your policy is complete, save it by pressing the **Save policy** +button at the bottom of the editor. Existing policies can also be +removed from the editor interface by clicking the **Delete policy** +button at the bottom of the editor. diff --git a/doc/user/application_security/vulnerabilities/img/vulnerability_page_v13_1.png b/doc/user/application_security/vulnerabilities/img/vulnerability_page_v13_1.png Binary files differdeleted file mode 100644 index 30a7195e1ab..00000000000 --- a/doc/user/application_security/vulnerabilities/img/vulnerability_page_v13_1.png +++ /dev/null diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index ffec4bf336d..ff383fdf553 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -9,10 +9,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. -Each security vulnerability in the [Security Dashboard](../security_dashboard/index.md#project-security-dashboard) has its own standalone -page. +Each security vulnerability in a project's [Security Dashboard](../security_dashboard/index.md#project-security-dashboard) has an individual page which includes: - +- Details of the vulnerability. +- The status of the vulnerability within the project. +- Available actions for the vulnerability. On the vulnerability page, you can interact with the vulnerability in several different ways: @@ -22,7 +23,7 @@ several different ways: - [Create issue](#creating-an-issue-for-a-vulnerability) - Create a new issue with the title and description pre-populated with information from the vulnerability report. By default, such issues are [confidential](../../project/issues/confidential_issues.md). -- [Solution](#automatic-remediation-solutions-for-vulnerabilities) - For some vulnerabilities, +- [Solution](#automatic-remediation-for-vulnerabilities) - For some vulnerabilities, a solution is provided for how to fix the vulnerability. ## Changing vulnerability status @@ -46,28 +47,7 @@ project the vulnerability came from, and pre-populates it with useful informatio the vulnerability report. After the issue is created, GitLab redirects you to the issue page so you can edit, assign, or comment on the issue. -## Automatic remediation solutions for vulnerabilities +## Automatic remediation for vulnerabilities You can fix some vulnerabilities by applying the solution that GitLab automatically -generates for you. GitLab supports the following scanners: - -- [Dependency Scanning](../dependency_scanning/index.md): Automatic Patch creation - is only available for Node.js projects managed with `yarn`. -- [Container Scanning](../container_scanning/index.md). - -When an automatic solution is available, the button in the header will show "Resolve with merge request": - - - -Selecting the button will create a merge request with the automatic solution. - -### Manually applying a suggested patch - -To manually apply the patch that was generated by GitLab for a vulnerability, select the dropdown arrow on the "Resolve -with merge request" button, then select the "Download patch to resolve" option: - - - -This will change the button text to "Download patch to resolve". Click on it to download the patch: - - +generates for you. [Read more about the automatic remediation for vulnerabilities feature](../index.md#solutions-for-vulnerabilities-auto-remediation). diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md new file mode 100644 index 00000000000..7b745577cc4 --- /dev/null +++ b/doc/user/clusters/agent/index.md @@ -0,0 +1,329 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# GitLab Kubernetes Agent **(PREMIUM ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. + +## Goals + +The [GitLab Kubernetes Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) is an active in-cluster component for solving GitLab and Kubernetes integration tasks in a secure and cloud native way. + +Features: + +1. Makes it possible to integrate GitLab with a Kubernetes cluster behind a firewall or NAT +1. Enables pull-based GitOps deployments by leveraging the [GitOps Engine](https://github.com/argoproj/gitops-engine) +1. Allows for real-time access to API endpoints within a cluster. +1. Many more features are planned. Please [review our roadmap](https://gitlab.com/groups/gitlab-org/-/epics/3329). + +## Architecture + +### GitLab Agent GitOps workflow + +```mermaid +sequenceDiagram + participant D as Developer + participant A as Application code repository + participant M as Manifest repository + participant K as Kubernetes agent + participant C as Agent configuration repository + K->C: Grab the configuration + D->>+A: Pushing code changes + A->>M: Updating manifest + loop Regularly + K-->>M: Watching changes + M-->>K: Pulling and applying changes + end +``` + +Please refer to our [full architecture documentation in the Agent project](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). + +## Getting started with GitOps using the GitLab Agent and the GitLab Cloud Native Helm chart + +There are several components that work in concert for the Agent to accomplish GitOps deployments: + +1. A Kubernetes cluster that is properly configured +1. A configuration repository that contains a `config.yaml` file. This `config.yaml` tells the Agent which repositories to synchronize with. +1. A manifest repository that contains a `manifest.yaml`. This `manifest.yaml` (which can be autogenerated) is tracked by the Agent and any changes to the file are automatically applied to the cluster. + +The setup process involves a few steps that, once completed, will enable GitOps deployments to work + +1. Installing the Agent server via GitLab Helm chart +1. Defining a configuration directory +1. Creating an Agent record in GitLab +1. Generating and copying a Secret token used to connect to the Agent +1. Installing the Agent into the cluster +1. Creating a `manifest.yaml` + +### Installing the Agent server via Helm + +Currently the GitLab Kubernetes Agent can only be deployed via our [Helm chart](https://gitlab.com/gitlab-org/charts/gitlab). + +NOTE: We are working quickly to [include the Agent in Official Linux Package](https://gitlab.com/gitlab-org/gitlab/-/issues/223060). + +If you don't already have GitLab installed via Helm please refer to our [installation documentation](https://docs.gitlab.com/charts/installation/) + +When installing/upgrading the GitLab Helm chart please consider the following Helm 2 example (if using Helm 3 please modify): + +```shell +helm upgrade --force --install gitlab . \ + --timeout 600 \ + --set global.hosts.domain=<YOUR_DOMAIN> \ + --set global.hosts.externalIP=<YOUR_IP> \ + --set certmanager-issuer.email=<YOUR_EMAIL> \ + --set name=gitlab-instance \ + --set global.kas.enabled=true +``` + +`global.kas.enabled=true` must be set in order for the Agent to be properly installed and configured. + +### Defining a configuration repository + +Next you will need a GitLab repository that will contain your Agent configuration. + +The minimal repository layout looks like this: + +`.gitlab/agents/<agent-name>/config.yaml` + +The `config.yaml` file contents should look like this: + +```yaml +gitops: + manifest_projects: + - id: "path-to/your-awesome-project" +``` + +### Creating an Agent record in GitLab + +Next you will need to create an GitLab Rails Agent record so that your GitLab project so that the Agent itself can associate with a GitLab project. This process will also yield a Secret that you will use to configure the Agent in subsequent steps. + +There are two ways to accomplish this: + +1. Via the Rails console +1. Via GraphQL + +To do this you could either run `rails c` or via GraphQL. From `rails c`: + +```ruby + project = ::Project.find_by_full_path("path-to/your-awesome-project") + agent = ::Clusters::Agent.create(name: "<agent-name>", project: project) + token = ::Clusters::AgentToken.create(agent: agent) + token.token # this will print out the token you need to use on the next step +``` + +or using GraphQL: + +with this approach, you'll need a premium license to use this feature. + +If you are new to using the GitLab GraphQL API please refer to the [Getting started with the GraphQL API page](../../../api/graphql/getting_started.md) or check out the [GraphQL Explorer](https://gitlab.com/-/graphql-explorer). + +```json + mutation createAgent { + createClusterAgent(input: { projectPath: "path-to/your-awesome-project", name: "<agent-name>" }) { + clusterAgent { + id + name + } + errors + } + } + + mutation createToken { + clusterAgentTokenCreate(input: { clusterAgentId: <cluster-agent-id-taken-from-the-previous-mutation> }) { + secret # This is the value you need to use on the next step + token { + createdAt + id + } + errors + } + } +``` + +Note that GraphQL will only show you the token once, after you've created it. + +### Creating the Kubernetes secret + +Once the token has been generated it needs to be applied to the Kubernetes cluster. + +If you didn't previously define or create a namespace you need to do that first: + +```shell +kubectl create namespace <YOUR-DESIRED-NAMESPACE> +``` + +Run the following command to create your Secret: + +```shell +kubectl create secret generic -n <YOUR-DESIRED-NAMESPACE> gitlab-agent-token --from-literal=token='YOUR_AGENT_TOKEN' +``` + +### Installing the Agent into the cluster + +Next you are now ready to install the in-cluster component of the Agent. The below is an example YAML file of the Kubernetes resources required for the Agent to be installed. + +Let's highlight a few of the details in the example below: + +1. You can replace `gitlab-agent` with <YOUR-DESIRED-NAMESPACE> +1. For the `kas-address` (Kubernetes Agent Server), you can replace `grpc://host.docker.internal:5005` with the address of the kas agent that was initialized via your Helm install. +1. If you defined your own secret name, then replace `gitlab-agent-token` with your secret name. + +`./resources.yml` + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: gitlab-agent +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: gitlab-agent +spec: + replicas: 1 + selector: + matchLabels: + app: gitlab-agent + template: + metadata: + labels: + app: gitlab-agent + spec: + serviceAccountName: gitlab-agent + containers: + - name: agent + image: "registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/agentk:latest" + args: + - --token-file=/config/token + - --kas-address + - grpc://host.docker.internal:5005 # {"$openapi":"kas-address"} + volumeMounts: + - name: token-volume + mountPath: /config + volumes: + - name: token-volume + secret: + secretName: gitlab-agent-token + strategy: + type: RollingUpdate + rollingUpdate: + maxSurge: 0 + maxUnavailable: 1 +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: gitlab-agent-write +rules: +- resources: + - '*' + apiGroups: + - '*' + verbs: + - create + - update + - delete + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: gitlab-agent-write-binding +roleRef: + name: gitlab-agent-write + kind: ClusterRole + apiGroup: rbac.authorization.k8s.io +subjects: +- name: gitlab-agent + kind: ServiceAccount + namespace: gitlab-agent +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: gitlab-agent-read +rules: +- resources: + - '*' + apiGroups: + - '*' + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: gitlab-agent-read-binding +roleRef: + name: gitlab-agent-read + kind: ClusterRole + apiGroup: rbac.authorization.k8s.io +subjects: +- name: gitlab-agent + kind: ServiceAccount + namespace: gitlab-agent + +``` + +```shell +kubectl apply -n gitlab-agent -f ./resources.yml +``` + +```plaintext +$ kubectl get pods --all-namespaces +NAMESPACE NAME READY STATUS RESTARTS AGE +gitlab-agent gitlab-agent-77689f7dcb-5skqk 1/1 Running 0 51s +kube-system coredns-f9fd979d6-n6wcw 1/1 Running 0 14m +kube-system etcd-minikube 1/1 Running 0 14m +kube-system kube-apiserver-minikube 1/1 Running 0 14m +kube-system kube-controller-manager-minikube 1/1 Running 0 14m +kube-system kube-proxy-j6zdh 1/1 Running 0 14m +kube-system kube-scheduler-minikube 1/1 Running 0 14m +kube-system storage-provisioner 1/1 Running 0 14m +``` + +### Creating a `manifest.yaml` + +In the above step, you configured a `config.yaml` to point to which GitLab projects the Agent should synchronize. Within each one of those projects, you need to create a `manifest.yaml` file which the Agent will monitor. This `manifest.yaml` can be autogenerated by a templating engine or other means. + +Example `manifest.yaml`: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: nginx-deployment +spec: + selector: + matchLabels: + app: nginx + replicas: 2 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 +``` + +The above file creates a simple NGINX deployment. + +Each time you commit and push a change to the `manifest.yaml` the Agent will observe the change. Example log: + +```plaintext +2020-09-15_14:09:04.87946 gitlab-k8s-agent : time="2020-09-15T10:09:04-04:00" level=info msg="Config: new commit" agent_id=1 commit_id=e6a3651f1faa2e928fe6120e254c122451be4eea +``` + +## Example projects + +Basic GitOps example deploying NGINX: [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent), [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 3b04c7aac18..2243ffa0cb1 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -123,14 +123,14 @@ GitLab. It is used in conjunction with [GitLab CI/CD](../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. -If the project is on GitLab.com, shared Runners are available +If the project is on GitLab.com, shared runners are available (the first 2000 minutes are free, you can -[buy more later](../../subscriptions/index.md#purchasing-additional-ci-minutes)) +[buy more later](../../subscriptions/gitlab_com/index.md#purchase-additional-ci-minutes)) and you do not have to deploy one if they are enough for your needs. If a -project-specific Runner is desired, or there are no shared Runners, it is easy +project-specific runner is desired, or there are no shared runners, it is easy to deploy one. -Note that the deployed Runner will be set as **privileged**, which means it will essentially +Note that the deployed runner will be set as **privileged**, which means it will essentially have root access to the underlying machine. This is required to build Docker images, so it is the default. Make sure you read the [security implications](../project/clusters/index.md#security-implications) @@ -529,7 +529,7 @@ fewer than 3 nodes or consisting of `f1-micro`, `g1-small`, `n1-standard-1`, or NOTE: **Note:** The Elastic Stack cluster application is intended as a log aggregation solution and is not related to our -[Advanced Global Search](../search/advanced_global_search.md) functionality, which uses a separate +[Advanced Search](../search/advanced_global_search.md) functionality, which uses a separate Elasticsearch cluster. #### Optional: deploy Kibana to perform advanced queries @@ -894,8 +894,8 @@ GitLab Runner is installed into the `gitlab-managed-apps` namespace of your clus In order for GitLab Runner to function, you **must** specify the following: -- `gitlabUrl` - the GitLab server full URL (for example, `https://example.gitlab.com`) to register the Runner against. -- `runnerRegistrationToken` - The registration token for adding new Runners to GitLab. This must be +- `gitlabUrl` - the GitLab server full URL (for example, `https://gitlab.example.com`) to register the Runner against. +- `runnerRegistrationToken` - The registration token for adding new runners to GitLab. This must be [retrieved from your GitLab instance](../../ci/runners/README.md). These values can be specified using [CI variables](../../ci/variables/README.md): @@ -910,7 +910,7 @@ management project. Refer to the available configuration options. NOTE: **Note:** -Support for installing the Runner managed application is provided by the GitLab Runner group. +Support for installing the GitLab Runner managed application is provided by the GitLab Runner group. If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Runner group](https://about.gitlab.com/handbook/product/product-categories/#runner-group). ### Install Cilium using GitLab CI/CD @@ -1421,7 +1421,7 @@ Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for After installing AppAmor, you can use profiles by adding Pod Annotations. If you're using Auto DevOps, you can [customize `auto-deploy-values.yaml`](../../topics/autodevops/customize.md#customize-values-for-helm-chart) -to annotate your pods. Although it's helpful to be aware of the [list of custom attributes](https://gitlab.com/gitlab-org/charts/auto-deploy-app#gitlabs-auto-deploy-helm-chart), you're only required to set +to annotate your pods. Although it's helpful to be aware of the [list of custom attributes](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app#gitlabs-auto-deploy-helm-chart), you're only required to set `podAnnotations` as follows: ```yaml @@ -1488,7 +1488,7 @@ The applications below can be upgraded. | Application | GitLab version | | ----------- | -------------- | -| Runner | 11.8+ | +| GitLab Runner | 11.8+ | To upgrade an application: diff --git a/doc/user/compliance/license_compliance/img/license-check_v13_4.png b/doc/user/compliance/license_compliance/img/license-check_v13_4.png Binary files differnew file mode 100644 index 00000000000..d3658cbaa18 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license-check_v13_4.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 47f14b93d29..79c2d97b972 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -16,7 +16,7 @@ is incompatible with yours, then you can deny the use of that license. You can take advantage of License Compliance by either [including the job](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance-ultimate) +[Auto License Compliance](../../../topics/autodevops/stages.md#auto-license-compliance) that is provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the License Compliance report, compares the licenses between the @@ -118,7 +118,7 @@ the `license_management` job, so you must migrate to the `license_scanning` job `License-Scanning.gitlab-ci.yml` template. The results will be saved as a -[License Compliance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportslicense_scanning-ultimate) +[License Compliance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportslicense_scanning) that you can later download and analyze. Due to implementation limitations, we always take the latest License Compliance artifact available. Behind the scenes, the [GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) @@ -265,37 +265,10 @@ license_scanning: You can supply a custom root certificate to complete TLS verification by using the `ADDITIONAL_CA_CERT_BUNDLE` [environment variable](#available-variables). -To bypass TLS verification, you can use a custom [`pip.conf`](https://pip.pypa.io/en/stable/user_guide/#config-file) -file to configure trusted hosts. - -The following `gitlab-ci.yml` file uses a [`before_script`](../../../ci/yaml/README.md#before_script-and-after_script) -to inject a custom [`pip.conf`](https://pip.pypa.io/en/stable/user_guide/#config-file): - -```yaml -include: - - template: Security/License-Scanning.gitlab-ci.yml - -license_scanning: - variables: - PIP_INDEX_URL: 'https://pypi.example.com/simple/' - before_script: - - mkdir -p ~/.config/pip/ - - cp pip.conf ~/.config/pip/pip.conf -``` - -The [`pip.conf`](https://pip.pypa.io/en/stable/reference/pip/) allows you to specify a list of -[trusted hosts](https://pip.pypa.io/en/stable/reference/pip/#cmdoption-trusted-host): - -```plaintext -[global] -trusted-host = pypi.example.com -``` - #### Using private Python repos If you have a private Python repository you can use the `PIP_INDEX_URL` [environment variable](#available-variables) -to specify its location. It's also possible to provide a custom `pip.conf` for -[additional configuration](#custom-root-certificates-for-python). +to specify its location. ### Configuring NPM projects @@ -643,8 +616,8 @@ To use License Compliance in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. @@ -705,9 +678,6 @@ with identifiers from the [SPDX license list](https://spdx.org/licenses/). A local copy of the SPDX license list is distributed with the GitLab instance. If needed, the GitLab instance's administrator can manually update it with a [Rake task](../../../raketasks/spdx.md). -Exact name matches are required for [project policies](#policies) -when running in an offline environment ([see related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/212388)). - ## License list > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13582) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.7. @@ -753,17 +723,21 @@ Developers of the project can view the policies configured in a project.  -### Enabling License Approvals within a project +## Enabling License Approvals within a project > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13067) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -`License-Check` is an approval rule you can enable to allow an approver, individual, or group to -approve a merge request that contains a `denied` license. +`License-Check` is a [security approval](../../application_security/index.md#enabling-security-approvals-within-a-project) rule you can enable to allow an individual or group to approve a +merge request that contains a `denied` license. You can enable `License-Check` one of two ways: -- Create a [project approval rule](../../project/merge_requests/merge_request_approvals.md#multiple-approval-rules-premium) - with the case-sensitive name `License-Check`. +1. Navigate to your project's **Settings > General** and expand **Merge request approvals**. +1. Click **Enable** or **Edit**. +1. Add or change the **Rule name** to `License-Check` (case sensitive). + + + - Create an approval group in the [project policies section for License Compliance](#policies). You must set this approval group's number of approvals required to greater than zero. Once you enable this group in your project, the approval rule is enabled for all merge requests. diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index f39d0d6c217..d3576ea33fe 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -501,7 +501,7 @@ introduced by [#25381](https://gitlab.com/gitlab-org/gitlab/-/issues/25381). > - It was deployed behind a feature flag, disabled by default. > - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) on GitLab 13.2. > - It's enabled on GitLab.com. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-batch-suggestions-core-only). +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-batch-suggestions). You can apply multiple suggestions at once to reduce the number of commits added to your branch to address your reviewers' requests. diff --git a/doc/user/feature_flags.md b/doc/user/feature_flags.md new file mode 100644 index 00000000000..c134b7c083c --- /dev/null +++ b/doc/user/feature_flags.md @@ -0,0 +1,43 @@ +--- +stage: none +group: Development +info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines" +description: "Understand what 'GitLab features deployed behind flags' means." +--- + +# GitLab functionality may be limited by feature flags + +> Feature flag documentation warnings were [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227806) in GitLab 13.4. + +GitLab releases some features in a disabled state using [feature flags](../development/feature_flags/index.md), +allowing them to be tested by specific groups of users and strategically +rolled out until they become enabled for everyone. + +As a GitLab user, this means that some features included in a GitLab release +may be unavailable to you. + +In this case, you'll see a warning like this in the feature documentation: + +CAUTION: **Warning:** +This feature might not be available to you. Review the **version history** note +on this page for details. + +In the version history note, you'll find information on the state of the +feature flag, including whether the feature is on ("enabled by default") or +off ("disabled by default") for self-managed GitLab instances and for users of +GitLab.com. To see the full notes: + +1. Click the three-dots icon (ellipsis) to expand version history notes: + +  + +1. Read the version history information: + +  + +If you're a user of a GitLab self-managed instance and you want to try to use a +disabled feature, you can ask a [GitLab administrator to enable it](../administration/feature_flags.md), +although changing a feature's default state isn't recommended. + +If you're a GitLab.com user and the feature is disabled, be aware that GitLab may +be working on the feature for potential release in the future. diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index aa9e6715335..67dd31efe2c 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -68,7 +68,7 @@ Below are the settings for [GitLab Pages](https://about.gitlab.com/stages-devops | IP address | `35.185.44.232` | - | | Custom domains support | yes | no | | TLS certificates support | yes | no | -| Maximum size (uncompressed) | 1G | 100M | +| Maximum size (compressed) | 1G | 100M | NOTE: **Note:** The maximum size of your Pages site is regulated by the artifacts maximum size @@ -80,16 +80,16 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | Setting | GitLab.com | Default | | ----------- | ----------------- | ------------- | -| Artifacts maximum size (uncompressed) | 1G | 100M | +| Artifacts maximum size (compressed) | 1G | 100M | | Artifacts [expiry time](../../ci/yaml/README.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | | Scheduled Pipeline Cron | `*/5 * * * *` | `19 * * * *` | | [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited | [Max CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) | `2` | Unlimited | | [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited | | [Max number of instance level variables](../../administration/instance_limits.md#number-of-instance-level-variables) | `25` | `25` | -| [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs-core-only) | 3 months | Never | +| [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs) | 3 months | Never | -## Repository size limit +## Account and limit settings GitLab.com has the following [account limits](../admin_area/settings/account_and_limit_settings.md) enabled. If a setting is not listed, it is set to the default value. @@ -99,6 +99,7 @@ or over the repository size limit, you can [reduce your repository size with Git | Setting | GitLab.com | Default | | ----------- | ----------- | ------------- | | Repository size including LFS | 10 GB | Unlimited | +| Maximum import size | 5 GB | 50 MB | NOTE: **Note:** `git push` and GitLab project imports are limited to 5 GB per request through Cloudflare. Git LFS and imports other than a file upload are not affected by this limit. @@ -124,20 +125,20 @@ A limit of: - 50 webhooks applies to groups. **(BRONZE ONLY)** - Payload is limited to 25MB -## Shared Runners +## Shared runners GitLab offers Linux and Windows shared runners hosted on GitLab.com for executing your pipelines. NOTE: **Note:** -Shared Runners provided by GitLab are **not** configurable. Consider [installing your own Runner](https://docs.gitlab.com/runner/install/) if you have specific configuration needs. +Shared runners provided by GitLab are **not** configurable. Consider [installing your own runner](https://docs.gitlab.com/runner/install/) if you have specific configuration needs. -### Linux Shared Runners +### Linux shared runners -Linux Shared Runners on GitLab.com run in [autoscale mode](https://docs.gitlab.com/runner/configuration/autoscale.html) and are powered by Google Cloud Platform. +Linux shared runners on GitLab.com run in [autoscale mode](https://docs.gitlab.com/runner/configuration/autoscale.html) and are powered by Google Cloud Platform. Autoscaling means reduced waiting times to spin up CI/CD jobs, and isolated VMs for each project, thus maximizing security. They're free to use for public open source projects and limited to 2000 CI minutes per month per group for private projects. More minutes -[can be purchased](../../subscriptions/index.md#purchasing-additional-ci-minutes), if +[can be purchased](../../subscriptions/gitlab_com/index.md#purchase-additional-ci-minutes), if needed. Read about all [GitLab.com plans](https://about.gitlab.com/pricing/). All your CI/CD jobs run on [n1-standard-1 instances](https://cloud.google.com/compute/docs/machine-types) with 3.75GB of RAM, CoreOS and the latest Docker Engine @@ -145,13 +146,13 @@ installed. Instances provide 1 vCPU and 25GB of HDD disk space. The default region of the VMs is US East1. Each instance is used only for one job, this ensures any sensitive data left on the system can't be accessed by other people their CI jobs. -The `gitlab-shared-runners-manager-X.gitlab.com` fleet of Runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They will not run untagged jobs and unlike the general fleet of shared Runners, the instances are re-used up to 40 times. +The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They will not run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. -Jobs handled by the shared Runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), +Jobs handled by the shared runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), **will be timed out after 3 hours**, regardless of the timeout configured in a project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. -Below are the shared Runners settings. +Below are the shared runners settings. | Setting | GitLab.com | Default | | ----------- | ----------------- | ---------- | @@ -162,8 +163,8 @@ Below are the shared Runners settings. #### Pre-clone script -Linux Shared Runners on GitLab.com provide a way to run commands in a CI -job before the Runner attempts to run `git init` and `git fetch` to +Linux shared runners on GitLab.com provide a way to run commands in a CI +job before the runner attempts to run `git init` and `git fetch` to download a GitLab repository. The [`pre_clone_script`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) can be used for: @@ -252,27 +253,27 @@ sentry_dsn = "X" BucketName = "bucket-name" ``` -### Windows Shared Runners (beta) +### Windows shared runners (beta) -The Windows Shared Runners are currently in +The Windows shared runners are currently in [beta](https://about.gitlab.com/handbook/product/#beta) and should not be used for production workloads. During the beta period, the -[shared runner pipeline quota](../admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only) -will apply for groups and projects in the same way as Linux Runners. +[shared runner pipeline quota](../admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota) +will apply for groups and projects in the same way as Linux runners. This may change when the beta period ends, as discussed in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30834). -Windows Shared Runners on GitLab.com automatically autoscale by +Windows shared runners on GitLab.com automatically autoscale by launching virtual machines on the Google Cloud Platform. This solution uses a new [autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). -Windows Shared Runners execute your CI/CD jobs on `n1-standard-2` instances with 2 +Windows shared runners execute your CI/CD jobs on `n1-standard-2` instances with 2 vCPUs and 7.5GB RAM. You can find a full list of available Windows packages in the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/master/cookbooks/preinstalled-software/README.md). -We want to keep iterating to get Windows Shared Runners in a stable state and +We want to keep iterating to get Windows shared runners in a stable state and [generally available](https://about.gitlab.com/handbook/product/#generally-available-ga). You can follow our work towards this goal in the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). @@ -343,7 +344,7 @@ VMTag = "windows" #### Example Below is a simple `.gitlab-ci.yml` file to show how to start using the -Windows Shared Runners: +Windows shared runners: ```yaml .shared_windows_runners: @@ -382,14 +383,14 @@ test: definition](https://about.gitlab.com/handbook/product/#beta). - The average provisioning time for a new Windows VM is 5 minutes. This means that you may notice slower build start times - on the Windows Shared Runner fleet during the beta. In a future + on the Windows shared runner fleet during the beta. In a future release we will update the autoscaler to enable the pre-provisioning of virtual machines. This will significantly reduce the time it takes to provision a VM on the Windows fleet. You can follow along in the [related issue](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/issues/32). -- The Windows Shared Runner fleet may be unavailable occasionally +- The Windows shared runner fleet may be unavailable occasionally for maintenance or updates. -- The Windows Shared Runner virtual machine instances do not use the +- The Windows shared runner virtual machine instances do not use the GitLab Docker executor. This means that you will not be able to specify [`image`](../../ci/yaml/README.md#image) or [`services`](../../ci/yaml/README.md#services) in your pipeline configuration. @@ -401,9 +402,9 @@ test: installation of additional software packages needs to be repeated for each job in your pipeline. - The job may stay in a pending state for longer than the - Linux shared Runners. + Linux shared runners. - There is the possibility that we introduce breaking changes which will - require updates to pipelines that are using the Windows Shared Runner + require updates to pipelines that are using the Windows shared runner fleet. ## Sidekiq @@ -413,7 +414,7 @@ and the following environment variables: | Setting | GitLab.com | Default | |-------- |----------- |-------- | -| `SIDEKIQ_DAEMON_MEMORY_KILLER` | - | - | +| `SIDEKIQ_DAEMON_MEMORY_KILLER` | - | `1` | | `SIDEKIQ_MEMORY_KILLER_MAX_RSS` | `2000000` | `2000000` | | `SIDEKIQ_MEMORY_KILLER_HARD_LIMIT_RSS` | - | - | | `SIDEKIQ_MEMORY_KILLER_CHECK_INTERVAL` | - | `3` | @@ -523,6 +524,15 @@ Source: - Search for `rate_limit_http_rate_per_minute` and `rate_limit_sessions_per_second` in [GitLab.com's current HAProxy settings](https://gitlab.com/gitlab-cookbooks/gitlab-haproxy/blob/master/attributes/default.rb). +### Pagination response headers + +For performance reasons, if a query returns more than 10,000 records, GitLab +doesn't return the following headers: + +- `X-Total`. +- `X-Total-Pages`. +- `rel="last"` `Link`. + ### Rack Attack initializer Details of rate limits enforced by [Rack Attack](../../security/rack_attack.md). diff --git a/doc/user/group/bulk_editing/index.md b/doc/user/group/bulk_editing/index.md index 35bdc6696eb..ec1e81bac2d 100644 --- a/doc/user/group/bulk_editing/index.md +++ b/doc/user/group/bulk_editing/index.md @@ -13,6 +13,9 @@ For more details, see [Bulk editing issues and merge requests at the project lev If you want to update attributes across multiple issues, epics, or merge requests in a group, you can do it by bulk editing them, that is, editing them together. +NOTE: **Note:** +Only the items visible on the current page are selected for bulk editing (up to 20). +  ## Bulk edit issues at the group level diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index e61b24f84f6..ebf38aef4a6 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -46,7 +46,7 @@ You can associate more than one Kubernetes cluster to your group, and maintain d for different environments, such as development, staging, and production. When adding another cluster, -[set an environment scope](#environment-scopes-premium) to help +[set an environment scope](#environment-scopes) to help differentiate the new cluster from your other clusters. ## GitLab-managed clusters @@ -162,10 +162,10 @@ For a consolidated view of which CI [environments](../../../ci/environments/inde are deployed to the Kubernetes cluster, see the documentation for [cluster environments](../../clusters/environments.md). -## Security of Runners +## Security of runners -For important information about securely configuring GitLab Runners, see -[Security of Runners](../../project/clusters/add_remove_clusters.md#security-of-gitlab-runners) +For important information about securely configuring runners, see +[Security of runners](../../project/clusters/add_remove_clusters.md#security-of-runners) documentation for project-level clusters. ## More information diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 04b57d13828..e8bcb7219fc 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -48,14 +48,14 @@ To learn what you can do with an epic, see [Manage epics](manage_epics.md). Poss - [Search for an epic from epics list page](manage_epics.md#search-for-an-epic-from-epics-list-page) - [Make an epic confidential](manage_epics.md#make-an-epic-confidential) - [Manage issues assigned to an epic](manage_epics.md#manage-issues-assigned-to-an-epic) -- [Manage multi-level child epics **(ULTIMATE)**](manage_epics.md#manage-multi-level-child-epics-ultimate) +- [Manage multi-level child epics **(ULTIMATE)**](manage_epics.md#manage-multi-level-child-epics) ## Relationships between epics and issues The possible relationships between epics and issues are: - An epic is the parent of one or more issues. -- An epic is the parent of one or more child epics. For details see [Multi-level child epics](#multi-level-child-epics-ultimate). **(ULTIMATE)** +- An epic is the parent of one or more child epics. For details see [Multi-level child epics](#multi-level-child-epics). **(ULTIMATE)** ```mermaid graph TD @@ -73,7 +73,7 @@ to add an issue to an epic, reorder issues, move issues between epics, or promot > - The health status of a closed issue [will be hidden](https://gitlab.com/gitlab-org/gitlab/-/issues/220867) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.3 or later. You can report on and quickly respond to the health of individual issues and epics by setting a -red, amber, or green [health status on an issue](../../project/issues/index.md#health-status-ultimate), +red, amber, or green [health status on an issue](../../project/issues/index.md#health-status), which will appear on your Epic tree. ### Disable Issue health status in Epic tree @@ -92,7 +92,7 @@ When you add an epic that's already linked to a parent epic, the link to its cur An epic can have multiple child epics up to the maximum depth of five. -See [Manage multi-level child epics](manage_epics.md#manage-multi-level-child-epics-ultimate) for +See [Manage multi-level child epics](manage_epics.md#manage-multi-level-child-epics) for steps to create, move, reorder, or delete child epics. ## Start date and due date @@ -145,7 +145,7 @@ then the parent epic's start date will reflect the change and this will propagat > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7327) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.10. -If your epic contains one or more [child epics](#multi-level-child-epics-ultimate) which +If your epic contains one or more [child epics](#multi-level-child-epics) which have a [start or due date](#start-date-and-due-date), a [roadmap](../roadmap/index.md) view of the child epics is listed under the parent epic. diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index aaa5d3a3034..c09032bffb2 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -164,21 +164,9 @@ To make an epic confidential: - **In an existing epic:** in the epic's sidebar, select **Edit** next to **Confidentiality** then select **Turn on**. -### Disable confidential epics **(PREMIUM ONLY)** - -The confidential epics feature is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it for your self-managed instance. - -To disable it: - -```ruby -Feature.disable(:confidential_epics) -``` - ## Manage issues assigned to an epic -### Add an issue to an epic +### Add a new issue to an epic You can add an existing issue to an epic, or, create a new issue that's automatically added to the epic. @@ -190,13 +178,13 @@ subgroups, are eligible to be added to the epic. Newly added issues appear at th issues in the **Epics and Issues** tab. An epic contains a list of issues and an issue can be associated with at most one epic. -When you add an issue that's already linked to an epic, the issue is automatically unlinked from its +When you add a new issue that's already linked to an epic, the issue is automatically unlinked from its current parent. -To add an issue to an epic: +To add a new issue to an epic: 1. Click the **Add** dropdown button. -1. Click **Add an issue**. +1. Click **Add a new issue**. 1. Identify the issue to be added, using either of the following methods: - Paste the link of the issue. - Search for the desired issue by entering part of the issue's title, then selecting the desired @@ -298,7 +286,7 @@ For more on epic templates, see [Epic Templates - Repeatable sets of issues](htt To add a child epic to an epic: 1. Click the **Add** dropdown button. -1. Click **Add an epic**. +1. Click **Add a new epic**. 1. Identify the epic to be added, using either of the following methods: - Paste the link of the epic. - Search for the desired issue by entering part of the epic's title, then selecting the desired @@ -313,7 +301,7 @@ To add a child epic to an epic: New child epics appear at the top of the list in the **Epics and Issues** tab. You can move child epics from one epic to another. -When you add an epic that's already linked to a parent epic, the link to its current parent is removed. +When you add a new epic that's already linked to a parent epic, the link to its current parent is removed. Issues and child epics cannot be intermingled. To move child epics to another epic: diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 22ad311ab4f..32b76cf9280 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -227,7 +227,7 @@ To change this setting for a specific group: To change this setting globally, see [Default branch protection](../admin_area/settings/visibility_and_access_controls.md#default-branch-protection). NOTE: **Note:** -In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../admin_area/settings/visibility_and_access_controls.md#disable-group-owners-from-updating-default-branch-protection-premium-only). +In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../admin_area/settings/visibility_and_access_controls.md#disable-group-owners-from-updating-default-branch-protection). ## Add projects to a group @@ -340,7 +340,7 @@ Group syncing allows LDAP groups to be mapped to GitLab groups. This provides mo Group links can be created using either a CN or a filter. These group links are created on the **Group Settings -> LDAP Synchronization** page. After configuring the link, it may take over an hour for the users to sync with the GitLab group. -For more information on the administration of LDAP and group sync, refer to the [main LDAP documentation](../../administration/auth/ldap/index.md#group-sync-starter-only). +For more information on the administration of LDAP and group sync, refer to the [main LDAP documentation](../../administration/auth/ldap/index.md#group-sync). NOTE: **Note:** If an LDAP user is a group member when LDAP Synchronization is added, and they are not part of the LDAP group, they will be removed from the group. @@ -363,7 +363,7 @@ To create group links via filter: 1. Select the **LDAP Server** for the link. 1. Select `LDAP user filter` as the **Sync method**. -1. Input your filter in the **LDAP User filter** box. Follow the [documentation on user filters](../../administration/auth/ldap/index.md#set-up-ldap-user-filter-core-only). +1. Input your filter in the **LDAP User filter** box. Follow the [documentation on user filters](../../administration/auth/ldap/index.md#set-up-ldap-user-filter). 1. In the **LDAP Access** section, select the [permission level](../permissions.md) for users synced in this group. 1. Click the `Add Synchronization` button to save this group link. @@ -480,7 +480,7 @@ To remove a group and its contents: This action either: - Removes the group, and also queues a background job to delete all projects in that group. -- Since [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [Premium or Silver](https://about.gitlab.com/pricing/premium/) or higher tiers, marks a group for deletion. The deletion will happen 7 days later by default, but this can be changed in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay-premium-only). +- Since [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on [Premium or Silver](https://about.gitlab.com/pricing/premium/) or higher tiers, marks a group for deletion. The deletion will happen 7 days later by default, but this can be changed in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). ### Restore a group **(PREMIUM)** @@ -660,7 +660,7 @@ Optionally, on [Premium or Silver](https://about.gitlab.com/pricing/) or higher you can configure the projects within a group to be deleted after a delayed interval. During this interval period, the projects will be in a read-only state and can be restored, if required. -The interval period defaults to 7 days, and can be modified by an admin in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay-premium-only). +The interval period defaults to 7 days, and can be modified by an admin in the [instance settings](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). To enable delayed deletion of projects: @@ -668,6 +668,9 @@ To enable delayed deletion of projects: 1. Expand the **Permissions, LFS, 2FA** section, and check **Enable delayed project removal**. 1. Click **Save changes**. +NOTE: **Note:** +The group setting for delayed deletion is not inherited by sub-groups and has to be individually defined for each group. + #### Prevent project forking outside group **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216987) in GitLab 13.3. @@ -711,7 +714,8 @@ If your namespace shows `N/A` as the total storage usage, you can trigger a reca #### Group push rules **(STARTER)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34370) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34370) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/224129) in GitLab 13.4. Group push rules allow group maintainers to set [push rules](../../push_rules/push_rules.md) for newly created projects within the specific group. @@ -724,18 +728,10 @@ When set, new subgroups have push rules set for them based on either: - The closest parent group with push rules defined. - Push rules set at the instance level, if no parent groups have push rules defined. -##### Enabling the feature - -This feature comes with the `:group_push_rules` feature flag disabled by default. It can be enabled for specific group using feature flag [API endpoint](../../api/features.md#set-or-create-a-feature) or by GitLab administrator with Rails console access by running: - -```ruby -Feature.enable(:group_push_rules) -``` - ### Maximum artifacts size **(CORE ONLY)** For information about setting a maximum artifact size for a group, see -[Maximum artifacts size](../admin_area/settings/continuous_integration.md#maximum-artifacts-size-core-only). +[Maximum artifacts size](../admin_area/settings/continuous_integration.md#maximum-artifacts-size). ## User contribution analysis **(STARTER)** @@ -747,6 +743,12 @@ and issues) performed by your group members. With [GitLab Issue Analytics](issues_analytics/index.md), you can see a bar chart of the number of issues created each month in your groups. +## Repositories analytics **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215104) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. + +With [GitLab Repositories Analytics](repositories_analytics/index.md), you can download a CSV of the latest coverage data for all the projects in your group. + ## Dependency Proxy **(PREMIUM)** Use GitLab as a [dependency proxy](../packages/dependency_proxy/index.md) for upstream Docker images. diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index f04472a29bb..20cbc043d83 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - It's enabled on GitLab.com. > - It's able to be enabled or disabled per-group. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-iterations-core-only). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#disable-iterations). **(CORE ONLY)** Iterations are a way to track issues over a period of time. This allows teams to track velocity and volatility metrics. Iterations can be used with [milestones](../../project/milestones/index.md) @@ -62,7 +62,7 @@ To edit an iteration, click the three-dot menu (**{ellipsis_v}**) > **Edit itera > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216158) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.2. To learn how to add an issue to an iteration, see the steps in -[Managing issues](../../project/issues/managing_issues.md#add-an-issue-to-an-iteration-starter). +[Managing issues](../../project/issues/managing_issues.md#add-an-issue-to-an-iteration). ## Disable Iterations **(CORE ONLY)** @@ -76,7 +76,7 @@ To enable it: # Instance-wide Feature.enable(:group_iterations) # or by group -Feature.enable(:group_iterations, Group.find(<group id>)) +Feature.enable(:group_iterations, Group.find(<group ID>)) ``` To disable it: @@ -85,7 +85,7 @@ To disable it: # Instance-wide Feature.disable(:group_iterations) # or by group -Feature.disable(:group_iterations, Group.find(<group id>)) +Feature.disable(:group_iterations, Group.find(<group ID>)) ``` <!-- ## Troubleshooting diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md new file mode 100644 index 00000000000..b013e371ed2 --- /dev/null +++ b/doc/user/group/repositories_analytics/index.md @@ -0,0 +1,67 @@ +--- +type: reference +stage: Verify +group: Analytics +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Repositories Analytics **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215104) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. +> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-repositories-analytics). **(CORE ONLY)** + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +You can get a CSV of the code coverage data for all of the projects in your group. This report has a maximum of 1000 records. To get the report: + +1. Go to your group's **Analytics > Repositories** page +1. Click **Download historic test coverage data (.csv)**, +1. In the popup, select the projects you want to include in the report. +1. Select the date range for the report from the preset options. +1. Click **Download test coverage data (.csv)**. + +The projects dropdown shows up to 100 projects from your group. If the project you want to check is not in the dropdown list, you can select **All projects** to download the report for all projects in your group, including any projects that are not listed. There is a plan to improve this behavior in this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/250684). + +For each day that a coverage report was generated by a job in a project's pipeline, there will be a row in the CSV which includes: + +- The date when the coverage job ran +- The name of the job that generated the coverage report +- The name of the project +- The coverage value + +If the project's code coverage was calculated more than once in a day, we will take the last value from that day. + +## Enable or disable Repositories Analytics **(CORE ONLY)** + +Repositories Analytics is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:group_coverage_reports) +``` + +To disable it: + +```ruby +Feature.disable(:group_coverage_reports) +``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/saml_sso/group_managed_accounts.md b/doc/user/group/saml_sso/group_managed_accounts.md index 126970ebbb6..7497d036d31 100644 --- a/doc/user/group/saml_sso/group_managed_accounts.md +++ b/doc/user/group/saml_sso/group_managed_accounts.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group Managed Accounts **(PREMIUM)** CAUTION: **Caution:** -This [Closed Beta](https://about.gitlab.com/handbook/product/#closed-beta) feature is being re-evaluated in favor of a different +This [Closed Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#sts=Closed%20Beta) feature is being re-evaluated in favor of a different [identity model](https://gitlab.com/gitlab-org/gitlab/-/issues/218631) that does not require separate accounts. We recommend that group administrators who haven't yet implemented this feature wait for the new solution. @@ -76,7 +76,8 @@ This restriction also applies to projects forked from or to those groups. Groups with group-managed accounts can disallow forking of projects to destinations outside the group. To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. -When enabled, projects within the group can only be forked to other destinations within the group (including its subgroups). +When enabled **at the parent group level**, projects within the group can be forked +only to other destinations within the group (including its subgroups). ## Credentials inventory for Group-managed accounts **(ULTIMATE)** @@ -104,7 +105,7 @@ Since personal access tokens are the only token needed for programmatic access t ### Setting a limit -Only a GitLab administrator or an owner of a group-managed account can set a limit. When this field is left empty, the [instance-level restriction](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only) on the lifetime of personal access tokens apply. +Only a GitLab administrator or an owner of a group-managed account can set a limit. When this field is left empty, the [instance-level restriction](../../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens) on the lifetime of personal access tokens apply. To set a limit on how long personal access tokens are valid for users in a group managed account: diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index f516f4080fa..57b9cc92c51 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -274,10 +274,10 @@ Group SAML on a self-managed instance is limited when compared to the recommende [instance-wide SAML](../../../integration/saml.md). The recommended solution allows you to take advantage of: - [LDAP compatibility](../../../administration/auth/ldap/index.md). -- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap-starter-only). -- [Required groups](../../../integration/saml.md#required-groups-starter-only). -- [Admin groups](../../../integration/saml.md#admin-groups-starter-only). -- [Auditor groups](../../../integration/saml.md#auditor-groups-starter-only). +- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap). +- [Required groups](../../../integration/saml.md#required-groups). +- [Admin groups](../../../integration/saml.md#admin-groups). +- [Auditor groups](../../../integration/saml.md#auditor-groups). ### Omnibus installations @@ -361,7 +361,7 @@ Here are possible causes and solutions: Getting both of these errors at the same time suggests the NameID capitalization provided by the Identity Provider didn't exactly match the previous value for that user. -This can be prevented by configuring the [NameID](#nameid) to return a consistent value. Fixing this for an individual user involves [unlinking SAML in the GitLab account](#unlinking-accounts), although this will cause group membership and Todos to be lost. +This can be prevented by configuring the [NameID](#nameid) to return a consistent value. Fixing this for an individual user involves [unlinking SAML in the GitLab account](#unlinking-accounts), although this will cause group membership and to-dos to be lost. ### Message: "Request to link SAML account must be authorized" @@ -377,7 +377,7 @@ Alternatively, when users need to [link SAML to their existing GitLab.com accoun | Cause | Solution | |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| As mentioned in the [NameID](#nameid) section, if the NameID changes for any user, the user can be locked out. This is a common problem when an email address is used as the identifier. | Follow the steps outlined in the ["SAML authentication failed: User has already been taken"](#message-saml-authentication-failed-user-has-already-been-taken) section. If many users are affected, we recommend that you use the appropriate API. | +| As mentioned in the [NameID](#nameid) section, if the NameID changes for any user, the user can be locked out. This is a common problem when an email address is used as the identifier. | Follow the steps outlined in the ["SAML authentication failed: User has already been taken"](#message-saml-authentication-failed-user-has-already-been-taken) section. | ### I need to change my SAML app diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 9a2bd2e8806..4f74e672392 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -159,7 +159,16 @@ application described above. ## User access and linking setup -As long as [Group SAML](index.md) has been configured, prior to turning on sync, existing GitLab.com users can link to their accounts in one of the following ways, before synchronization is active: +The following diagram is a general outline on what happens when you add users to your SCIM app: + +```mermaid +graph TD + A[Add User to SCIM app] -->|IdP sends user info to GitLab| B(GitLab: Does the email exists?) + B -->|No| C[GitLab creates user with SCIM identity] + B -->|Yes| D[GitLab sends message back 'Email exists'] +``` + +As long as [Group SAML](index.md) has been configured, existing GitLab.com users can link to their accounts in one of the following ways: - By updating their *primary* email address in their GitLab.com user account to match their identity provider's user profile email address. - By following these steps: @@ -168,21 +177,41 @@ As long as [Group SAML](index.md) has been configured, prior to turning on sync, 1. Click on the GitLab app in the identity provider's dashboard or visit the **GitLab single sign-on URL**. 1. Click on the **Authorize** button. +We recommend users do this prior to turning on sync, because while synchronization is active, there may be provisioning errors for existing users. + New users and existing users on subsequent visits can access the group through the identify provider's dashboard or by visiting links directly. For role information, please see the [Group SAML page](index.md#user-access-and-management) ### Blocking access -To rescind access to the group, we recommend removing the user from the identity +To rescind access to the group, remove the user from the identity provider or users list for the specific app. -Upon the next sync, the user will be deprovisioned, which means that the user will be removed from the group. The user account will not be deleted unless using [group managed accounts](group_managed_accounts.md). +Upon the next sync, the user is deprovisioned, which means that the user is removed from the group. + +NOTE: **Note:** +Deprovisioning does not delete the user account. + +```mermaid +graph TD + A[Remove User from SCIM app] -->|IdP sends request to GitLab| B(GitLab: Is the user part of the group?) + B -->|No| C[Nothing to do] + B -->|Yes| D[GitLab removes user from GitLab group] +``` ## Troubleshooting This section contains possible solutions for problems you might encounter. +### How come I can't add a user after I removed them? + +As outlined in the [Blocking access section](#blocking-access), when you remove a user, they are removed from the group. However, their account is not deleted. + +When the user is added back to the SCIM app, GitLab cannot create a new user because the user already exists. + +Solution: Have a user sign in directly to GitLab, then [manually link](#user-access-and-linking-setup) their account. + ### Azure #### How do I verify my SCIM configuration is correct? @@ -236,7 +265,7 @@ Alternatively, the [SCIM API](../../../api/scim.md#get-a-list-of-saml-users) can For example: ```shell -curl 'https://example.gitlab.com/api/scim/v2/groups/GROUP_NAME/Users?startIndex=1"' --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json" +curl 'https://gitlab.example.com/api/scim/v2/groups/GROUP_NAME/Users?startIndex=1"' --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json" ``` To see how this compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](index.md#saml-debugging-tools). diff --git a/doc/user/group/settings/img/import_panel_v13_1.png b/doc/user/group/settings/img/import_panel_v13_1.png Binary files differdeleted file mode 100644 index ce2eb579446..00000000000 --- a/doc/user/group/settings/img/import_panel_v13_1.png +++ /dev/null diff --git a/doc/user/group/settings/img/import_panel_v13_4.png b/doc/user/group/settings/img/import_panel_v13_4.png Binary files differnew file mode 100644 index 00000000000..e4e5b0e91a1 --- /dev/null +++ b/doc/user/group/settings/img/import_panel_v13_4.png diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md index ae83c8da462..77cb862a49d 100644 --- a/doc/user/group/settings/import_export.md +++ b/doc/user/group/settings/import_export.md @@ -52,7 +52,7 @@ The following items are exported: The following items are **not** exported: - Projects -- Runners token +- Runner tokens - SAML discovery tokens NOTE: **Note:** @@ -94,7 +94,7 @@ on an existing group's page. 1. On the New Group page, select the **Import group** tab. -  +  1. Enter your group name. diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 235855b6e3a..6de38354c5e 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -115,10 +115,10 @@ When you add a member to a subgroup, they inherit the membership and permission level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. -Jobs for pipelines in subgroups can use [Runners](../../../ci/runners/README.md) registered to the parent group(s). +Jobs for pipelines in subgroups can use [runners](../../../ci/runners/README.md) registered to the parent group(s). This means secrets configured for the parent group are available to subgroup jobs. -In addition, maintainers of projects that belong to subgroups can see the details of Runners registered to parent group(s). +In addition, maintainers of projects that belong to subgroups can see the details of runners registered to parent group(s). The group permissions for a member can be changed only by Owners, and only on the **Members** page of the group the member was added. diff --git a/doc/user/img/feature_flags_history_note_info_v13_2.png b/doc/user/img/feature_flags_history_note_info_v13_2.png Binary files differnew file mode 100644 index 00000000000..403a6002603 --- /dev/null +++ b/doc/user/img/feature_flags_history_note_info_v13_2.png diff --git a/doc/user/img/version_history_notes_collapsed_v13_2.png b/doc/user/img/version_history_notes_collapsed_v13_2.png Binary files differnew file mode 100644 index 00000000000..42ea11ae8ff --- /dev/null +++ b/doc/user/img/version_history_notes_collapsed_v13_2.png diff --git a/doc/user/index.md b/doc/user/index.md index 1ab3541575c..ce8713591ab 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -57,7 +57,7 @@ With GitLab Enterprise Edition, you can also: - [Multiple Issue Boards](project/issue_board.md#multiple-issue-boards). - Create formal relationships between issues with [Related Issues](project/issues/related_issues.md). - Use [Burndown Charts](project/milestones/burndown_charts.md) to track progress during a sprint or while working on a new version of their software. -- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. +- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Search](search/advanced_global_search.md) and [Advanced Search Syntax](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. - [Authenticate users with Kerberos](../integration/kerberos.md). - [Mirror a repository](project/repository/repository_mirroring.md) from elsewhere on your local server. - [Export issues as CSV](project/issues/csv_export.md). @@ -134,10 +134,10 @@ the best of GitLab Flavored Markdown in your threads, comments, issues and merge requests descriptions, and everywhere else GFM is supported. -## Todos +## To-Do List -Never forget to reply to your collaborators. [GitLab Todos](todos.md) -are a tool for working faster and more effectively with your team, +Never forget to reply to your collaborators. [GitLab To-Do List](todos.md) +is a tool for working faster and more effectively with your team, by listing all user or group mentions, as well as issues and merge requests you're assigned to. @@ -151,6 +151,10 @@ requests you're assigned to. you have quick access to. You can also gather feedback on them through [Discussions](#discussions). +## Features behind feature flags + +Understand what [features behind feature flags](feature_flags.md) mean. + ## Keyboard shortcuts There are many [keyboard shortcuts](shortcuts.md) in GitLab to help you navigate between @@ -175,9 +179,9 @@ Automate GitLab via [API](../api/README.md). Learn what is [Git](../topics/git/index.md) and its best practices. -## Instance statistics +## Instance-level analytics -See [various statistics](instance_statistics/index.md) of your GitLab instance. +See [various statistics](admin_area/analytics/index.md) of your GitLab instance. ## Operations Dashboard **(PREMIUM)** diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 2a49ca18aaf..227a67f8c8a 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -376,17 +376,18 @@ can configure this manually as follows: ### Example `.gitlab-ci.yaml` file ```yaml -image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest +default: + image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest + + cache: + key: example-production + paths: + - ${TF_ROOT}/.terraform variables: TF_ROOT: ${CI_PROJECT_DIR}/environments/example/production TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/example-production -cache: - key: example-production - paths: - - ${TF_ROOT}/.terraform - before_script: - cd ${TF_ROOT} @@ -433,18 +434,19 @@ apply: ### Multiple Terraform Plan reports -Starting with 13.2, you can display mutiple reports on the Merge Request page. The reports will also display the `artifacts: name:`. See example below for a suggested setup. +Starting with 13.2, you can display multiple reports on the Merge Request page. The reports will also display the `artifacts: name:`. See example below for a suggested setup. ```yaml -image: - name: registry.gitlab.com/gitlab-org/gitlab-build-images:terraform - entrypoint: - - '/usr/bin/env' - - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' - -cache: - paths: - - .terraform +default: + image: + name: registry.gitlab.com/gitlab-org/gitlab-build-images:terraform + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + + cache: + paths: + - .terraform stages: - build diff --git a/doc/user/instance_statistics/convdev.md b/doc/user/instance_statistics/convdev.md deleted file mode 100644 index a1a4eca191c..00000000000 --- a/doc/user/instance_statistics/convdev.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -redirect_to: '../instance_statistics/dev_ops_score.md' ---- - -This document was moved to [another location](../instance_statistics/dev_ops_score.md). diff --git a/doc/user/instance_statistics/img/cohorts.png b/doc/user/instance_statistics/img/cohorts.png Binary files differdeleted file mode 100644 index 19250e385aa..00000000000 --- a/doc/user/instance_statistics/img/cohorts.png +++ /dev/null diff --git a/doc/user/instance_statistics/img/dev_ops_score_v12_6.png b/doc/user/instance_statistics/img/dev_ops_score_v12_6.png Binary files differdeleted file mode 100644 index af07e9323d6..00000000000 --- a/doc/user/instance_statistics/img/dev_ops_score_v12_6.png +++ /dev/null diff --git a/doc/user/instance_statistics/img/instance_statistics_button_v12_6.png b/doc/user/instance_statistics/img/instance_statistics_button_v12_6.png Binary files differdeleted file mode 100644 index e5f033141ca..00000000000 --- a/doc/user/instance_statistics/img/instance_statistics_button_v12_6.png +++ /dev/null diff --git a/doc/user/instance_statistics/index.md b/doc/user/instance_statistics/index.md deleted file mode 100644 index b09651e04ee..00000000000 --- a/doc/user/instance_statistics/index.md +++ /dev/null @@ -1,15 +0,0 @@ -# Instance statistics - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41416) in GitLab 11.2. - -Instance statistics gives users or admins access to instance-wide analytics. -They are accessible to all users by default (GitLab admins can restrict its -visibility in the [Admin Area](../admin_area/settings/usage_statistics.md)), -and can be accessed via the top bar. - - - -There are two kinds of statistics: - -- [Dev Ops Score](dev_ops_score.md): Provides an overview of your entire instance's feature usage. -- [User Cohorts](user_cohorts.md): Display the monthly cohorts of new users and their activities over time. diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 12d65f75b37..03a4e4cb244 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -438,6 +438,11 @@ GFM recognizes the following: | commit range comparison | `9ba12248...b19a04f5` | `namespace/project@9ba12248...b19a04f5` | `project@9ba12248...b19a04f5` | | repository file references | `[README](doc/README)` | | | | repository file line references | `[README](doc/README#L13)` | | | +| [alert](../operations/incident_management/alerts.md) | `^alert#123` | `namespace/project^alert#123` | `project^alert#123` | + +For example, referencing an issue by using `#123` will format the output as a link +to issue number 123 with text `#123`. Likewise, a link to issue number 123 will be +recognized and formatted with text `#123`. In addition to this, links to some objects are also recognized and formatted. Some examples of these are: @@ -1419,13 +1424,15 @@ Example: | cell 1 | cell 2 | cell 3 | | cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It eventually wraps the text when the cell is too large for the display size. | | cell 7 | | cell <br> 9 | +| cell 10 | <ul><li> - [ ] Task One </li></ul> | <ul><li> - [ ] Task Two </li><li> - [ ] Task Three </li></ul> | ``` | header 1 | header 2 | header 3 | | --- | ------ |---------:| | cell 1 | cell 2 | cell 3 | -| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's okay. It eventually wraps the text when the cell is too large for the display size. | +| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It eventually wraps the text when the cell is too large for the display size. | | cell 7 | | cell <br> 9 | +| cell 10 | <ul><li> - [ ] Task One </li></ul> | <ul><li> - [ ] Task Two </li><li> - [ ] Task Three </li></ul> | Additionally, you can choose the alignment of text within columns by adding colons (`:`) to the sides of the "dash" lines in the second row. This affects every cell in the column. diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 9b1f23f6d59..89e02b4847c 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -79,12 +79,17 @@ git push origin v1.0.0 Now that the basics of our project is completed, we can publish the package. To publish the package, you need: -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +- A personal access token or `CI_JOB_TOKEN`. + + ([Deploy tokens](./../../project/deploy_tokens/index.md) are not yet supported for use with Composer.) + - Your project ID which can be found on the home page of your project. To publish the package hosted on GitLab, make a `POST` request to the GitLab package API. A tool like `curl` can be used to make this request: +You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. For example: + ```shell curl --data tag=<tag> 'https://__token__:<personal-access-token>@gitlab.com/api/v4/projects/<project_id>/packages/composer' ``` @@ -97,6 +102,21 @@ Where: If the above command succeeds, you now should be able to see the package under the **Packages & Registries** section of your project page. +### Publishing the package with CI/CD + +To work with Composer commands within [GitLab CI/CD](./../../../ci/README.md), you can +publish Composer packages by using `CI_JOB_TOKEN` in your `.gitlab-ci.yml` file: + +```yaml +stages: + - deploy + +deploy: + stage: deploy + script: + - 'curl --header "Job-Token: $CI_JOB_TOKEN" --data tag=<tag> "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/packages/composer"' +``` + ### Installing a package To install your package, you need: @@ -130,11 +150,8 @@ You also need to create a `auth.json` file with your GitLab credentials: ```json { - "http-basic": { - "gitlab.com": { - "username": "___token___", - "password": "<personal_access_token>" - } + "gitlab-token": { + "gitlab.com": "<personal_access_token>" } } ``` @@ -155,4 +172,4 @@ CAUTION: **Important:** Make sure to never commit the `auth.json` file to your repository. To install packages from a CI job, consider using the [`composer config`](https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#authentication) tool with your personal access token stored in a [GitLab CI/CD environment variable](../../../ci/variables/README.md) or in -[Hashicorp Vault](../../../ci/examples/authenticating-with-hashicorp-vault/index.md). +[Hashicorp Vault](../../../ci/secrets/index.md). diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index e8014ad2b84..7c3082e0f83 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -83,15 +83,13 @@ conan new Hello/0.1 -t Next, create a package for that recipe by running `conan create` providing the Conan user and channel: ```shell -conan create . my-org+my-group+my-project/beta +conan create . mycompany/beta ``` NOTE: **Note:** -Current [naming restrictions](#package-recipe-naming-convention) require you to name the `user` value as the `+` separated path of your project on GitLab. +If you are using the [instance level remote](#instance-level-remote), a specific [naming convention](#package-recipe-naming-convention-for-instance-level-remote) must be followed. -The example above would create a package belonging to this project: `https://gitlab.com/my-org/my-group/my-project` with a channel of `beta`. - -These two example commands generate a final package with the recipe `Hello/0.1@my-org+my-group+my-project/beta`. +These two example commands generate a final package with the recipe `Hello/0.1@mycompany/beta`. For more advanced details on creating and managing your packages, refer to the [Conan docs](https://docs.conan.io/en/latest/creating_packages.html). @@ -99,6 +97,38 @@ You are now ready to upload your package to the GitLab registry. To get started, ## Adding the GitLab Package Registry as a Conan remote +You can add the GitLab Package Registry as a Conan remote at the project or instance level. + +### Project level remote + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11679) in GitLab 13.4. + +The project level remote allows you to work with packages within a given project. +The advantage of using the project level remote is there are no restrictions to your +package name, however all GitLab Conan packages require a full recipe +with the user and channel (`package_name/version@user/channel`). + +To add the remote: + +```shell +conan remote add gitlab https://gitlab.example.com/api/v4/projects/<project_id>/packages/conan +``` + +Once the remote is set, you can use the remote when running Conan commands by adding `--remote=gitlab` to the end of your commands. + +For example: + +```shell +conan search Hello* --all --remote=gitlab +``` + +### Instance level remote + +The instance level remote allows you to use a single remote to access packages accross your entire +GitLab instance. However, when using this remote, there are certain +[package naming restrictions](#package-recipe-naming-convention-for-instance-level-remote) +that must be followed. + Add a new remote to your Conan configuration: ```shell @@ -110,9 +140,28 @@ Once the remote is set, you can use the remote when running Conan commands by ad For example: ```shell -conan search Hello* --all --remote=gitlab +conan search 'Hello*' --remote=gitlab ``` +#### Package recipe naming convention for instance level remote + +The standard Conan recipe convention looks like `package_name/version@user/channel`, +but if you're using the [instance level remote](#instance-level-remote), the recipe +`user` must be the plus sign (`+`) separated project path. + +The following table shows some example recipes you can give your package based on +the project name and path. + +| Project | Package | Supported | +| ---------------------------------- | ----------------------------------------------- | --------- | +| `foo/bar` | `my-package/1.0.0@foo+bar/stable` | Yes | +| `foo/bar-baz/buz` | `my-package/1.0.0@foo+bar-baz+buz/stable` | Yes | +| `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes | +| `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No | + +NOTE: **Note:** +[Project level remotes](#project-level-remote) allow for more flexible package names. + ## Authenticating to the GitLab Conan Repository You need a personal access token or deploy token. @@ -142,7 +191,7 @@ Alternatively, you could explicitly include your credentials in any given comman For example: ```shell -CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@my-group+my-project/beta --all --remote=gitlab +CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@mycompany/beta --all --remote=gitlab ``` ### Setting a default remote to your project (optional) @@ -150,7 +199,7 @@ CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=< If you'd like Conan to always use GitLab as the registry for your package, you can tell Conan to always reference the GitLab remote for a given package recipe: ```shell -conan remote add_ref Hello/0.1@my-group+my-project/beta gitlab +conan remote add_ref Hello/0.1@mycompany/beta gitlab ``` NOTE: **Note:** @@ -165,34 +214,19 @@ The rest of the example commands in this documentation assume that you've added ## Uploading a package -First you need to [create your Conan package locally](https://docs.conan.io/en/latest/creating_packages/getting_started.html). In order to work with the GitLab Package Registry, a specific [naming convention](#package-recipe-naming-convention) must be followed. +First you need to [create your Conan package locally](https://docs.conan.io/en/latest/creating_packages/getting_started.html). + +NOTE: **Note:** +If you are using the [instance level remote](#instance-level-remote), a specific [naming convention](#package-recipe-naming-convention-for-instance-level-remote) must be followed. Ensure you have a project created on GitLab and that the personal access token you're using has the correct permissions for write access to the container registry by selecting the `api` [scope](../../../user/profile/personal_access_tokens.md#limiting-scopes-of-a-personal-access-token). You can upload your package to the GitLab Package Registry using the `conan upload` command: ```shell -conan upload Hello/0.1@my-group+my-project/beta --all +conan upload Hello/0.1@mycompany/beta --all ``` -### Package recipe naming convention - -Standard Conan recipe convention looks like `package_name/version@user/channel`. - -**The recipe user must be the `+` separated project path**. The package -name may be anything, but it is preferred that the project name be used unless -it's not possible due to a naming collision. For example: - -| Project | Package | Supported | -| ---------------------------------- | ----------------------------------------------- | --------- | -| `foo/bar` | `my-package/1.0.0@foo+bar/stable` | Yes | -| `foo/bar-baz/buz` | `my-package/1.0.0@foo+bar-baz+buz/stable` | Yes | -| `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes | -| `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No | - -NOTE: **Note:** -A future iteration will extend support to [project and group level](https://gitlab.com/gitlab-org/gitlab/-/issues/11679) remotes which allows for more flexible naming conventions. - ## Installing a package Conan packages are commonly installed as dependencies using the `conanfile.txt` file. @@ -204,7 +238,7 @@ Add the Conan recipe to the `[requires]` section of the file: ```ini [requires] - Hello/0.1@my-group+my-project/beta + Hello/0.1@mycompany/beta [generators] cmake @@ -253,7 +287,7 @@ To search using a partial name, use the wildcard symbol `*`, which should be pla ```shell conan search Hello --all --remote=gitlab conan search He* --all --remote=gitlab -conan search Hello/0.1@my-group+my-project/beta --all --remote=gitlab +conan search Hello/0.1@mycompany/beta --all --remote=gitlab ``` The scope of your search includes all projects you have permission to access, this includes your private projects as well as all public projects. @@ -263,7 +297,7 @@ The scope of your search includes all projects you have permission to access, th The `conan info` command returns information about a given package: ```shell -conan info Hello/0.1@my-group+my-project/beta +conan info Hello/0.1@mycompany/beta ``` ## List of supported CLI commands diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index f46ad99e573..077666bc036 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -70,12 +70,12 @@ This view allows you to: - Filter image repositories by their name. - [Delete](#delete-images-from-within-gitlab) one or more image repository. - Navigate to the image repository details page. -- Show a **Quick start** dropdown with the most common commands to log in, build and push +- Show a **Quick start** dropdown with the most common commands to log in, build and push. - Show a banner if the optional [cleanup policy](#cleanup-policy) is enabled for this project. ### Control Container Registry for your group -Navigate to your groups's **{package}** **Packages & Registries > Container Registry**. +Navigate to your group's **{package}** **Packages & Registries > Container Registry**.  @@ -193,7 +193,7 @@ Before diving into the details, some things you should be aware of: longer, but it means you don’t get stuck without security patches for base images. - Doing an explicit `docker pull` before each `docker run` fetches the latest image that was just built. This is especially important if you are - using multiple Runners that cache images locally. Using the Git SHA in your + using multiple runners that cache images locally. Using the Git SHA in your image tag makes this less necessary since each job is unique and you shouldn't ever have a stale image. However, it's still possible to have a stale image if you re-build a given commit after a dependency has changed. @@ -240,8 +240,8 @@ There are three ways to authenticate to the Container Registry via ### Container Registry examples with GitLab CI/CD -If you're using Docker-in-Docker on your Runners, this is how your `.gitlab-ci.yml` -should look similar to this: +If you're using Docker-in-Docker on your runners, this is how your `.gitlab-ci.yml` +should look: ```yaml build: @@ -533,6 +533,11 @@ The cleanup policy: 1. Excludes from the list any tags matching the `name_regex_keep` value (tags to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. +CAUTION: **Warning:** +On GitLab.com, the execution time for the cleanup policy is limited, and some of the tags may remain in +the Container Registry after the policy runs. The next time the policy runs, the remaining tags are included, +so it may take multiple runs for all tags to be deleted. + ### Create a cleanup policy You can create a cleanup policy in [the API](#use-the-cleanup-policy-api) or the UI. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index f98a8eb9c6d..7329725a643 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -20,7 +20,7 @@ NOTE: **Note:** This option is available only if your GitLab administrator has [enabled support for the Maven repository](../../../administration/packages/index.md). -After the Packages feature is enabled, the Maven Repository will be available for +After the Packages feature is enabled, the Maven Repository is available for all new projects by default. To enable it for existing projects, or if you want to disable it: @@ -34,7 +34,7 @@ repository. ## Getting Started with Maven -This section will cover installing Maven and building a package. This is a +This section covers installing Maven and building a package. This is a quickstart to help if you're new to building Maven packages. If you're already using Maven and understand how to build your own packages, move onto the [next section](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -107,7 +107,7 @@ You should see a new directory where you ran this command matching your ## Getting started with Gradle -This section will cover installing Gradle and initializing a Java project. This is a +This section covers installing Gradle and initializing a Java project. This is a quickstart to help if you're new to Gradle. If you're already using Gradle and understand how to build your own packages, move onto the [next section](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -128,7 +128,7 @@ If you want to use an existing Gradle project, installation is not necessary. Simply execute `gradlew` (on Linux) or `gradlew.bat` (on Windows) in the project directory instead. -You should see something imilar to the below printed in the output: +You should see something similar to the below printed in the output: ```plaintext ------------------------------------------------------------ @@ -191,7 +191,7 @@ Select build script DSL: Enter selection (default: Groovy) [1..2] ``` -Choose `1` to create a new Java Library project which will be described in Groovy DSL. The output should be: +Choose `1` to create a new Java Library project which is described in Groovy DSL. The output should be: ```plaintext Select test framework: @@ -213,7 +213,7 @@ Enter a project name or hit enter to use the directory name as project name. The next step is to add the GitLab Package Registry as a Maven remote. If a project is private or you want to upload Maven artifacts to GitLab, -credentials will need to be provided for authorization too. Support is available +credentials must be provided for authorization too. Support is available for [personal access tokens](#authenticating-with-a-personal-access-token), [CI job tokens](#authenticating-with-a-ci-job-token), and [deploy tokens](../../project/deploy_tokens/index.md) only. Regular username/password @@ -388,7 +388,7 @@ repositories { To download and upload packages from GitLab, you need a `repository` and `distributionManagement` section in your `pom.xml` file. If you're following the -steps from above, then you'll need to add the following information to your +steps from above, then you must add the following information to your `my-project/pom.xml` file. Depending on your workflow and the amount of Maven packages you have, there are @@ -462,13 +462,13 @@ project's ID can be used for uploading. If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the group level endpoint for all your Maven packages stored within one GitLab group. Only packages you have access to -will be available for download. +are available for download. The group level endpoint works with any package names, which means the you have the flexibility of naming compared to [instance level endpoint](#instance-level-maven-endpoint). -However, GitLab will not guarantee the uniqueness of the package names within +However, GitLab does not guarantee the uniqueness of the package names within the group. You can have two projects with the same package name and package -version. As a result, GitLab will serve whichever one is more recent. +version. As a result, GitLab serves whichever one is more recent. The example below shows how the relevant `repository` section of your `pom.xml` would look like. You still need a project specific URL for uploading a package in @@ -524,7 +524,7 @@ For retrieving artifacts, you can use either the If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the instance level endpoint for -all maven packages stored in GitLab and the packages you have access to will be available +all maven packages stored in GitLab and the packages you have access to are available for download. Note that **only packages that have the same path as the project** are exposed via @@ -662,7 +662,7 @@ artifacts or even delete them. Installing a package from the GitLab Package Registry requires that you set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) -as above. Once this is completed, there are two ways for installaing a package. +as above. Once this is completed, there are two ways to install a package. ### Install using Maven with `mvn install` @@ -732,7 +732,7 @@ you can configure GitLab CI/CD to build new packages automatically. The example below shows how to create a new package each time the `master` branch is updated: -1. Create a `ci_settings.xml` file that will serve as Maven's `settings.xml` file. +1. Create a `ci_settings.xml` file that serves as Maven's `settings.xml` file. Add the server section with the same ID you defined in your `pom.xml` file. For example, in our case it's `gitlab-maven`: @@ -792,9 +792,9 @@ is updated: 1. Push those files to your repository. -The next time the `deploy` job runs, it will copy `ci_settings.xml` to the +The next time the `deploy` job runs, it copies `ci_settings.xml` to the user's home location (in this case the user is `root` since it runs in a -Docker container), and Maven will utilize the configured CI +Docker container), and Maven uses the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). ### Creating Maven packages with GitLab CI/CD using Gradle diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 5b90ec6f18c..2a1c12c2afd 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -308,7 +308,7 @@ stages: deploy: stage: deploy script: - - echo '//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}'>.npmrc + - echo "//gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc - npm publish ``` @@ -316,9 +316,9 @@ Learn more about [using `CI_JOB_TOKEN` to authenticate to the GitLab NPM registr ## Troubleshooting -### Error running yarn with NPM registry +### Error running Yarn with NPM registry -If you are using [yarn](https://classic.yarnpkg.com/en/) with the NPM registry, you may get +If you are using [Yarn](https://classic.yarnpkg.com/en/) with the NPM registry, you may get an error message like: ```shell @@ -377,6 +377,7 @@ NPM_TOKEN=<your_token> npm install ### `npm install` returns `npm ERR! 403 Forbidden` - Check that your token is not expired and has appropriate permissions. +- Check that [your token does not begin with `-`](https://gitlab.com/gitlab-org/gitlab/-/issues/235473). - Check if you have attempted to publish a package with a name that already exists within a given scope. - Ensure the scoped packages URL includes a trailing slash: - Correct: `//gitlab.com/api/v4/packages/npm/` diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index fd250c9ac95..9f954627b05 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -26,19 +26,19 @@ For information on how to create and upload a package, view the GitLab documenta ## Use GitLab CI/CD to build packages You can use [GitLab CI/CD](../../../ci/README.md) to build packages. -For Maven, NuGet and NPM packages, and Composer dependencies, you can +For Maven, NuGet, NPM, Conan, and PyPI packages, and Composer dependencies, you can authenticate with GitLab by using the `CI_JOB_TOKEN`. CI/CD templates, which you can use to get started, are in [this repo](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). -Learn more about [using CI/CD to build Maven packages](../maven_repository/index.md#creating-maven-packages-with-gitlab-cicd), [NPM packages](../npm_registry/index.md#publishing-a-package-with-cicd) and [NuGet Packages](../nuget_repository/index.md#publishing-a-nuget-package-with-cicd). +Learn more about [using CI/CD to build Maven packages](../maven_repository/index.md#creating-maven-packages-with-gitlab-cicd), [NPM packages](../npm_registry/index.md#publishing-a-package-with-cicd), [Composer packages](../composer_repository/index.md#publishing-the-package-with-cicd), [NuGet Packages](../nuget_repository/index.md#publishing-a-nuget-package-with-cicd), [Conan Packages](../conan_repository/index.md#using-gitlab-ci-with-conan-packages), and [PyPI packages](../pypi_repository/index.md#using-gitlab-ci-with-pypi-packages). If you use CI/CD to build a package, extended activity information is displayed when you view the package details:  -You can view which pipeline published the package, as well as the commit and +When using Maven and NPM, you can view which pipeline published the package, as well as the commit and user who triggered it. ## Download a package @@ -54,13 +54,11 @@ To download a package: You cannot edit a package after you publish it in the Package Registry. Instead, you must delete and recreate it. -- You cannot delete packages from the group view. You must delete them from the project view instead. - See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/227714) for details. -- You must have suitable [permissions](../../permissions.md). +To delete a package, you must have suitable [permissions](../../permissions.md). You can delete packages by using [the API](../../../api/packages.md#delete-a-project-package) or the UI. -To delete a package in the UI: +To delete a package in the UI, from your group or project: 1. Go to **{package}** **Packages & Registries > Package Registry**. 1. Find the name of the package you want to delete. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 63e6cd7b5b4..97f3f69d676 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -204,8 +204,27 @@ password = <deploy token> When uploading packages, note that: - The maximum allowed size is 50 Megabytes. -- If you upload the same package with the same version multiple times, each consecutive upload - is saved as a separate file. When installing a package, GitLab serves the most recent file. +- You cannot upload the same version of a package multiple times. If you try, you receive the error `Validation failed: File name has already been taken`. + +### Ensure your version string is valid + +If your version string (for example, `0.0.1`) is invalid, it will be rejected. GitLab uses the following regex to validate the version string. + +```ruby +\A(?: + v? + (?:([0-9]+)!)? (?# epoch) + ([0-9]+(?:\.[0-9]+)*) (?# release segment) + ([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release) + ((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release) + ([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release) + (?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version) +)\z}xi +``` + +You can play around with the regex and try your version strings on [this regular expression editor](https://rubular.com/r/FKM6d07ouoDaFV). + +For more details about the regex used, please check the [documentation here](https://www.python.org/dev/peps/pep-0440/#appendix-b-parsing-version-strings-with-regular-expressions)) ### Upload packages with Twine @@ -229,6 +248,13 @@ Uploading mypypipackage-0.0.1.tar.gz This indicates that the package was uploaded successfully. You can then navigate to your project's **Packages & Registries** page and see the uploaded packages. +If you would rather not use a `.pypirc` file to define your repository source, +you can upload to the repository with the authentication inline: + +```shell +TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.com/api/v4/projects/<project_id>/packages/pypi dist/* +``` + If you did not follow the guide above, then you need to ensure your package has been properly built and you [created a PyPi package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). @@ -273,3 +299,35 @@ Collecting mypypipackage Installing collected packages: mypypipackage Successfully installed mypypipackage-0.0.1 ``` + +## Using GitLab CI with PyPI packages + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. + +To work with PyPI commands within [GitLab CI/CD](./../../../ci/README.md), you can use +`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. + +For example: + +```yaml +image: python:latest + +run: + script: + - pip install twine + - python setup.py sdist bdist_wheel + - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/packages/pypi dist/* +``` + +You can also use `CI_JOB_TOKEN` in a `~/.pypirc` file that you check into GitLab: + +```ini +[distutils] +index-servers = + gitlab + +[gitlab] +repository = https://gitlab.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi +username = gitlab-ci-token +password = ${env.CI_JOB_TOKEN} +``` diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index a7bc4436d0e..571cda09e69 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -89,3 +89,10 @@ is created, you are ready to [upload your package](../conan_repository/index.md# ```shell CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> conan upload MyPackage/1.0.0@foo+bar+my-proj/channel --all --remote=gitlab ``` + +#### Composer + +It is currently not possible to publish a Composer package to a project that is different from where its code resides. + +If you attempt to publish a Composer package to a different project, you get a `404 Branch Not Found` +or `404 Tag Not Found` error. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index a89d534c782..e2baac1a962 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -55,7 +55,7 @@ The following table depicts the various user permission levels in a project. | View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | | View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View GitLab Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control-core) | ✓ | ✓ | ✓ | ✓ | ✓ | +| View GitLab Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control) | ✓ | ✓ | ✓ | ✓ | ✓ | | View wiki pages | ✓ | ✓ | ✓ | ✓ | ✓ | | See a list of jobs | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | See a job log | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | @@ -122,6 +122,7 @@ The following table depicts the various user permission levels in a project. | Manage Feature Flags **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | Run CI/CD pipeline against a protected branch | | | ✓ (*5*) | ✓ | ✓ | +| Request a CVE ID **(FREE ONLY)** | | | | ✓ | ✓ | | Use environment terminals | | | | ✓ | ✓ | | Run Web IDE's Interactive Web Terminals **(ULTIMATE ONLY)** | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | @@ -134,7 +135,7 @@ The following table depicts the various user permission levels in a project. | Share (invite) projects with groups | | | | ✓ (*8*) | ✓ (*8*)| | Add deploy keys to project | | | | ✓ | ✓ | | Configure project hooks | | | | ✓ | ✓ | -| Manage Runners | | | | ✓ | ✓ | +| Manage runners | | | | ✓ | ✓ | | Manage job triggers | | | | ✓ | ✓ | | Manage CI/CD variables | | | | ✓ | ✓ | | Manage GitLab Pages | | | | ✓ | ✓ | @@ -212,17 +213,14 @@ Find the current permissions on the Value Stream Analytics dashboard, as describ ### Issue Board permissions -Developers and users with higher permission level can use all -the functionality of the Issue Board, that is create/delete lists -and drag issues around. Read through the -[documentation on Issue Boards permissions](project/issue_board.md#permissions) -to learn more. +Find the current permissions for interacting with the Issue Board feature in the +[Issue Boards permissions page](project/issue_board.md#permissions). ### File Locking permissions **(PREMIUM)** The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. -Read through the documentation on [permissions for File Locking](project/file_lock.md#permissions-on-file-locking) to learn more. +Read through the documentation on [permissions for File Locking](project/file_lock.md#permissions) to learn more. ### Confidential Issues permissions @@ -250,7 +248,7 @@ group. | Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | | Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | -| Create project in group | | | ✓ (3) | ✓ (3) | ✓ (3) | +| Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | | Share (invite) groups with groups | | | | | ✓ | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | | Create/edit/delete iterations | | | ✓ | ✓ | ✓ | @@ -285,6 +283,7 @@ group. - The [instance level](admin_area/settings/visibility_and_access_controls.md#default-project-creation-protection). - The [group level](group/index.md#default-project-creation-level). 1. Does not apply to subgroups. +1. Developers can push commits to the default branch of a new project only if the [default branch protection](group/index.md#changing-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". ### Subgroup permissions @@ -368,7 +367,7 @@ are unable to browse the project's repository, for example). TIP: **Tip:** To prevent a guest user from creating projects, as an admin, you can edit the -user's profile to mark the user as [external](#external-users-core-only). +user's profile to mark the user as [external](#external-users). Beware though that even if a user is external, if they already have Reporter or higher permissions in any project or group, they are **not** counted as a free guest user. @@ -475,9 +474,9 @@ for details about the pipelines security model. ## LDAP users permissions Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. -Read through the documentation on [LDAP users permissions](group/index.md#manage-group-memberships-via-ldap-starter-only) to learn more. +Read through the documentation on [LDAP users permissions](group/index.md#manage-group-memberships-via-ldap) to learn more. ## Project aliases Project aliases can only be read, created and deleted by a GitLab administrator. -Read through the documentation on [Project aliases](../user/project/index.md#project-aliases-premium-only) to learn more. +Read through the documentation on [Project aliases](../user/project/index.md#project-aliases) to learn more. diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index 336c1b8f254..73a83b08a23 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -143,7 +143,9 @@ Users will be notified of the following events: | New SSH key added | User | Security email, always sent. | | New email added | User | Security email, always sent. | | Email changed | User | Security email, always sent. | -| Password changed | User | Security email, always sent. | +| Password changed | User | Security email, always sent when user changes their own password | +| Password changed by administrator | User | Security email, always sent when an administrator changes the password of another user | +| Two-factor authentication disabled | User | Security email, always sent. | | New user created | User | Sent on user creation, except for OmniAuth (LDAP)| | User added to project | User | Sent when user is added to project | | Project access level changed | User | Sent when user project access level is changed | @@ -183,6 +185,7 @@ To minimize the number of notifications that do not require any action, from [Gi | Close merge request | | | Reopen merge request | | | Merge merge request | | +| Merge when pipeline succeeds ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211961) in GitLab 13.4) | | | Change milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected | | Remove milestone merge request | Subscribers, participants mentioned, and Custom notification level with this event selected | | New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index ae73842dd98..1b8c16f401c 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -20,37 +20,19 @@ Personal access tokens expire on the date you define, at midnight UTC. - GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that expire in under seven days. The owners of these tokens are notified by email. - GitLab runs a check at 02:00 AM UTC every day to identify personal access tokens that expired on the current date. The owners of these tokens are notified by email. -To turn on the notification for expired personal access tokens in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-notification-for-expired-personal-access-token-core-only). **(CORE ONLY)** -- In GitLab Ultimate, administrators may [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only). -- In GitLab Ultimate, administrators may [toggle enforcement of personal access token expiry](../admin_area/settings/account_and_limit_settings.md#optional-enforcement-of-personal-access-token-expiry-ultimate-only). +- In GitLab Ultimate, administrators may [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens). +- In GitLab Ultimate, administrators may [toggle enforcement of personal access token expiry](../admin_area/settings/account_and_limit_settings.md#optional-enforcement-of-personal-access-token-expiry). For examples of how you can use a personal access token to authenticate with the API, see the following section from our [API Docs](../../api/README.md#personalproject-access-tokens). GitLab also offers [impersonation tokens](../../api/README.md#impersonation-tokens) which are created by administrators via the API. They're a great fit for automated authentication as a specific user. -## Enable or disable notification for Expired personal access token **(CORE ONLY)** - -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can enable it for your instance. - -To enable it: - -```ruby -Feature.enable(:expired_pat_email_notification) -``` - -To disable it: - -```ruby -Feature.disable(:expired_pat_email_notification) -``` - ## Creating a personal access token You can create as many personal access tokens as you like from your GitLab profile. -1. Log in to GitLab. +1. Sign in to GitLab. 1. In the upper-right corner, click your avatar and select **Settings**. 1. On the **User Settings** menu, select **Access Tokens**. 1. Choose a name and optional expiry date for the token. @@ -103,7 +85,7 @@ token.save! ``` This can be shortened into a single-line shell command using the -[GitLab Rails Runner](../../administration/troubleshooting/debug.md#using-the-rails-runner): +[Rails runner](../../administration/troubleshooting/debug.md#using-the-rails-runner): ```shell sudo gitlab-rails runner "token = User.find_by_username('automation-bot').personal_access_tokens.create(scopes: [:read_user, :read_repository], name: 'Automation token'); token.set_token('token-string-here123'); token.save!" @@ -131,7 +113,7 @@ token.revoke! ``` This can be shorted into a single-line shell command using the -[GitLab Rails Runner](../../administration/troubleshooting/debug.md#using-the-rails-runner): +[Rails runner](../../administration/troubleshooting/debug.md#using-the-rails-runner): ```shell sudo gitlab-rails runner "PersonalAccessToken.find_by_token('token-string-here123').revoke!" diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index b94ae958d3b..f84fc1ae898 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -114,7 +114,7 @@ You have 8 options here that you can use for your default dashboard view: - Your projects' activity - Starred projects' activity - Your groups -- Your [Todos](../todos.md) +- Your [to-dos](../todos.md) - Assigned Issues - Assigned Merge Requests - Operations Dashboard **(PREMIUM)** @@ -182,6 +182,12 @@ Manage the availability of integrated code intelligence features powered by Sourcegraph. View [the Sourcegraph feature documentation](../../integration/sourcegraph.md#enable-sourcegraph-in-user-preferences) for more information. +### Gitpod + +Enable and disable the [GitLab-Gitpod integration](../../integration/gitpod.md). This is only +visible after the integration is configured by a GitLab administrator. View +[the Gitpod feature documentation](../../integration/gitpod.md) for more information. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/bulk_editing.md b/doc/user/project/bulk_editing.md index c4a6aea807c..98584a939ea 100644 --- a/doc/user/project/bulk_editing.md +++ b/doc/user/project/bulk_editing.md @@ -14,6 +14,9 @@ For more details, see If you want to update attributes across multiple issues or merge requests, you can do it by bulk editing them, that is, editing them together. +NOTE: **Note:** +Only the items visible on the current page are selected for bulk editing (up to 20). +  ## Bulk edit issues at the project level diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 852baf1f628..afce3869cbf 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -48,9 +48,9 @@ Canary deployments require that you properly configure Deploy Boards: template for canary deployments that GitLab provides. Depending on the deploy, the label should be either `stable` or `canary`. -Usually, `stable` and blank or missing label means the same thing, and `canary` -or any other track means canary/temporary. -This allows GitLab to discover whether deployment is stable or canary (temporary). +GitLab assumes the track label is `stable` if the label is blank or missing. +Any other track label is considered `canary` (temporary). +This allows GitLab to discover whether a deployment is stable or canary (temporary). Once all of the above are set up and the pipeline has run at least once, navigate to the environments page under **Pipelines > Environments**. diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index d5713f20257..b3b1b51a543 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -141,7 +141,7 @@ To create and add a new Kubernetes cluster to your project, group, or instance: 1. Choose your cluster's settings: - **Kubernetes cluster name** - The name you wish to give the cluster. - **Environment scope** - The [associated environment](index.md#setting-the-environment-scope) to this cluster. - - **Kubernetes version** - The Kubernetes version to use. Currently the only version supported is 1.14. + - **Kubernetes version** - The [Kubernetes version](index.md#supported-cluster-versions) to use. - **Service role** - Select the **EKS IAM role** you created earlier to allow Amazon EKS and the Kubernetes control plane to manage AWS resources on your behalf. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index e4a750084c9..18d9fa67ee1 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -110,10 +110,10 @@ GitLab creates the following resources for ABAC clusters. | Environment namespace | `ServiceAccount` | Uses namespace of environment | Deploying to a cluster | | Environment namespace | `Secret` | Token for environment ServiceAccount | Deploying to a cluster | -### Security of GitLab Runners +### Security of runners -GitLab Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode) -enabled by default, which allows them to execute special commands and running +Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode) +enabled by default, which allows them to execute special commands and run Docker in Docker. This functionality is needed to run some of the [Auto DevOps](../../../topics/autodevops/index.md) jobs. This implies the containers are running in privileged mode and you should, @@ -124,14 +124,14 @@ turn can do almost everything that the host can do. Be aware of the inherent security risk associated with performing `docker run` operations on arbitrary images as they effectively have root access. -If you don't want to use GitLab Runner in privileged mode, either: +If you don't want to use a runner in privileged mode, either: -- Use shared Runners on GitLab.com. They don't have this security issue. -- Set up your own Runners using the configuration described at - [Shared Runners](../../gitlab_com/index.md#shared-runners). This involves: +- Use shared runners on GitLab.com. They don't have this security issue. +- Set up your own runners using the configuration described at + [shared runners](../../gitlab_com/index.md#shared-runners). This involves: 1. Making sure that you don't have it installed via [the applications](index.md#installing-applications). - 1. Installing a Runner + 1. Installing a runner [using `docker+machine`](https://docs.gitlab.com/runner/executors/docker_machine.html). ## Create new cluster @@ -206,7 +206,7 @@ To add a Kubernetes cluster to your project, group, or instance: apiVersion: v1 kind: ServiceAccount metadata: - name: gitlab-admin + name: gitlab namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 @@ -219,7 +219,7 @@ To add a Kubernetes cluster to your project, group, or instance: name: cluster-admin subjects: - kind: ServiceAccount - name: gitlab-admin + name: gitlab namespace: kube-system ``` @@ -245,23 +245,23 @@ To add a Kubernetes cluster to your project, group, or instance: Output: ```shell - serviceaccount "gitlab-admin" created + serviceaccount "gitlab" created clusterrolebinding "gitlab-admin" created ``` - 1. Retrieve the token for the `gitlab-admin` service account: + 1. Retrieve the token for the `gitlab` service account: ```shell - kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep gitlab-admin | awk '{print $1}') + kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep gitlab | awk '{print $1}') ``` Copy the `<authentication_token>` value from the output: ```yaml - Name: gitlab-admin-token-b5zv4 + Name: gitlab-token-b5zv4 Namespace: kube-system Labels: <none> - Annotations: kubernetes.io/service-account.name=gitlab-admin + Annotations: kubernetes.io/service-account.name=gitlab kubernetes.io/service-account.uid=bcfe66ac-39be-11e8-97e8-026dce96b6e8 Type: kubernetes.io/service-account-token diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 98078854050..8d188f00ceb 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -22,8 +22,8 @@ Using the GitLab project Kubernetes integration, you can: - Detect and [monitor Kubernetes](#monitoring-your-kubernetes-cluster). - Use it with [Auto DevOps](#auto-devops). - Use [Web terminals](#web-terminals). -- Use [Deploy Boards](#deploy-boards-premium). **(PREMIUM)** -- Use [Canary Deployments](#canary-deployments-premium). **(PREMIUM)** +- Use [Deploy Boards](#deploy-boards). **(PREMIUM)** +- Use [Canary Deployments](#canary-deployments). **(PREMIUM)** - View [Logs](#viewing-pod-logs). - Run serverless workloads on [Kubernetes with Knative](serverless/index.md). @@ -46,11 +46,11 @@ version. The range of supported versions is based on the evaluation of: Currently, GitLab supports the following Kubernetes versions: +- 1.17 - 1.16 - 1.15 - 1.14 - 1.13 (deprecated, support ends on November 22, 2020) -- 1.12 (deprecated, support ends on September 22, 2020) NOTE: **Note:** Some GitLab features may support versions outside the range provided here. diff --git a/doc/user/project/clusters/securing.md b/doc/user/project/clusters/securing.md index 5b9f776080b..a15660051f7 100644 --- a/doc/user/project/clusters/securing.md +++ b/doc/user/project/clusters/securing.md @@ -36,7 +36,7 @@ At a high level, the required steps include the following: Minimum requirements (depending on the GitLab Manage Application you want to install): - Your cluster is connected to GitLab (ModSecurity, Cilium, and Falco). -- At least one GitLab Runner is installed (Cilium and Falco only). +- At least one runner is installed (Cilium and Falco only). ### Understanding how GitLab Managed Apps are installed @@ -62,7 +62,7 @@ deployment logs. The Web Application Firewall feature uses this installation met However, the next generation of GitLab Managed Apps V2 ([CI/CD-based GitLab Managed Apps](https://gitlab.com/groups/gitlab-org/-/epics/2103)) don't use Sidekiq to deploy. All the applications are deployed using a GitLab CI/CD pipeline and -therefore GitLab Runners. +therefore, by runners. ```mermaid sequenceDiagram @@ -91,14 +91,14 @@ the Web Application Firewall from the project or group Kubernetes page. Note that your project doesn't have to be hosted or deployed through GitLab. You can manage a cluster independent of the applications that use the cluster. -## Set up a GitLab Runner +## Set up a runner -To install CI/CD-based GitLab Managed Apps, a pipeline using a GitLab Runner must be running in -GitLab. You can [install a GitLab Runner](../../clusters/applications.md#gitlab-runner) +To install CI/CD-based GitLab Managed Apps, a pipeline using a runner must be running in +GitLab. You can [install a runner](../../clusters/applications.md#gitlab-runner) in the Kubernetes cluster added in the previous step, or use one of the shared runners provided by GitLab if you're using GitLab.com. -With your cluster connected to GitLab and a GitLab Runner in place, you can proceed to the next +With your cluster connected to GitLab and a runner in place, you can proceed to the next steps and start installing the Cilium and Falco GitLab Managed Apps to secure your applications hosted on this cluster. diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 6af08b06294..1157c2c5632 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -52,7 +52,7 @@ To run Knative on GitLab, you will need: The simplest way to get started is to add a cluster using GitLab's [GKE integration](../add_remove_clusters.md). The set of minimum recommended cluster specifications to run Knative is 3 nodes, 6 vCPUs, and 22.50 GB memory. 1. **GitLab Runner:** A runner is required to run the CI jobs that will deploy serverless - applications or functions onto your cluster. You can install the GitLab Runner + applications or functions onto your cluster. You can install GitLab Runner onto the existing Kubernetes cluster. See [Installing Applications](../index.md#installing-applications) for more information. 1. **Domain Name:** Knative will provide its own load balancer using Istio. It will provide an external IP address or hostname for all the applications served by Knative. You will be prompted to enter a diff --git a/doc/user/project/code_intelligence.md b/doc/user/project/code_intelligence.md index be34053cdc7..f56673e69b7 100644 --- a/doc/user/project/code_intelligence.md +++ b/doc/user/project/code_intelligence.md @@ -26,10 +26,9 @@ Enable code intelligence for a project by adding a GitLab CI/CD job to the proje ```yaml code_navigation: - image: golang:1.14.0 + image: sourcegraph/lsif-go:v1 allow_failure: true # recommended script: - - go get github.com/sourcegraph/lsif-go/cmd/lsif-go - lsif-go artifacts: reports: @@ -40,38 +39,18 @@ The generated LSIF file must be less than 170MiB. After the job succeeds, code intelligence data can be viewed while browsing the code: - + ## Find references > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217392) in GitLab 13.2. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/225621) on GitLab 13.3. -> - It's enabled on GitLab.com. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235735) in GitLab 13.4. To find where a particular object is being used, you can see links to specific lines of code under the **References** tab:  -### Enable or disable find references - -Find references is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:code_navigation_references) -``` - -To enable it: - -```ruby -Feature.enable(:code_navigation_references) -``` - ## Language support Generating an LSIF file requires a language server indexer implementation for the diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index dbe3f3dc891..730a9ada428 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -9,7 +9,6 @@ type: reference > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6916) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.3. -> - [Support for group namespaces](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53182) added in GitLab Starter 12.1. > - Code Owners for Merge Request approvals was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4418) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.9. ## Introduction @@ -74,7 +73,7 @@ Once you've added Code Owners to a project, you can configure it to be used for merge request approvals: - As [merge request eligible approvers](merge_requests/merge_request_approvals.md#code-owners-as-eligible-approvers). -- As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium). **(PREMIUM)** +- As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners). **(PREMIUM)** NOTE: **Note:** Developer or higher [permissions](../permissions.md) are required in order to @@ -88,11 +87,11 @@ While the `CODEOWNERS` file can be used in addition to Merge Request [Approval R it can also be used as the sole driver of merge request approvals (without using [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules)). To do so, create the file in one of the three locations specified above and -set the code owners as required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium). +set the code owners as required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners). Use [the syntax of Code Owners files](code_owners.md#the-syntax-of-code-owners-files) to specify the actual owners and granular permissions. -Using Code Owners in conjunction with [Protected Branches](protected_branches.md#protected-branches-approval-by-code-owners-premium) +Using Code Owners in conjunction with [Protected Branches](protected_branches.md#protected-branches-approval-by-code-owners) will prevent any user who is not specified in the `CODEOWNERS` file from pushing changes for the specified files/paths, even if their role is included in the **Allowed to push** column. This allows for a more inclusive push strategy, as @@ -108,49 +107,58 @@ in the `.gitignore` file followed by one or more of: - A user's `@username`. - A user's email address. - The `@name` of one or more groups that should be owners of the file. +- Lines starting with `#` are escaped. -Groups must be added as [members of the project](members/index.md), -or they will be ignored. +The order in which the paths are defined is significant: the last pattern that +matches a given path will be used to find the code owners. -Starting in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/32432), -you can additionally specify groups or subgroups from the project's upper group -hierarchy as potential code owners, without having to invite them specifically -to the project. Groups outside the project's hierarchy or children beneath the -hierarchy must still be explicitly invited to the project in order to show as -Code Owners. +### Groups as Code Owners -For example, consider the following hierarchy for the example project -`example_project`: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53182) in GitLab Starter 12.1. +> - Group and subgroup hierarchy support was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32432) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.0. -```plaintext -group >> sub-group >> sub-subgroup >> example_project >> file.md -``` +Groups and subgroups members are inherited as eligible Code Owners to a +project, as long as the hierarchy is respected. + +For example, consider a given group called "Group X" (slug `group-x`) and a +"Subgroup Y" (slug `group-x/subgroup-y`) that belongs to the Group X, and +suppose you have a project called "Project A" within the group and a +"Project B" within the subgroup. -Any of the following groups would be eligible to be specified as code owners: +The eligible Code Owners to Project B are both the members of the Group X and +the Subgroup Y. And the eligible Code Owners to the Project A are just the +members of the Group X, given that Project A doesn't belong to the Subgroup Y: -- `@group` -- `@group/sub-group` -- `@group/sub-group/sub-subgroup` + -In addition, any groups that have been invited to the project using the -**Members** tool will also be recognized as eligible code owners. +But you have the option to [invite](members/share_project_with_groups.md) +the Subgroup Y to the Project A so that their members also become eligible +Code Owners: -The order in which the paths are defined is significant: the last -pattern that matches a given path will be used to find the code -owners. + -Starting a line with a `#` indicates a comment. This needs to be -escaped using `\#` to address files for which the name starts with a -`#`. +Once invited, any member (`@user`) of the group or subgroup can be set +as Code Owner to files of the Project A or B, as well as the entire Group X +(`@group-x`) or Subgroup Y (`@group-x/subgroup-y`), as exemplified below: + +```plaintext +# A member of the group or subgroup as Code Owner to a file +file.md @user + +# All group members as Code Owners to a file +file.md @group-x + +# All subgroup members as Code Owners to a file +file.md @group-x/subgroup-y + +# All group and subgroup members as Code Owners to a file +file.md @group-x @group-x/subgroup-y +``` ### Code Owners Sections **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. -> - It's deployed behind a feature flag, enabled by default. -> - It's enabled on GitLab.com. -> - It can be enabled or disabled per-project. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-code-owner-sections-core-only). **(CORE ONLY)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2 behind a feature flag, enabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Code Owner rules can be grouped into named sections. This allows for better organization of broader categories of Code Owner rules to be applied. @@ -212,28 +220,6 @@ the rules for "Groups" and "Documentation" sections:  -#### Enable or disable Code Owner Sections **(CORE ONLY)** - -Sections is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:sectional_codeowners) -``` - -To enable it: - -```ruby -Feature.enable(:sectional_codeowners) -``` - -CAUTION: **Caution:** -Disabling Sections will **not** refresh Code Owner Approval Rules on existing merge requests. - ## Example `CODEOWNERS` file ```plaintext diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 50fb24b555b..8146f39ef87 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -81,8 +81,8 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ [OpenShift docs](https://docs.openshift.com/container-platform/3.7/dev_guide/deployments/kubernetes_deployments.html#kubernetes-deployments-vs-deployment-configurations) and [GitLab issue #4584](https://gitlab.com/gitlab-org/gitlab/-/issues/4584). -1. [Configure GitLab Runner](../../ci/runners/README.md) with the [Docker](https://docs.gitlab.com/runner/executors/docker.html) or - [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html) executor. +1. [Configure GitLab Runner](../../ci/runners/README.md) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or + [`kubernetes`](https://docs.gitlab.com/runner/executors/kubernetes.html) executor. 1. Configure the [Kubernetes integration](clusters/index.md) in your project for the cluster. The Kubernetes namespace is of particular note as you will need it for your deployment scripts (exposed by the `KUBE_NAMESPACE` env variable). @@ -105,6 +105,8 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ re-deploy your application. If you are using Auto DevOps, this will be done automatically and no action is necessary. + If you are using GCP to manage clusters, you can see the deployment details in GCP itself by going to **Workloads > deployment name > Details**: +  Once all of the above are set up and the pipeline has run at least once, diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index ed80e5f6a32..6fd33901621 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -1,108 +1,230 @@ --- stage: Create group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers type: reference, howto --- -# File Locking **(PREMIUM)** +# File Locking **(CORE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in [GitLab Premium](https://about.gitlab.com/pricing/) 8.9. +Preventing wasted work caused by unresolvable merge conflicts requires +a different way of working. This means explicitly requesting write permissions, +and verifying no one else is editing the same file before you start. -Working with multiple people on the same file can be a risk. Conflicts when merging a non-text file are hard to overcome and will require a lot of manual work to resolve. File Locking helps you avoid these merge conflicts and better manage your binary files. +Although branching strategies usually work well enough for source code and +plain text because different versions can be merged together, they do not work +for binary files. -With File Locking, you can lock any file or directory, make your changes, and -then unlock it so another member of the team can edit it. +When file locking is setup, lockable files are **read only** by default. -## Overview +When a file is locked, only the user who locked the file may modify it. This +user is said to "hold the lock" or have "taken the lock", since only one user +can lock a file at a time. When a file or directory is unlocked, the user is +said to have "released the lock". -Working with multiple people on the same file can be a risk. Conflicts -when merging a non-text file are hard to overcome and will require a lot -of manual work to resolve. With GitLab Premium, File -Locking helps you avoid merge conflicts and better manage your binary -files by preventing everyone, except you, from modifying a specific file -or entire directory. +GitLab supports two different modes of file locking: -## Use-cases +- [Exclusive file locks](#exclusive-file-locks) for binary files: done **through + the command line** with Git LFS and `.gitattributes`, it prevents locked + files from being modified on any branch. **(CORE)** +- [Default branch locks](#default-branch-file-and-directory-locks): done + **through the GitLab UI**, it prevents locked files and directories being + modified on the default branch. **(PREMIUM)** -The file locking feature is useful in situations when: +## Permissions -- Multiple people are working on the same file and you want to avoid merge - conflicts. -- Your repository contains binary files in which situation there is no easy - way to tell the diff between yours and your colleagues' changes. -- Prevent design assets from being overwritten. +Locks can be created by any person who has at least +[Developer permissions](../permissions.md) to the repository. -Locked directories are locked recursively, which means that everything that -lies under them is also locked. +Only the user who locked the file or directory can edit locked files. Others +users will be prevented from modifying locked files by pushing, merging, +or any other means, and will be shown an error like: `The path '.gitignore' is +locked by Administrator`. -## Locking a file or a directory +## Exclusive file locks -NOTE: **Note:** -Locking only works for the default branch you have set in the project's settings -(usually `master`). +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/35856) in GitLab 10.5. -To lock a file: +This process allows you to lock single files or file extensions and it is +done through the command line. It doesn't require GitLab paid subscriptions. -1. Navigate to your project's **Repository > Files**. -1. Pick the file you want to lock. -1. Click the "Lock" button. +Git LFS is well known for tracking files to reduce the storage of +Git repositories, but it can also be user for [locking files](https://github.com/git-lfs/git-lfs/wiki/File-Locking). +This is the method used for Exclusive File Locks. -  +### Install Git LFS + +Before getting started, make sure you have [Git LFS installed](../../topics/git/lfs/index.md) in your computer. Open a terminal window and run: + +```shell +git-lfs --version +``` + +If it doesn't recognize this command, you'll have to install it. There are +several [installation methods](https://git-lfs.github.com/) that you can +choose according to your OS. To install it with Homebrew: + +```shell +brew install git-lfs +``` + +Once installed, **open your local repository in a terminal window** and +install Git LFS in your repo. If you're sure that LFS is already installed, +you can skip this step. If you're unsure, re-installing it won't do any harm: + +```shell +git lfs install +``` + +Check this document to learn more about [using Git LFS](../../topics/git/lfs/index.md#using-git-lfs). + +### Configure Exclusive File Locks -To lock an entire directory, look for the "Lock" link next to "History". +You need [Maintainer permissions](../permissions.md) to configure +Exclusive File Locks for your project through the command line. -After you lock a file or directory, it will appear as locked in the repository -view. +The first thing to do before using File Locking is to tell Git LFS which +kind of files are lockable. The following command will store PNG files +in LFS and flag them as lockable: - +```shell +git lfs track "*.png" --lockable +``` -Once locked, any merge request to the default branch will fail -to merge until the file becomes unlocked. +After executing the above command a file named `.gitattributes` will be +created or updated with the following content: -## Unlocking a file or a directory +```shell +*.png filter=lfs diff=lfs merge=lfs -text lockable +``` -To unlock a file or a directory, follow the same procedure as when you locked -them. For a detailed view of every existing lock, see the next section on -"Viewing and managing existing locks". +You can also register a file type as lockable without using LFS (to be able, for example, +to lock/unlock a file you need in a remote server that +implements the LFS File Locking API). To do that you can edit the +`.gitattributes` file manually: -You can unlock a file that yourself or someone else previously locked as long -as you have Maintainer or above [permissions](../permissions.md) to the project. +```shell +*.pdf lockable +``` -## Viewing and managing existing locks +The `.gitattributes` file is key to the process and **must** +be pushed to the remote repository for the changes to take effect. -To view or manage every existing lock, navigate to the -**Project > Repository > Locked Files** area. There, you can view all existing -locks and [remove the ones you have permission for](#permissions-on-file-locking). +After a file type has been registered as lockable, Git LFS will make +them read-only on the file system automatically. This means you will +need to **lock the file** before [editing it](#edit-lockable-files). -## Permissions on file locking +### Lock files -The user that locks a file or directory **is the only one** that can edit and -push their changes back to the repository where the locked objects are located. +By locking a file, you verify that no one else is editing it, and +prevent anyone else from editing the file until you’re done. On the other +hand, when you unlock a file, you communicate that you've finished editing +and allow other people to edit it. -Locks can be created by any person who has [push access](../permissions.md) to the repository; i.e., -Developer and higher level, and can be removed solely by their author and any -user with Maintainer permissions and above. +To lock or unlock a file with Exclusive File Locking, open a terminal window +in your repository directory and run the commands as described below. -If a file is locked and you are not the author of its locked state, a -pre-receive hook will reject your changes when you try to push. In the -following example, a user who has no permissions on the locked `.gitignore` -file will see the message below: +To **lock** a file: ```shell -Counting objects: 3, done. -Delta compression using up to 4 threads. -Compressing objects: 100% (3/3), done. -Writing objects: 100% (3/3), 320 bytes | 0 bytes/s, done. -Total 3 (delta 1), reused 0 (delta 0) -remote: GitLab: The path '.gitignore' is locked by Administrator -To https://example.com/gitlab-org/gitlab-foss.git - ! [remote rejected] master -> master (pre-receive hook declined) - error: failed to push some refs to 'https://example.com/gitlab-org/gitlab-foss.git' +git lfs lock path/to/file.png ``` -Similarly, when a user that is not the author of the locked state of a file -accepts a merge request, an error message will appear stating that the file -is locked. +To **unlock** a file: + +```shell +git lfs unlock path/to/file.png +``` + +You can also unlock by file ID (given by LFS when you [view locked files](#view-exclusively-locked-files)): + +```shell +git lfs unlock --id=123 +``` + +If for some reason you need to unlock a file that was not locked by +yourself, you can use the `--force` flag as long as you have **Maintainer** +permissions to the project: + +```shell +git lfs unlock --id=123 --force +``` + +You can normally push files to GitLab whether they're locked or unlocked. + +NOTE: **Note:** +Although multi-branch file locks can be created and managed through the Git LFS +command line interface, file locks can be created for any file. + +### View exclusively-locked files + +To list all the files locked with LFS locally, open a terminal window in your +repo and run: + +```shell +git lfs locks +``` + +The output lists the locked files followed by the user who locked each of them +and the files' IDs. + +On the repository file tree, GitLab will display an LFS badge for files +tracked by Git LFS plus a padlock icon on exclusively-locked files: + + + +You can also [view and remove existing locks](#view-and-remove-existing-locks) from the GitLab UI. + +NOTE: **Note:** +When you rename an exclusively-locked file, the lock is lost. You'll have to +lock it again to keep it locked. + +### Edit lockable files + +Once the file is [configured as lockable](#configure-exclusive-file-locks), it is set to read-only. +Therefore, you need to lock it before editing it. + +Suggested workflow for shared projects: + +1. Lock the file. +1. Edit the file. +1. Commit your changes. +1. Push to the repo. +1. Get your changes reviewed, approved, and merged. +1. Unlock the file. + +## Default branch file and directory locks **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in GitLab Enterprise Edition 8.9. Available in [GitLab Premium](https://about.gitlab.com/pricing/). + +This process allows you to lock one file at a time through the GitLab UI and +requires access to [GitLab Premium, GitLab.com Silver](https://about.gitlab.com/pricing/), or higher tiers. + +Default branch file and directory locks only apply to the default branch set in +the project's settings (usually `master`). + +Changes to locked files on the default branch will be blocked, including merge +requests that modify locked files. Unlock the file to allow changes. + +### Lock a file or a directory + +To lock a file: + +1. Open the file or directory in GitLab. +1. Click the **Lock** button, located near the Web IDE button. + +  + +An **Unlock** button will be displayed if the file is already locked, and +will be disabled if you do not have permission to unlock the file. + +If you did not lock the file, hovering your cursor over the button will show +who locked the file. + +### View and remove existing locks + +The **Locked Files**, accessed from **Project > Repository** left menu, lists +all file and directory locks. Locks can be removed by their author, or any user +with Maintainer permissions and above. - +This list shows all the files locked either through LFS or GitLab UI. diff --git a/doc/user/project/img/code_intelligence_v13_1.png b/doc/user/project/img/code_intelligence_v13_1.png Binary files differdeleted file mode 100644 index 744195caed2..00000000000 --- a/doc/user/project/img/code_intelligence_v13_1.png +++ /dev/null diff --git a/doc/user/project/img/code_intelligence_v13_4.png b/doc/user/project/img/code_intelligence_v13_4.png Binary files differnew file mode 100644 index 00000000000..bbb6ce67bcc --- /dev/null +++ b/doc/user/project/img/code_intelligence_v13_4.png diff --git a/doc/user/project/img/code_owners_invite_members_v13_4.png b/doc/user/project/img/code_owners_invite_members_v13_4.png Binary files differnew file mode 100644 index 00000000000..852a5f68b36 --- /dev/null +++ b/doc/user/project/img/code_owners_invite_members_v13_4.png diff --git a/doc/user/project/img/code_owners_members_v13_4.png b/doc/user/project/img/code_owners_members_v13_4.png Binary files differnew file mode 100644 index 00000000000..e37fe72cd6e --- /dev/null +++ b/doc/user/project/img/code_owners_members_v13_4.png diff --git a/doc/user/project/img/file_lock_merge_request_error_message.png b/doc/user/project/img/file_lock_merge_request_error_message.png Binary files differdeleted file mode 100644 index 64bcc86ac0d..00000000000 --- a/doc/user/project/img/file_lock_merge_request_error_message.png +++ /dev/null diff --git a/doc/user/project/img/file_lock_repository_view.png b/doc/user/project/img/file_lock_repository_view.png Binary files differdeleted file mode 100644 index ced14198da9..00000000000 --- a/doc/user/project/img/file_lock_repository_view.png +++ /dev/null diff --git a/doc/user/project/img/lfs_locked_files_v13_2.png b/doc/user/project/img/lfs_locked_files_v13_2.png Binary files differnew file mode 100644 index 00000000000..0c31ce979de --- /dev/null +++ b/doc/user/project/img/lfs_locked_files_v13_2.png diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index 56266718d12..89130d5822f 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -76,3 +76,6 @@ If you've accidentally started the import process with the wrong account, follow 1. Revoke GitLab access to your Bitbucket account, essentially reversing the process in the following procedure: [Import your Bitbucket repositories](#import-your-bitbucket-repositories). 1. Sign out of the Bitbucket account. Follow the procedure linked from the previous step. + +NOTE: **Note:** +To import a repository including LFS objects from a Bitbucket server repository, use the [Repo by URL](../import/repo_by_url.md) importer. diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index f0e730564d8..d0499730bfe 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -37,7 +37,12 @@ Import your projects from Bitbucket Server to GitLab with minimal effort. empty changes. 1. Attachments in Markdown are currently not imported. 1. Task lists are not imported. -1. Emoji reactions are not imported +1. Emoji reactions are not imported. +1. [LFS objects](../../../topics/git/lfs/index.md) are not imported. + + NOTE: **Note:** + To import a repository including LFS objects from a Bitbucket server repository, use the [Repo by URL](../import/repo_by_url.md) importer. + 1. Project filtering does not support fuzzy search (only `starts with` or `full match strings` are currently supported) @@ -62,6 +67,25 @@ The importer will create any new namespaces (groups) if they don't exist or in the case the namespace is taken, the repository will be imported under the user's namespace that started the import process. +#### User assignment by username + +Alternatively, user assignment by username is available behind a `bitbucket_server_user_mapping_by_username` feature flag. +The importer will try to find a user in the GitLab user database using author's `username` or `slug` or `displayName`. +Falls back to author's `email` if user is not found by username. +Similarly to user assignment by email, if no such user is available, the project creator is set as the author. + +To enable or disable user assignment by username: + +Start a [Rails console](../../../administration/troubleshooting/debug.md#starting-a-rails-console-session). + +```ruby +# Enable +Feature.enable(:bitbucket_server_user_mapping_by_username) + +# Disable +Feature.disable(:bitbucket_server_user_mapping_by_username) +``` + ## Importing your Bitbucket repositories 1. Sign in to GitLab and go to your dashboard. @@ -83,4 +107,9 @@ namespace that started the import process. ## Troubleshooting +If the GUI-based import tool does not work, you can try to: + +- Use the [GitLab Import API](../../../api/import.md#import-repository-from-bitbucket-server) Bitbucket server endpoint. +- Set up [Repository Mirroring](../repository/repository_mirroring.md), which provides verbose error output. + See the [troubleshooting](bitbucket.md#troubleshooting) section for [Bitbucket](bitbucket.md). diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index d2e79458526..2957b33c20e 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -25,10 +25,10 @@ The following list illustrates the main differences between CVS and Git: are not atomic. If an operation on the repository is interrupted in the middle, the repository can be left in an inconsistent state. - **Storage method.** Changes in CVS are per file (changeset), while in Git - a committed file(s) is stored in its entirety (snapshot). That means that's + a committed file(s) is stored in its entirety (snapshot). That means it's very easy in Git to revert or undo a whole change. - **Revision IDs.** The fact that in CVS changes are per files, the revision ID - is depicted by version numbers, for example `1.4` reflects how many time a + is depicted by version numbers, for example `1.4` reflects how many times a given file has been changed. In Git, each version of a project as a whole (each commit) has its unique name given by SHA-1. - **Merge tracking.** Git uses a commit-before-merge approach rather than @@ -54,7 +54,7 @@ Wikipedia article on [comparing the different version control software](https:// CVS is old with no new release since 2008. Git provides more tools to work with (`git bisect` for one) which makes for a more productive workflow. -Migrating to Git/GitLab there is: +Migrating to Git/GitLab will benefit you: - **Shorter learning curve**, Git has a big community and a vast number of tutorials to get you started (see our [Git topic](../../../topics/git/index.md)). diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index 3838289aec4..f21ec26bdef 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -83,7 +83,7 @@ back to both GitLab and GitHub when completed.  - Your project is now mirrored on GitLab, where the Runners will be able to access + Your project is now mirrored on GitLab, where the runners will be able to access your source code and run your tests. Optional step: If you set this up on GitLab.com, make sure the project is @@ -105,7 +105,7 @@ back to both GitLab and GitHub when completed. 1. The result of the job will be visible directly from the pipeline view: -  +  NOTE: **Note:** If you don't commit very often to your project, you may want to use diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 531b308111a..4cd0c9e02c7 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -12,18 +12,6 @@ your self-managed GitLab instance. ## Overview -NOTE: **Note:** -These instructions work for users on GitLab.com, but if you are an -administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, -you must enable [GitHub integration](../../../integration/github.md). GitHub integration is the only method for -importing from GitHub Enterprise. If you are using GitLab.com, you can alternatively import -GitHub repositories using a [personal access token](#using-a-github-token), -but this method is not recommended because it cannot associate all user activity -(such as issues and pull requests) with matching GitLab users. -If you are an administrator of a self-managed GitLab instance, you can also use the -[GitHub Rake task](../../../administration/raketasks/github_import.md) to import projects from -GitHub without the constraints of a Sidekiq worker. - The following aspects of a project are imported: - Repository description (GitLab.com & 7.7+) @@ -36,12 +24,37 @@ The following aspects of a project are imported: - Release note descriptions (GitLab.com & 8.12+) - Pull request review comments (GitLab.com & 10.2+) - Regular issue and pull request comments +- [Git Large File Storage (LFS) Objects](../../../topics/git/lfs/index.md) References to pull requests and issues are preserved (GitLab.com & 8.7+), and each imported repository maintains visibility level unless that [visibility level is restricted](../../../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects), in which case it defaults to the default project visibility. +The namespace is a user or group in GitLab, such as `gitlab.com/janedoe` or `gitlab.com/customer-success`. You can do some bulk actions to move projects to different namespaces in the rails console. + +This process does not migrate or import any types of groups or organizations from GitHub to GitLab. + +### If you're using GitLab.com + +If you're using GitLab.com, you can alternatively import +GitHub repositories using a [personal access token](#using-a-github-token), +but we don't recommend this method because it can't associate all user activity +(such as issues and pull requests) with matching GitLab users. + +### If you're importing from GitLab Enterprise + +If you're importing from GitHub Enterprise, you must enable [GitHub integration][gh-import]. + +### If you're using a self-managed GitLab instance + +If you're an administrator of a self-managed GitLab instance, you must enable +[GitHub integration][gh-import]. + +If you're an administrator of a self-managed GitLab instance, you can also use the +[GitHub Rake task](../../../administration/raketasks/github_import.md) to import projects from +GitHub without the constraints of a Sidekiq worker. + ## How it works When issues and pull requests are being imported, the importer attempts to find their GitHub authors and @@ -135,8 +148,8 @@ your GitHub repositories are listed. ## Mirroring and pipeline status sharing -Depending your GitLab tier, [project mirroring](../repository/repository_mirroring.md) can be set up to keep -your imported project in sync with its GitHub copy. +Depending on your GitLab tier, [repository mirroring](../repository/repository_mirroring.md) can be set up to keep +your imported repository in sync with its GitHub copy. Additionally, you can configure GitLab to send pipeline status updates back GitHub with the [GitHub Project Integration](../integrations/github.md). **(PREMIUM)** diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 4e5b924a1b7..c79f2be1d3f 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -37,6 +37,8 @@ When you create a project in GitLab, you'll have access to a large number of - [Signing commits](gpg_signed_commits/index.md): use GPG to sign your commits - [Deploy tokens](deploy_tokens/index.md): Manage project-based deploy tokens that allow permanent access to the repository and Container Registry. - [Web IDE](web_ide/index.md) +- [CVE ID Requests](../application_security/cve_id_request.md): Request a CVE identifier to track a + vulnerability in your project. **Issues and merge requests:** @@ -192,12 +194,16 @@ To delete a project, first navigate to the home page for that project. ### Delayed deletion **(PREMIUM)** -By default, clicking to delete a project is followed by a seven day delay. Admins can restore the project during this period of time. -This delay [may be changed by an admin](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay-premium-only). +By default, projects in a personal namespace are deleted after a seven day delay. + +Admins can restore the project during this period of time. +This delay [may be changed by an admin](../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). Admins can view all projects pending deletion. If you're an administrator, go to the top navigation bar, click **Projects > Your projects**, and then select the **Deleted projects** tab. From this tab an admin can restore any project. +For information on delay deletion of projects within a group, please see [Enabling delayed Project removal](../group/index.md#enabling-delayed-project-removal) + ## CI/CD for external repositories **(PREMIUM)** Instead of importing a repository directly to GitLab, you can connect your repository @@ -318,7 +324,7 @@ through Git. For example: ```plaintext -machine example.gitlab.com +machine gitlab.example.com login <gitlab_user_name> password <personal_access_token> ``` diff --git a/doc/user/project/integrations/ewm.md b/doc/user/project/integrations/ewm.md new file mode 100644 index 00000000000..be89323a246 --- /dev/null +++ b/doc/user/project/integrations/ewm.md @@ -0,0 +1,30 @@ +# IBM Engineering Workflow Management (EWM) Integration **(CORE)** + +This service allows you to navigate from GitLab to EWM work items mentioned in merge request descriptions and commit messages. Each work item reference is automatically converted to a link back to the work item. + +NOTE: **Note:** +This IBM product was [formerly named Rational Team Concert](https://jazz.net/blog/index.php/2019/04/23/renaming-the-ibm-continuous-engineering-portfolio/)(RTC). This integration is also compatible with all versions of RTC and EWM. + +1. From a GitLab project, navigate to **Settings > Integrations**, and then click **EWM**. +1. Enter the information listed below. + + | Field | Description | + | ----- | ----------- | + | `project_url` | URL of the EWM project area to link to the GitLab project. To obtain your project area URL, navigate to the path `/ccm/web/projects` and copy the listed project's URL. For example, `https://example.com/ccm/web/Example%20Project` | + | `issues_url` | URL to the work item editor in the EWM project area. The format is `<your-server-url>/resource/itemName/com.ibm.team.workitem.WorkItem/:id`. For example, `https://example.com/ccm/resource/itemName/com.ibm.team.workitem.WorkItem/:id` | + | `new_issue_url` | URL to create a new work item in the EWM project area. Append the following fragment to your project area URL: `#action=com.ibm.team.workitem.newWorkItem`. For example, `https://example.com/ccm/web/projects/JKE%20Banking#action=com.ibm.team.workitem.newWorkItem` | + +## Reference EWM work items in commit messages + +You can use any of the keywords supported by the EWM Git Integration Toolkit to refer to work items. Work items can be referenced using the format: `<keyword> <id>`. + +You can use the following keywords: + +- `bug` +- `task` +- `defect` +- `rtcwi` +- `workitem` +- `work item` + +For more details, see the EWM documentation page [Creating links from commit comments](https://www.ibm.com/support/knowledgecenter/SSYMRC_7.0.0/com.ibm.team.connector.cq.doc/topics/t_creating_links_through_comments.html), which recommends against using the additionally-supported keyword `#` because of incompatibility with GitLab. diff --git a/doc/user/project/integrations/generic_alerts.md b/doc/user/project/integrations/generic_alerts.md index dc6aa40ea82..0e8e082859b 100644 --- a/doc/user/project/integrations/generic_alerts.md +++ b/doc/user/project/integrations/generic_alerts.md @@ -1,124 +1,5 @@ --- -stage: Monitor -group: Health -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: '../../../operations/incident_management/generic_alerts.md' --- -# Generic alerts integration - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13203) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.4. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.8. - -GitLab can accept alerts from any source via a generic webhook receiver. -When you set up the generic alerts integration, a unique endpoint will -be created which can receive a payload in JSON format, and will in turn -create an issue with the payload in the body of the issue. You can always -[customize the payload](#customizing-the-payload) to your liking. - -The entire payload will be posted in the issue discussion as a comment -authored by the GitLab Alert Bot. - -NOTE: **Note:** -In GitLab versions 13.1 and greater, you can configure -[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances) -to use this endpoint. - -## Setting up generic alerts - -To obtain credentials for setting up a generic alerts integration: - -- Sign in to GitLab as a user with maintainer [permissions](../../permissions.md) for a project. -- Navigate to the **Operations** page for your project, depending on your installed version of GitLab: - - *In GitLab versions 13.1 and greater,* navigate to **Settings > Operations** in your project. - - *In GitLab versions prior to 13.1,* navigate to **Settings > Integrations** in your project. GitLab will display a banner encouraging you to enable the Alerts endpoint in **Settings > Operations** instead. -- Click **Alerts endpoint**. -- Toggle the **Active** alert setting to display the **URL** and **Authorization Key** for the webhook configuration. - -## Customizing the payload - -You can customize the payload by sending the following parameters. All fields other than `title` are optional: - -| Property | Type | Description | -| -------- | ---- | ----------- | -| `title` | String | The title of the incident. Required. | -| `description` | String | A high-level summary of the problem. | -| `start_time` | DateTime | The time of the incident. If none is provided, a timestamp of the issue will be used. | -| `service` | String | The affected service. | -| `monitoring_tool` | String | The name of the associated monitoring tool. | -| `hosts` | String or Array | One or more hosts, as to where this incident occurred. | -| `severity` | String | The severity of the alert. Must be one of `critical`, `high`, `medium`, `low`, `info`, `unknown`. Default is `critical`. | -| `fingerprint` | String or Array | The unique identifier of the alert. This can be used to group occurrences of the same alert. | - -You can also add custom fields to the alert's payload. The values of extra parameters -are not limited to primitive types, such as strings or numbers, but can be a nested -JSON object. For example: - -```json -{ "foo": { "bar": { "baz": 42 } } } -``` - -TIP: **Payload size:** -Ensure your requests are smaller than the [payload application limits](../../../administration/instance_limits.md#generic-alert-json-payloads). - -Example request: - -```shell -curl --request POST \ - --data '{"title": "Incident title"}' \ - --header "Authorization: Bearer <authorization_key>" \ - --header "Content-Type: application/json" \ - <url> -``` - -The `<authorization_key>` and `<url>` values can be found when [setting up generic alerts](#setting-up-generic-alerts). - -Example payload: - -```json -{ - "title": "Incident title", - "description": "Short description of the incident", - "start_time": "2019-09-12T06:00:55Z", - "service": "service affected", - "monitoring_tool": "value", - "hosts": "value", - "severity": "high", - "fingerprint": "d19381d4e8ebca87b55cda6e8eee7385", - "foo": { - "bar": { - "baz": 42 - } - } -} -``` - -## Triggering test alerts - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3066) in GitLab Core in 13.2. - -After a [project maintainer or owner](#setting-up-generic-alerts) -[configures generic alerts](#setting-up-generic-alerts), you can trigger a -test alert to confirm your integration works properly. - -1. Sign in as a user with Developer or greater [permissions](../../../user/permissions.md). -1. Navigate to **Settings > Operations** in your project. -1. Click **Alerts endpoint** to expand the section. -1. Enter a sample payload in **Alert test payload** (valid JSON is required). -1. Click **Test alert payload**. - -GitLab displays an error or success message, depending on the outcome of your test. - -## Automatic grouping of identical alerts **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214557) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. - -In GitLab versions 13.2 and greater, GitLab groups alerts based on their payload. -When an incoming alert contains the same payload as another alert (excluding the -`start_time` and `hosts` attributes), GitLab groups these alerts together and -displays a counter on the -[Alert Management List](../../../operations/incident_management/incidents.md) -and details pages. - -If the existing alert is already `resolved`, then a new alert will be created instead. - - +This document was moved to [another location](../../../operations/incident_management/generic_alerts.md). diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index f2e769dcfc0..443ca11be27 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -53,7 +53,7 @@ Irker accepts channel names of the form `chan` and `#chan`, both for the case, `Aorimn` is treated as a nick and no more as a channel name. Irker can also join password-protected channels. Users need to append -`?key=thesecretpassword` to the chan name. When using this feature remember to +`?key=thesecretpassword` to the channel name. When using this feature remember to **not** put the `#` sign in front of the channel name; failing to do so will result on irker joining a channel literally named `#chan?key=password` henceforth leaking the channel key through the `/whois` IRC command (depending on IRC server diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index f11cd4d9539..3e0b6492477 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Jira integration -If you need to use Jira to track work that's implemented in GitLab, GitLab's Jira integrations make the process of working across systems more efficent. +If you need to use Jira to track work that's implemented in GitLab, GitLab's Jira integrations make the process of working across systems more efficient. This page is about the GitLab Jira integration, which is available in every GitLab project by default, allowing you to connect it to any Jira instance, whether Cloud or self-managed. To compare features with the complementary Jira Development Panel integration, see [Jira integrations](jira_integrations.md). @@ -22,7 +22,9 @@ Features include: - The Jira issue shows the activity and is closed or otherwise transitioned as specified in your GitLab settings. - **View a list of Jira issues directly in GitLab** **(PREMIUM)** -For additional features, you can install the [Jira Development Panel integration](../../../integration/jira_development_panel.md). This enables you to: +For additional features, you can install the +[Jira Development Panel integration](../../../integration/jira_development_panel.md) **(PREMIUM)**. +This enables you to: - In a Jira issue, display relevant GitLab information in the [development panel](https://support.atlassian.com/jira-software-cloud/docs/view-development-information-for-an-issue/), including related branches, commits, and merge requests. - Use Jira [Smart Commits](https://confluence.atlassian.com/fisheye/using-smart-commits-960155400.html) in GitLab to add Jira comments, log time spent on the issue, or apply any issue transition. diff --git a/doc/user/project/integrations/jira_integrations.md b/doc/user/project/integrations/jira_integrations.md index 90cd9bf3acb..dd22c26be36 100644 --- a/doc/user/project/integrations/jira_integrations.md +++ b/doc/user/project/integrations/jira_integrations.md @@ -18,7 +18,7 @@ Although you can [migrate](../../../user/project/import/jira.md) your Jira issue The following Jira integrations allow different types of cross-referencing between GitLab activity and Jira issues, with additional features: - [**Jira integration**](jira.md) - This is built in to GitLab. In a given GitLab project, it can be configured to connect to any Jira instance, self-managed or Cloud. -- [**Jira development panel integration**](../../../integration/jira_development_panel.md) **(PREMIUM)** - This connects all GitLab projects under a specified group or personal namespace. +- [**Jira development panel integration**](../../../integration/jira_development_panel.md) - This connects all GitLab projects under a specified group or personal namespace. - If you're using Jira Cloud and GitLab.com, install the [GitLab for Jira](https://marketplace.atlassian.com/apps/1221011/gitlab-for-jira) app in the Atlassian Marketplace and see its [documentation](../../../integration/jira_development_panel.md#gitlab-for-jira-app). - For all other environments, use the [Jira DVCS Connector configuration instructions](../../../integration/jira_development_panel.md#configuration). diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index f179cd6b98e..7a1f757c138 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [Emails on push](emails_on_push.md) | Email the commits and diff of each push to a list of recipients | No | | External Wiki | Replaces the link to the internal wiki with a link to an external wiki | No | | Flowdock | Flowdock is a collaboration web app for technical teams | No | -| [Generic alerts](generic_alerts.md) **(ULTIMATE)** | Receive alerts on GitLab from any source | No | +| [Generic alerts](../../../operations/incident_management/generic_alerts.md) **(ULTIMATE)** | Receive alerts on GitLab from any source | No | | [GitHub](github.md) **(PREMIUM)** | Sends pipeline notifications to GitHub | No | | [Hangouts Chat](hangouts_chat.md) | Receive events notifications in Google Hangouts Chat | No | | [HipChat](hipchat.md) | Private group chat and IM | No | @@ -59,6 +59,7 @@ Click on the service links to see further configuration instructions and details | [Prometheus](prometheus.md) | Monitor the performance of your deployed apps | No | | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | No | | [Redmine](redmine.md) | Redmine issue tracker | No | +| [EWM](ewm.md) | EWM work item tracker | No | | [Unify Circuit](unify_circuit.md) | Receive events notifications in Unify Circuit | No | | [Webex Teams](webex_teams.md) | Receive events notifications in Webex Teams | No | | [YouTrack](youtrack.md) | YouTrack issue tracker | No | @@ -82,9 +83,9 @@ Read more about [Service templates](services_templates.md). ## Project integration management -Project integraton management lets you control integration settings across all projects +Project integration management lets you control integration settings across all projects of an instance. On the project level, administrators you can choose whether to inherit the -instance configuraton or provide custom settings. +instance configuration or provide custom settings. Read more about [Project integration management](../../admin_area/settings/project_integration_management.md). diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index eda6f64ccac..0d3042463c9 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -18,6 +18,8 @@ The [Prometheus service](../prometheus.md) must be enabled. NGINX server metrics are detected, which tracks the pages and content directly served by NGINX. +[`environment_filter`](../../../../operations/metrics/dashboards/variables.md#environment_filter) is one of the predefined variables that metrics dashboards support. + | Name | Query | | ---- | ----- | | Throughput (req/sec) | `sum(rate(nginx_server_requests{server_zone!="*", server_zone!="_", %{environment_filter}}[2m])) by (code)` | diff --git a/doc/user/project/integrations/servicenow.md b/doc/user/project/integrations/servicenow.md new file mode 100644 index 00000000000..d549921b9d9 --- /dev/null +++ b/doc/user/project/integrations/servicenow.md @@ -0,0 +1,41 @@ +--- +stage: Create +group: Ecosystem +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# ServiceNow integration + +ServiceNow offers several integrations to help centralize and automate your +management of GitLab workflows. + +## GitLab spoke + +With the GitLab spoke in ServiceNow, you can automate actions for GitLab +projects, groups, users, issues, merge requests, branches, and repositories. + +For a full list of features, see the +[GitLab spoke documentation](https://docs.servicenow.com/bundle/orlando-servicenow-platform/page/administer/integrationhub-store-spokes/concept/gitlab-spoke.html). + +You must [configure GitLab as an OAuth2 authentication service provider](../../../integration/oauth_provider.md), +which involves creating an application and then providing the Application ID +and Secret in ServiceNow. + +## GitLab SCM and Continuous Integration for DevOps + +In ServiceNow DevOps, you can integrate with GitLab repositories and GitLab CI/CD +to centralize your view of GitLab activity and your change management processes. +You can: + +- Track information about activity in GitLab repositories and CI/CD pipelines in + ServiceNow. +- Integrate with GitLab CI/CD pipelines, by automating the creation of change + tickets and determining criteria for changes to auto-approve. + +For more information, refer to the following ServiceNow resources: + +- [ServiceNow DevOps home page](https://www.servicenow.com/products/devops.html) +- [Install DevOps](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/activate-dev-ops.html) +- [Install DevOps Integrations](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/activate-dev-ops-integrations.html) +- [GitLab SCM and Continuous Integration for DevOps](https://store.servicenow.com/sn_appstore_store.do#!/store/application/54dc4eacdbc2dcd02805320b7c96191e/) +- [Model a GitLab CI pipeline in DevOps](https://docs.servicenow.com/bundle/paris-devops/page/product/enterprise-dev-ops/task/model-gitlab-pipeline-dev-ops.html). diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 7be58ce4ecb..f8172a0f988 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -15,59 +15,45 @@ organize, and visualize a workflow for a feature or product release. It can be used as a [Kanban](https://en.wikipedia.org/wiki/Kanban_(development)) or a [Scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)) board. -It pairs issue tracking and project management, -keeping everything in the same place, so that you don't need to jump -between different platforms to organize your workflow. +It pairs issue tracking and project management, keeping everything in the same place, +so that you don't need to jump between different platforms to organize your workflow. -With issue boards, you organize your issues in lists that correspond to -their assigned labels, visualizing issues designed as cards throughout those lists. +Issue boards build on the existing [issue tracking functionality](issues/index.md#issues-list) and +[labels](labels.md). Your issues appear as cards in vertical lists, organized by their assigned +labels, [milestones](#milestone-lists), or [assignees](#assignee-lists). -You define your process, and GitLab organizes it for you. You add your labels -then create the corresponding list to pull in your existing issues. When -you're ready, you can drag and drop your issue cards from one step to the next. +Issue boards help you to visualize and manage your entire process in GitLab. +You add your labels, and then create the corresponding list for your existing issues. +When you're ready, you can drag your issue cards from one step to another one. - - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -Watch a [video presentation](https://youtu.be/UWsJ8tkHAa8) of -the Issue Board feature (introduced in GitLab 8.11 - August 2016). - -### Advanced features of issue boards - -- Create multiple issue boards per project. -- Create multiple issue boards per group. **(PREMIUM)** -- Add lists for [assignees](#assignee-lists-premium) and [milestones](#milestone-lists-premium). **(PREMIUM)** - -Check all the [GitLab Enterprise features for issue boards](#gitlab-enterprise-features-for-issue-boards). +An issue board can show you what issues your team is working on, who is assigned to each, +and where in the workflow those issues are. - +To let your team members organize their own workflows, use +[multiple issue boards](#use-cases-for-multiple-issue-boards). This allows creating multiple issue +boards in the same project. -## How it works + -The Issue Board feature builds on GitLab's existing -[issue tracking functionality](issues/index.md#issues-list) and -[labels](labels.md) by using them as lists of the Scrum board. +Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), +as shown in the following table: -With issue boards you can have a different view of your issues while -maintaining the same filtering and sorting abilities you see across the -issue tracker. An issue board is based on its project's label structure, so it -applies the same descriptive labels to indicate placement on the board, keeping -consistency throughout the entire development lifecycle. +| Tier | Number of project issue boards | Number of [group issue boards](#group-issue-boards) | [Configurable issue boards](#configurable-issue-boards) | [Assignee lists](#assignee-lists) | +|------------------|--------------------------------|------------------------------|---------------------------|----------------| +| Core / Free | Multiple | 1 | No | No | +| Starter / Bronze | Multiple | 1 | Yes | No | +| Premium / Silver | Multiple | Multiple | Yes | Yes | +| Ultimate / Gold | Multiple | Multiple | Yes | Yes | -An issue board shows you what issues your team is working on, who is assigned to each, -and where in the workflow those issues are. +To learn more, visit [GitLab Enterprise features for issue boards](#gitlab-enterprise-features-for-issue-boards) below. -You create issues, host code, perform reviews, build, test, -and deploy from one single platform. Issue boards help you to visualize -and manage the entire process in GitLab. + -With [multiple issue boards](#use-cases-for-multiple-issue-boards), -you go even further, as you can not only keep yourself and your project -organized from a broader perspective with one issue board per project, -but also allow your team members to organize their own workflow by creating -multiple issue boards within the same project. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +Watch a [video presentation](https://youtu.be/vjccjHI7aGI) of +the Issue Board feature. -## Use cases +## Issue boards use cases You can tailor GitLab issue boards to your own preferred workflow. Here are some common use cases for issue boards. @@ -138,8 +124,7 @@ to improve their workflow with multiple boards. #### Quick assignments -Create lists for each of your team members and quickly drag and drop issues onto each team member's -list. +Create lists for each of your team members and quickly drag issues onto each team member's list. ## Issue board terminology @@ -155,8 +140,8 @@ that belong to it. Types of lists include: Always appears as the leftmost list. - **Closed** (default): all closed issues. Always appears as the rightmost list. - **Label list**: all open issues for a label. -- [**Assignee list**](#assignee-lists-premium): all open issues assigned to a user. -- [**Milestone list**](#milestone-lists-premium): all open issues for a milestone. +- [**Assignee list**](#assignee-lists): all open issues assigned to a user. +- [**Milestone list**](#milestone-lists): all open issues for a milestone. A **Card** is a box on a list, and it represents an issue. You can drag cards from one list to another to change their label, assignee, or milestone. The information you can see on a @@ -172,22 +157,36 @@ card includes: Users with the [Reporter and higher roles](../permissions.md) can use all the functionality of the Issue Board feature to create or delete lists and drag issues from one list to another. -## GitLab Enterprise features for issue boards +## How GitLab orders issues in a list -GitLab issue boards are available on GitLab Core and GitLab.com Free tiers, but some -advanced functionality is present in [higher tiers only](https://about.gitlab.com/pricing/). +When visiting a board, issues appear ordered in any list. You're able to change +that order by dragging the issues. The changed order is saved, so that anybody who visits the same +board later sees the reordering, with some exceptions. -### Summary of features per tier +The first time a given issue appears in any board (that is, the first time a user +loads a board containing that issue), it is ordered in relation to other issues in that list +according to [label priority](labels.md#label-priority). -Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), -as shown in the following table: +At this point, that issue is assigned a relative order value by the system, +representing its relative order with respect to the other issues in the list. Any time +you reorder that issue by dragging, its relative order value changes accordingly. -| Tier | Number of Project issue boards | Number of Group issue boards | Configurable issue boards | Assignee lists | -|------------------|--------------------------------|------------------------------|---------------------------|----------------| -| Core / Free | Multiple | 1 | No | No | -| Starter / Bronze | Multiple | 1 | Yes | No | -| Premium / Silver | Multiple | Multiple | Yes | Yes | -| Ultimate / Gold | Multiple | Multiple | Yes | Yes | +Also, any time that issue appears in any board when it's loaded by a user, +the updated relative order value is used for the ordering. It's only the first +time an issue appears that it takes from the priority order mentioned above. This means that +if issue `A` is reordered by dragging to be above issue `B` by any user in +a given board inside your GitLab instance, any time those two issues are subsequently +loaded in any board in the same instance (could be a different project board or a different group +board, for example), that ordering is maintained. + +This ordering also affects [issue lists](issues/sorting_issue_lists.md). +Changing the order in an issue board changes the ordering in an issue list, +and vice versa. + +## GitLab Enterprise features for issue boards + +GitLab issue boards are available on GitLab Core and GitLab.com Free tiers, but some +advanced functionality is present in [higher tiers only](https://about.gitlab.com/pricing/). ### Multiple issue boards @@ -248,6 +247,10 @@ clicking **View scope**.  +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +Watch a [video presentation](https://youtu.be/m5UTNCSqaDk) of +the Configurable Issue Board feature. + ### Focus mode > - [Introduced]((https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#issue-boards-focus-mode-ees-eep)) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1. @@ -265,7 +268,7 @@ is hidden, allowing you to focus on issues in the board. The top of each list indicates the sum of issue weights for the issues that belong to that list. This is useful when using boards for capacity allocation, -especially in combination with [assignee lists](#assignee-lists-premium). +especially in combination with [assignee lists](#assignee-lists).  @@ -362,7 +365,6 @@ status. - [Create workflows](#create-workflows). - [Drag issues between lists](#drag-issues-between-lists). - [Multi-select issue cards](#multi-select-issue-cards). -- [Re-order issues in lists](#issue-ordering-in-a-list). - Drag and reorder the lists. - Change issue labels (by dragging an issue between lists). - Close an issue (by dragging it to the **Done** list). @@ -441,8 +443,9 @@ You can filter by author, assignee, milestone, and label. ### Create workflows By reordering your lists, you can create workflows. As lists in issue boards are -based on labels, it works out of the box with your existing issues. So if you've -already labeled things with 'Backend' and 'Frontend', the issue appears in +based on labels, it works out of the box with your existing issues. + +So if you've already labeled things with **Backend** and **Frontend**, the issue appears in the lists as you create them. In addition, this means you can easily move something between lists by changing a label. @@ -456,20 +459,22 @@ A typical workflow of using an issue board would be: 1. You move issues around in lists so that your team knows who should be working on what issue. 1. When the work by one team is done, the issue can be dragged to the next list - so someone else can pick up. + so someone else can pick it up. 1. When the issue is finally resolved, the issue is moved to the **Done** list and gets automatically closed. -For instance you can create a list based on the label of 'Frontend' and one for -'Backend'. A designer can start working on an issue by adding it to the -'Frontend' list. That way, everyone knows that this issue is now being -worked on by the designers. Then, once they're done, all they have to do is -drag it over to the next list, 'Backend', where a backend developer can +For example, you can create a list based on the label of **Frontend** and one for +**Backend**. A designer can start working on an issue by adding it to the +**Frontend** list. That way, everyone knows that this issue is now being +worked on by the designers. + +Then, once they're done, all they have to do is +drag it to the next list, **Backend**, where a backend developer can eventually pick it up. Once they’re done, they move it to **Done**, to close the issue. This process can be seen clearly when visiting an issue since with every move -to another list the label changes and a system not is recorded. +to another list the label changes and a system note is recorded.  @@ -497,33 +502,6 @@ To select and move multiple cards:  -### Issue ordering in a list - -When visiting a board, issues appear ordered in any list. You're able to change -that order by dragging and dropping the issues. The changed order will be saved -to the system so that anybody who visits the same board later will see the reordering, -with some exceptions. - -The first time a given issue appears in any board (that is, the first time a user -loads a board containing that issue), it is ordered with -respect to other issues in that list according to [Priority order](labels.md#label-priority). - -At that point, that issue is assigned a relative order value by the system -representing its relative order with respect to the other issues in the list. Any time -you drag-and-drop reorder that issue, its relative order value changes accordingly. - -Also, any time that issue appears in any board when it's loaded by a user, -the updated relative order value is used for the ordering. (It's only the first -time an issue appears that it takes from the Priority order mentioned above.) This means that -if issue `A` is drag-and-drop reordered to be above issue `B` by any user in -a given board inside your GitLab instance, any time those two issues are subsequently -loaded in any board in the same instance (could be a different project board or a different group -board, for example), that ordering is maintained. - -This ordering also affects [issue lists](issues/sorting_issue_lists.md). -Changing the order in an issue board changes the ordering in an issue list, -and vice versa. - ## Tips A few things to remember: @@ -537,4 +515,4 @@ A few things to remember: and show only the issues from all lists that have that label. - For performance and visibility reasons, each list shows the first 20 issues by default. If you have more than 20 issues, start scrolling down and the next - 20 appears. + 20 appear. diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index 5e456c7986c..7c9278c8403 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -1,3 +1,9 @@ +--- +stage: Create +group: Knowledge +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +--- + # Design Management > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. @@ -72,39 +78,12 @@ and connect to GitLab through a personal access token. The details are explained ## The Design Management section > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223193) in GitLab 13.2, Designs are displayed directly on the issue description rather than on a separate tab. -> - The new display is deployed behind a feature flag, enabled by default. -> - It's enabled on GitLab.com. -> - It cannot be enabled or disabled per-project. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-displaying-designs-on-the-issue-description-core-only). If disabled, it will move Designs back to the **Designs** tab. +> - New display's feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/223197) in GitLab 13.4. You can find to the **Design Management** section in the issue description:  -### Enable or disable displaying Designs on the issue description **(CORE ONLY)** - -Displaying Designs on the issue description is under development but ready for -production use. It is deployed behind a feature flag that is **enabled by -default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can opt to disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:design_management_moved) -``` - -To enable it: - -```ruby -Feature.enable(:design_management_moved) -``` - -By disabling this feature, designs will be displayed on the **Designs** tab -instead of directly on the issue description. - ## Adding designs To upload Design images, drag files from your computer and drop them in the Design Management section, @@ -252,13 +231,47 @@ Note that your resolved comment pins will disappear from the Design to free up s However, if you need to revisit or find a resolved discussion, all of your resolved threads will be available in the **Resolved Comment** area at the bottom of the right sidebar. +## Add To-Do for Designs + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198439) in GitLab 13.4. +> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-the-design-to-do-button). **(CORE ONLY)** + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +Add a to-do for a design by clicking **Add a To-Do** on the design sidebar: + + + +### Enable or disable the design To-Do button **(CORE ONLY)** + +The **Add a To-Do** button for Designs is under development but ready for production use. It is +deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:design_management_todo_button) +``` + +To disable it: + +```ruby +Feature.disable(:design_management_todo_button) +``` + ## Referring to designs in Markdown > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217160) in **GitLab 13.1**. > - It is deployed behind a feature flag, disabled by default. > - It is disabled on GitLab.com. > - It is not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-design-references-core-only). **(CORE ONLY)** +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-design-references). **(CORE ONLY)** We support referring to designs in [Markdown](../../markdown.md), which is available throughout the application, including in merge request and issue descriptions, in discussions and comments, and in wiki pages. diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index 56fb4ca5cc7..55b45bf9d3d 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -44,9 +44,9 @@ the icon and the date colored red. You can sort issues by those that are  -Due dates also appear in your [todos list](../../todos.md). +Due dates also appear in your [to-do list](../../todos.md). - + The day before an open issue is due, an email will be sent to all participants of the issue. Like the due date, the "day before the due date" is determined by the diff --git a/doc/user/project/issues/img/design_todo_button_v13_4.png b/doc/user/project/issues/img/design_todo_button_v13_4.png Binary files differnew file mode 100644 index 00000000000..62bbecf4ed9 --- /dev/null +++ b/doc/user/project/issues/img/design_todo_button_v13_4.png diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index a6911d183c1..060266a478f 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -93,7 +93,7 @@ must be set. While you can view and manage the full details of an issue on the [issue page](#issue-page), you can also work with multiple issues at a time using the [Issues List](#issues-list), -[Issue Boards](#issue-boards), Issue references, and [Epics](#epics-premium)**(PREMIUM)**. +[Issue Boards](#issue-boards), Issue references, and [Epics](#epics)**(PREMIUM)**. Key actions for Issues include: @@ -112,8 +112,6 @@ and modify them if you have the necessary [permissions](../../permissions.md). #### Real-time sidebar **(CORE ONLY)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 13.3. -> - It cannot be enabled or disabled per-project. -> - It's not recommended for production use. Assignees in the sidebar are updated in real time. This feature is **disabled by default**. To enable, you need to enable [ActionCable in-app mode](https://docs.gitlab.com/omnibus/settings/actioncable.html). @@ -186,8 +184,8 @@ requires [GraphQL](../../../api/graphql/index.md) to be enabled. ### Health status **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36427) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. - +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36427) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +> - Health status of closed issues [can't be edited](https://gitlab.com/gitlab-org/gitlab/-/issues/220867) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.4 and later. To help you track the status of your issues, you can assign a status to each issue to flag work that's progressing as planned or needs attention to keep on schedule: @@ -197,8 +195,11 @@ that's progressing as planned or needs attention to keep on schedule:  +After an issue is closed, its health status can't be edited and the "Edit" button becomes disabled +until the issue is reopened. + You can then see issue statuses on the -[Epic tree](../../group/epics/index.md#issue-health-status-in-epic-tree-ultimate). +[Epic tree](../../group/epics/index.md#issue-health-status-in-epic-tree). #### Disable issue health status @@ -220,4 +221,4 @@ Feature.disable(:save_issuable_health_status) - [Export issues](csv_export.md) - [Issues API](../../../api/issues.md) - Configure an [external issue tracker](../../../integration/external-issue-tracker.md) - such as Jira, Redmine, or Bugzilla. + such as Jira, Redmine, Bugzilla, or EWM. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 77c50f9178c..5356e6aeb40 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -21,13 +21,13 @@ You can find all the information for that issue on one screen. - **1.** [New Issue, close issue (reopen issue, report issue)](#new-issue-close-issue-reopen-issue-report-issue) - **2.** [To Do](#to-do) - **3.** [Assignee](#assignee) - - **3.1.** [Multiple Assignees **(STARTER)**](#multiple-assignees-starter) -- **4.** [Epic **(PREMIUM)**](#epic-premium) + - **3.1.** [Multiple Assignees **(STARTER)**](#multiple-assignees) +- **4.** [Epic **(PREMIUM)**](#epic) - **5.** [Milestone](#milestone) - **6.** [Time tracking](#time-tracking) - **7.** [Due date](#due-date) - **8.** [Labels](#labels) -- **9.** [Weight **(STARTER)**](#weight-starter) +- **9.** [Weight **(STARTER)**](#weight) - **10.** [Confidentiality](#confidentiality) - **11.** [Lock issue](#lock-issue) - **12.** [Participants](#participants) @@ -36,7 +36,7 @@ You can find all the information for that issue on one screen. - **15.** [Edit](#edit) - **16.** [Description](#description) - **17.** [Mentions](#mentions) -- **18.** [Related Issues **(STARTER)**](#related-issues-starter) +- **18.** [Related Issues **(STARTER)**](#related-issues) - **19.** [Related Merge Requests](#related-merge-requests) - **20.** [Award emoji](#award-emoji) - **21.** [Show all activity](#show-all-activity) @@ -88,7 +88,7 @@ An issue can be assigned to: - Yourself. - Another person. -- [Many people](#multiple-assignees-starter). **(STARTER)** +- [Many people](#multiple-assignees). **(STARTER)** The assignee(s) can be changed as often as needed. The idea is that the assignees are responsible for that issue until it's reassigned to someone else to take it from there. @@ -196,7 +196,7 @@ allowing many formatting options. ### Mentions You can mention a user or a group present in your GitLab instance with `@username` or -`@groupname` and they will be notified via todos and email, unless they have disabled +`@groupname` and they will be notified via to-dos and email, unless they have disabled all notifications in their profile settings. This is controlled in the [notification settings](../../profile/notifications.md). diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index 7cbd9906800..8a8359a4b02 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -11,7 +11,7 @@ etc. The available sorting options can change based on the context of the list. For sorting by issue priority, see [Label Priority](../labels.md#label-priority). In group and project issue lists, it is also possible to order issues manually, -similar to [issue boards](../issue_board.md#issue-ordering-in-a-list). +similar to [issue boards](../issue_board.md#how-gitlab-orders-issues-in-a-list). ## Manual sorting @@ -31,6 +31,6 @@ a given list inside your GitLab instance, any time those two issues are subseque loaded in any list in the same instance (could be a different project issue list or a different group issue list, for example), that ordering will be maintained. -This ordering also affects [issue boards](../issue_board.md#issue-ordering-in-a-list). +This ordering also affects [issue boards](../issue_board.md#how-gitlab-orders-issues-in-a-list). Changing the order in an issue list changes the ordering in an issue board, and vice versa. diff --git a/doc/user/project/merge_requests/browser_performance_testing.md b/doc/user/project/merge_requests/browser_performance_testing.md index 10457e40e0b..040ca4b439b 100644 --- a/doc/user/project/merge_requests/browser_performance_testing.md +++ b/doc/user/project/merge_requests/browser_performance_testing.md @@ -37,7 +37,7 @@ Consider the following workflow: ## How browser performance testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). +[Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance). GitLab then checks this report, compares key performance metrics for each page between the source and target branches, and shows the information in the merge request. @@ -85,7 +85,7 @@ The example uses a CI/CD template that is included in all GitLab installations s or older, you must [add the configuration manually](#gitlab-versions-123-and-older) The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance), -and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) +and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance) that you can later download and analyze. This implementation always takes the latest Browser Performance artifact available. If [GitLab Pages](../pages/index.md) is enabled, you can view the report directly in your browser. @@ -93,7 +93,7 @@ you can view the report directly in your browser. You can also customize the jobs with environment variables: - `SITESPEED_IMAGE`: Configure the Docker image to use for the job (default `sitespeedio/sitespeed.io`), but not the image version. -- `SITESPEED_VERSION`: Configure the version of the Docker image to use for the job (default `13.3.0`). +- `SITESPEED_VERSION`: Configure the version of the Docker image to use for the job (default `14.1.0`). - `SITESPEED_OPTIONS`: Configure any additional sitespeed.io options as required (default `nil`). Refer to the [sitespeed.io documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) for more details. For example, you can override the number of runs sitespeed.io @@ -196,13 +196,13 @@ performance: image: docker:git variables: URL: https://example.com - SITESPEED_VERSION: 13.3.0 + SITESPEED_VERSION: 14.1.0 SITESPEED_OPTIONS: '' services: - docker:stable-dind script: - mkdir gitlab-exporter - - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js + - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/1.1.0/index.js - mkdir sitespeed-results - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:$SITESPEED_VERSION --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL $SITESPEED_OPTIONS - mv sitespeed-results/data/performance.json performance.json @@ -226,7 +226,7 @@ performance: - docker:stable-dind script: - mkdir gitlab-exporter - - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js + - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/1.1.0/index.js - mkdir sitespeed-results - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL - mv sitespeed-results/data/performance.json performance.json diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index 3c697e22cf5..e03d4e99b86 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -23,7 +23,7 @@ Code Quality: Quality](https://gitlab.com/gitlab-org/ci-cd/codequality) project using [default Code Climate configurations](https://gitlab.com/gitlab-org/ci-cd/codequality/-/tree/master/codeclimate_defaults). - Can make use of a [template](#example-configuration). - Is available with [Auto - DevOps](../../../topics/autodevops/stages.md#auto-code-quality-starter). + DevOps](../../../topics/autodevops/stages.md#auto-code-quality). - Can be extended through [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) or a [custom tool](#implementing-a-custom-tool). ## Code Quality Widget @@ -69,7 +69,7 @@ For instance, consider the following workflow: This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. It requires GitLab 11.11 or later, and GitLab Runner 11.5 or later. If you are using -GitLab 11.4 or ealier, you can view the deprecated job definitions in the +GitLab 11.4 or earlier, you can view the deprecated job definitions in the [documentation archive](https://docs.gitlab.com/12.10/ee/user/project/merge_requests/code_quality.html#previous-job-definitions). First, you need GitLab Runner configured: @@ -77,7 +77,7 @@ First, you need GitLab Runner configured: - For the [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). - With enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB. -Once you set up the Runner, include the Code Quality template in your CI configuration: +Once you set up GitLab Runner, include the Code Quality template in your CI configuration: ```yaml include: @@ -102,6 +102,16 @@ code_quality: CODE_QUALITY_IMAGE: "registry.example.com/codequality-fork:latest" ``` +In [GitLab 13.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/11100), you can override the [Code Quality environment variables](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables): + +```yaml +variables: + TIMEOUT_SECONDS: 1 + +include: + - template: Code-Quality.gitlab-ci.yml +``` + By default, report artifacts are not downloadable. If you need them downloadable on the job details page, you can add `gl-code-quality-report.json` to the artifact paths like so: @@ -126,7 +136,7 @@ This information will be automatically extracted and shown right in the merge re CAUTION: **Caution:** On self-managed instances, if a malicious actor compromises the Code Quality job -definition they will be able to execute privileged Docker commands on the Runner +definition they will be able to execute privileged Docker commands on the runner host. Having proper access control policies mitigates this attack vector by allowing access only to trusted actors. @@ -276,7 +286,7 @@ This adds SonarJava to the `plugins:` section of the [default `.codeclimate.yml` included in your project. Changes to the `plugins:` section do not affect the `exclude_patterns` section of the -defeault `.codeclimate.yml`. See the Code Climate documentation for +default `.codeclimate.yml`. See the Code Climate documentation for [excluding files and folders](https://docs.codeclimate.com/docs/excluding-files-and-folders) for more details. diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index c7cabf3c73b..a0be32e0708 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -53,7 +53,7 @@ When you start a new merge request, you can immediately include the following options, or add them later by clicking the **Edit** button on the merge request's page at the top-right side: -- [Assign](#assignee) the merge request to a colleague for review. With GitLab Starter and higher tiers, you can [assign it to more than one person at a time](#multiple-assignees-starter). +- [Assign](#assignee) the merge request to a colleague for review. With GitLab Starter and higher tiers, you can [assign it to more than one person at a time](#multiple-assignees). - Set a [milestone](../milestones/index.md) to track time-sensitive changes. - Add [labels](../labels.md) to help contextualize and filter your merge requests over time. - Require [approval](merge_request_approvals.md) from your team. **(STARTER)** diff --git a/doc/user/project/merge_requests/img/browser_performance_testing.png b/doc/user/project/merge_requests/img/browser_performance_testing.png Binary files differindex 016abb89f7c..a3d7022bcfc 100644 --- a/doc/user/project/merge_requests/img/browser_performance_testing.png +++ b/doc/user/project/merge_requests/img/browser_performance_testing.png diff --git a/doc/user/project/merge_requests/img/update_approval_rule_v13_4.png b/doc/user/project/merge_requests/img/update_approval_rule_v13_4.png Binary files differnew file mode 100644 index 00000000000..af713b48140 --- /dev/null +++ b/doc/user/project/merge_requests/img/update_approval_rule_v13_4.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index a9ee9d8e507..69276f0677b 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -19,7 +19,7 @@ A. Consider you're a software developer working in a team: 1. You checkout a new branch, and submit your changes through a merge request 1. You gather feedback from your team 1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) -1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD +1. You verify your changes with [Unit test reports](../../../ci/unit_test_reports.md) in GitLab CI/CD 1. You avoid using dependencies whose license is not compatible with your project with [License Compliance reports](../../compliance/license_compliance/index.md) **(ULTIMATE)** 1. You request the [approval](merge_request_approvals.md) from your manager **(STARTER)** 1. Your manager: diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index 97f4f202ab3..daebd71e14f 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -28,7 +28,7 @@ GET calls to a popular API endpoint in your application to see how it performs. ## How Load Performance Testing works First, define a job in your `.gitlab-ci.yml` file that generates the -[Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +[Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance). GitLab checks this report, compares key load performance metrics between the source and target branches, and then shows the information in a merge request widget: @@ -102,7 +102,7 @@ job. An example configuration workflow: -1. Set up a GitLab Runner that can run Docker containers, such as a Runner using the +1. Set up GitLab Runner to run Docker containers, like the [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). 1. Configure the default Load Performance Testing CI job in your `.gitlab-ci.yml` file. You need to include the template and configure it with variables: @@ -140,7 +140,7 @@ For example, you can override the duration of the test with a CLI option: GitLab only displays the key performance metrics in the MR widget if k6's results are saved via [summary export](https://k6.io/docs/results-visualization/json#summary-export) -as a [Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance-premium). +as a [Load Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsload_performance). The latest Load Performance artifact available is always used, using the summary values from the test. @@ -152,17 +152,20 @@ The CI/CD YAML configuration example above works for testing against static envi but it can be extended to work with [review apps](../../../ci/review_apps) or [dynamic environments](../../../ci/environments) with a few extra steps. -The best approach is to capture the dynamic URL into a custom environment variable that -is then [inherited](../../../ci/variables/README.md#inherit-environment-variables) -by the `load_performance` job. The k6 test script to be run should then be configured to -use that environment URL, such as: ``http.get(`${__ENV.ENVIRONMENT_URL`})``. +The best approach is to capture the dynamic URL in a [`.env` file](https://docs.docker.com/compose/env-file/) +as a job artifact to be shared, then use a custom environment variable we've provided named `K6_DOCKER_OPTIONS` +to configure the k6 Docker container to use the file. With this, k6 can then use any +environment variables from the `.env` file in scripts using standard JavaScript, +such as: ``http.get(`${__ENV.ENVIRONMENT_URL`})``. For example: 1. In the `review` job: - 1. Capture the dynamic URL and save it into a `.env` file, e.g. `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. - 1. Set the `.env` file to be an [`artifacts:reports:dotenv` report](../../../ci/variables/README.md#inherit-environment-variables). -1. Set the `load_performance` job to depend on the review job, so it inherits the environment variable. + 1. Capture the dynamic URL and save it into a `.env` file, e.g. `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. + 1. Set the `.env` file to be a [job artifact](../../../ci/pipelines/job_artifacts.md#job-artifacts). +1. In the `load_performance` job: + 1. Set it to depend on the review job, so it inherits the env file. + 1. Set the `K6_DOCKER_OPTIONS` variable with the [Docker cli option for env files](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file), for example `--env-file review.env`. 1. Configure the k6 test script to use the environment variable in it's steps. Your `.gitlab-ci.yml` file might be similar to: @@ -184,15 +187,16 @@ review: - run_deploy_script - echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env artifacts: - reports: - dotenv: - review.env + paths: + - review.env rules: - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. load_performance: dependencies: - review + variables: + K6_DOCKER_OPTIONS: '--env-file review.env' rules: - if: '$CI_COMMIT_BRANCH' # Modify to match your pipeline rules, or use `only/except` if needed. ``` diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index 407fc5db425..185ab0e6298 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -36,7 +36,7 @@ Required approvals enable multiple use cases: database, and so on, for all proposed code changes. - Designating [Code Owners as eligible approvers](#code-owners-as-eligible-approvers), determined by the files changed in a merge request. -- [Requiring approval from a security team](#security-approvals-in-merge-requests-ultimate) +- [Requiring approval from a security team](#security-approvals-in-merge-requests) before merging code that could introduce a vulnerability.**(ULTIMATE)** ### Approval Rules @@ -52,7 +52,7 @@ minimum number of required approvers can still be set in the [project settings f You can opt to define one single rule to approve a merge request among the available rules or choose more than one. Single approval rules are available in GitLab Starter and higher tiers, -while [multiple approval rules](#multiple-approval-rules-premium) are available in +while [multiple approval rules](#multiple-approval-rules) are available in [GitLab Premium](https://about.gitlab.com/pricing/) and above. NOTE: **Note:** @@ -61,6 +61,8 @@ group is public. #### Eligible Approvers +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10294) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.3, when an eligible approver comments on a merge request, it appears in the **Commented by** column of the Approvals widget. + The following users can approve merge requests: - Users who have been added as approvers at the project or merge request levels with @@ -84,8 +86,7 @@ if [**Prevent author approval**](#allowing-merge-request-authors-to-approve-thei and [**Prevent committers approval**](#prevent-approval-of-merge-requests-by-their-committers) (disabled by default) are enabled on the project settings. -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10294) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.3, -when an eligible approver comments on a merge request, it appears in the **Commented by** column of the Approvals widget, +When an eligible approver comments on a merge request, it appears in the **Commented by** column of the Approvals widget, indicating who has engaged in the merge request review. Authors and reviewers can also easily identify who they should reach out to if they have any questions or inputs about the content of the merge request. @@ -118,7 +119,30 @@ users with Developer or higher permissions, as well as by Code Owners, indistinguishably. Alternatively, you can **require** -[Code Owner's approvals for Protected Branches](../protected_branches.md#protected-branches-approval-by-code-owners-premium). **(PREMIUM)** +[Code Owner's approvals for Protected Branches](../protected_branches.md#protected-branches-approval-by-code-owners). **(PREMIUM)** + +#### Merge Request approval segregation of duties + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40491) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.4. + +Managers or operators with [Reporter permissions](../../permissions.md#project-members-permissions) +to a project sometimes need to be required approvers of a merge request, +before a merge to a protected branch begins. These approvers aren't allowed +to push or merge code to any branches. + +To enable this access: + +1. [Create a new group](../../group/index.md#create-a-new-group), and then + [add the user to the group](../../group/index.md#add-users-to-a-group), + ensuring you select the Reporter role for the user. +1. [Share the project with your group](../members/share_project_with_groups.md#sharing-a-project-with-a-group-of-users), + based on the Reporter role. +1. Navigate to your project's **Settings > General**, and in the + **Merge request approvals** section, click **Expand**. +1. [Add the group](../../group/index.md#create-a-new-group) to the permission list + for the protected branch. + + #### Adding / editing a default approval rule @@ -204,7 +228,7 @@ Alternatively, you can select a very specific protected branch from the **Target  -To enable this configuration, see [Code Owner’s approvals for protected branches](../protected_branches.md#protected-branches-approval-by-code-owners-premium). +To enable this configuration, see [Code Owner’s approvals for protected branches](../protected_branches.md#protected-branches-approval-by-code-owners). ### Adding or removing an approval @@ -242,9 +266,9 @@ The project settings for Merge request approvals are found by going to #### Prevent overriding default approvals -By default, users are able to edit the approval rules in merge requests. If disabled, -the approval rules for all new merge requests will be determined by the -[default approval rules](#adding--editing-a-default-approval-rule). To disable this feature: +Regardless of the approval rules you choose for your project, users can edit them in every merge +request, overriding the rules you set as [default](#adding--editing-a-default-approval-rule). +To prevent that from happening: 1. Uncheck the **Can override approvers and approvals required per merge request** checkbox. 1. Click **Save changes**. @@ -267,14 +291,15 @@ from the UI. However, approvals will be reset if the target branch is changed. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3349) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.3. -You can allow merge request authors to self-approve merge requests. Authors -also need to be included in the approvers list in order to be able to -approve their merge request. To enable this feature: +By default, projects are configured to prevent merge requests from being approved by +their own authors. To change this setting: -1. Uncheck the **Prevent approval of merge requests by merge request author** checkbox, - which is enabled by default. +1. Go to your project's **Settings > General**, expand **Merge request approvals**. +1. Uncheck the **Prevent approval of merge requests by merge request author** checkbox. 1. Click **Save changes**. +Note that users can edit the approval rules in every merge request and override pre-defined settings unless it's set [**not to allow** overrides](#prevent-overriding-default-approvals). + #### Prevent approval of merge requests by their committers > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10441) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.10. diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index 4e821145339..3a18cacde64 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, reference --- -# Reviewing and managing merge requests +# Reviewing and managing merge requests **(CORE)** Merge requests are the primary method of making changes to files in a GitLab project. Changes are proposed by [creating and submitting a merge request](creating_merge_requests.md), @@ -67,13 +67,21 @@ list.  +### Collapsed files in the Changes view + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232820) in GitLab 13.4. + +When you review changes in the **Changes** tab, files with a large number of changes are collapsed +to improve performance. When files are collapsed, a warning appears at the top of the changes. +Click **Expand file** on any file to view the changes for that file. + ### File-by-file diff navigation > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222790) in GitLab 13.2. > - It's deployed behind a feature flag, enabled by default. > - It's recommended for production use. > - It's enabled on GitLab.com. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-file-by-file-diff-navigation-core-only). +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-file-by-file-diff-navigation). For larger merge requests it might sometimes be useful to review single files at a time. To enable, from your avatar on the top-right navbar, click **Settings**, and go to **Preferences** on the left @@ -156,7 +164,7 @@ in a Merge Request. To do so, click the **{comment}** **comment** icon in the gu > - It's enabled on GitLab.com. > - It can be disabled or enabled per-project. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-multiline-comments-core-only). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-multiline-comments). **(CORE ONLY)** GitLab provides a way to select which lines of code a comment refers to. After starting a comment a dropdown selector is shown to select the first line that this comment refers to. @@ -203,6 +211,11 @@ If there's an [environment](../../../ci/environments/index.md) and the applicati successfully deployed to it, the deployed environment and the link to the Review App will be shown as well. +NOTE: **Note:** +When the default branch (for example, `main`) is red due to a failed CI pipeline, the `merge` button +When the pipeline fails in a merge request but it can be merged nonetheless, +the **Merge** button will be colored in red. + ### Post-merge pipeline status When a merge request is merged, you can see the post-merge pipeline status of @@ -284,15 +297,37 @@ the command line. NOTE: **Note:** This section might move in its own document in the future. -### Checkout merge requests locally +### Copy the branch name for local checkout + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23767) in GitLab 13.4. + +The merge request sidebar contains the branch reference for the source branch +used to contribute changes for this merge request. + +To copy the branch reference into your clipboard, click the **Copy branch name** button +(**{copy-to-clipboard}**) in the right sidebar. You can then use it to checkout the branch locally +via command line by running `git checkout <branch-name>`. + +### Checkout merge requests locally through the `head` ref A merge request contains all the history from a repository, plus the additional commits added to the branch associated with the merge request. Here's a few -tricks to checkout a merge request locally. +ways to checkout a merge request locally. Please note that you can checkout a merge request locally even if the source project is a fork (even a private fork) of the target project. +This relies on the merge request `head` ref (`refs/merge-requests/:iid/head`) +that is available for each merge request. It allows checking out a merge +request via its ID instead of its branch. + +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223156) in GitLab +13.4, 14 days after a merge request gets closed or merged, the merge request +`head` ref will be deleted. This means that the merge request will not be available +for local checkout via the merge request `head` ref anymore. The merge request +can still be re-opened. Also, as long as the merge request's branch +exists, you can still check out the branch as it won't be affected. + #### Checkout locally by adding a Git alias Add the following alias to your `~/.gitconfig`: diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 7f752b2cc39..69a0dd6e84f 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -65,14 +65,27 @@ meaningful commit messages and: ## Enabling squash for a merge request Anyone who can create or edit a merge request can choose for it to be squashed -on the merge request form: +on the merge request form. Users can select or unselect the checkbox at the moment +they are creating the merge request:  -This can then be overridden at the time of accepting the merge request: +After the merge request is submitted, Squash and Merge can still be enabled or disabled +by editing the merge request description: + +1. Scroll to the top of the merge request page and click **Edit**. +1. Scroll down to the end of the merge request form and select the checkbox +**Squash commits when merge request is accepted**. + +This setting can then be overridden at the time of accepting the merge request. +At the end of the merge request widget, next to the **Merge** button, the **Squash commits** checkbox +can be either selected or unselected:  +Note that Squash and Merge might not be available depending on the project's configuration +for [Squash Commit Options](#squash-commits-options). + ## Commit metadata for squashed commits The squashed commit has the following metadata: @@ -97,7 +110,7 @@ squashing can itself be considered equivalent to rebasing. > - It's enabled on GitLab.com. > - It can be enabled per project. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-squash-commit-options-core-only). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-squash-commit-options). **(CORE ONLY)** With Squash Commits Options you can configure the behavior of Squash and Merge for your project. To set it up, navigate to your project's **Settings > General** and expand **Merge requests**. @@ -133,7 +146,7 @@ To enable it: # Instance-wide Feature.enable(:squash_options) # or by project -Feature.enable(:squash_options, Project.find(<project id>)) +Feature.enable(:squash_options, Project.find(<project ID>)) ``` To disable it: @@ -142,7 +155,7 @@ To disable it: # Instance-wide Feature.disable(:squash_options) # or by project -Feature.disable(:squash_options, Project.find(<project id>)) +Feature.disable(:squash_options, Project.find(<project ID>)) ``` <!-- ## Troubleshooting diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 6751dde155c..56b5774f15b 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -8,9 +8,9 @@ type: reference, howto # Test Coverage Visualization **(CORE ONLY)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It can be enabled or disabled per-project. +> - [Feature flag enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/211410) in GitLab 13.4. +> - It's enabled on GitLab.com. +> - It can be disabled per-project. > - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-feature). **(CORE ONLY)** With the help of [GitLab CI/CD](../../../ci/README.md), you can collect the test @@ -76,11 +76,9 @@ test: ## Enabling the feature -This feature comes with the `:coverage_report_view` feature flag disabled by -default. It is disabled on GitLab.com. This feature is disabled due to some performance issues with very large -data sets. When [the performance issue](https://gitlab.com/gitlab-org/gitlab/-/issues/211410) -is resolved, the feature will be enabled by default. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. Test coverage visualization can be enabled or disabled per-project. +This feature comes with the `:coverage_report_view` feature flag enabled by +default. It is enabled on GitLab.com. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can disable it for your instance. Test coverage visualization can be enabled or disabled per-project. To enable it: diff --git a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md index e5ebc46d58f..b298c62a5e6 100644 --- a/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md +++ b/doc/user/project/merge_requests/testing_and_reports_in_merge_requests.md @@ -19,7 +19,7 @@ or link to useful information directly from merge requests: | [Code Quality](code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | | [Display arbitrary job artifacts](../../../ci/yaml/README.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../../../ci/pipelines/job_artifacts.md) in merge requests. | | [GitLab CI/CD](../../../ci/README.md) | Build, test, and deploy your code in a per-branch basis with built-in CI/CD. | -| [JUnit test reports](../../../ci/junit_test_reports.md) | Configure your CI jobs to use JUnit test reports, and let GitLab display a report on the merge request so that it’s easier and faster to identify the failure without having to check the entire job log. | +| [Unit test reports](../../../ci/unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it’s easier and faster to identify the failure without having to check the entire job log. | | [License Compliance](../../compliance/license_compliance/index.md) **(ULTIMATE)** | Manage the licenses of your dependencies. | | [Metrics Reports](../../../ci/metrics_reports.md) **(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. | | [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. | diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index aea5eef5efc..9d02a22f91e 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -117,9 +117,9 @@ From the project issue/merge request list pages and the group issue/merge reques ### Filtering in issue boards - From [project issue boards](../issue_board.md), you can filter by both group milestones and project milestones in the [search and filter bar](../../search/index.md#issue-boards). -- From [group issue boards](../issue_board.md#group-issue-boards-premium), you can filter by only group milestones in the [search and filter bar](../../search/index.md#issue-boards). **(PREMIUM)** -- From [project issue boards](../issue_board.md), you can filter by both group milestones and project milestones in the [issue board configuration](../issue_board.md#configurable-issue-boards-starter). **(STARTER)** -- From [group issue boards](../issue_board.md#group-issue-boards-premium) you can filter by only group milestones in the [issue board configuration](../issue_board.md#configurable-issue-boards-starter). **(STARTER)** +- From [group issue boards](../issue_board.md#group-issue-boards), you can filter by only group milestones in the [search and filter bar](../../search/index.md#issue-boards). **(PREMIUM)** +- From [project issue boards](../issue_board.md), you can filter by both group milestones and project milestones in the [issue board configuration](../issue_board.md#configurable-issue-boards). **(STARTER)** +- From [group issue boards](../issue_board.md#group-issue-boards) you can filter by only group milestones in the [issue board configuration](../issue_board.md#configurable-issue-boards). **(STARTER)** ### Special milestone filters @@ -154,9 +154,9 @@ For project milestones in [GitLab Starter](https://about.gitlab.com/pricing/), a  -### Group Burndown Charts **(PREMIUM)** +### Group Burndown Charts **(STARTER)** -For group milestones in [GitLab Premium](https://about.gitlab.com/pricing/), a [burndown chart](burndown_charts.md) is in the milestone view, showing the progress of completing a milestone. +For group milestones in [GitLab Starter](https://about.gitlab.com/pricing/), a [burndown chart](burndown_charts.md) is in the milestone view, showing the progress of completing a milestone. ### Milestone sidebar diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index cfcbf11a407..c3825371030 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -54,7 +54,7 @@ It is important to note that we have a few types of users: Administrator will have to be a member of it in order to have access to it via another project's job. -- **External users**: CI jobs created by [external users](../permissions.md#external-users-core-only) will have +- **External users**: CI jobs created by [external users](../permissions.md#external-users) will have access only to projects to which the user has at least Reporter access. This rules out accessing all internal projects by default. @@ -65,7 +65,7 @@ Let's consider the following scenario: hosted in private repositories and you have multiple CI jobs that make use of these repositories. -1. You invite a new [external user](../permissions.md#external-users-core-only). CI jobs created by that user do not +1. You invite a new [external user](../permissions.md#external-users). CI jobs created by that user do not have access to internal repositories, because the user also doesn't have the access from within GitLab. You as an employee have to grant explicit access for this user. This allows us to prevent from accidental data leakage. @@ -83,9 +83,9 @@ We try to make sure that this token doesn't leak by: 1. Masking the job token from job logs. 1. Granting permissions to the job token **only** when the job is running. -However, this brings a question about the Runners security. To make sure that +However, this brings up a question about the runner's security. To make sure that this token doesn't leak, you should also make sure that you configure -your Runners in the most possible secure way, by avoiding the following: +your runners in the most possible secure way, by avoiding the following: 1. Any usage of Docker's `privileged` mode is risky if the machines are re-used. 1. Using the `shell` executor since jobs run on the same machine. @@ -95,13 +95,13 @@ to steal the tokens of other jobs. ## Before GitLab 8.12 -In versions before GitLab 8.12, all CI jobs would use the CI Runner's token +In versions before GitLab 8.12, all CI jobs would use the runner's token to checkout project sources. -The project's Runner's token was a token that you could find under the +The project's runner token was a token that you could find under the project's **Settings > Pipelines** and was limited to access only that project. -It could be used for registering new specific Runners assigned to the project +It could be used for registering new specific runners assigned to the project and to checkout project sources. It could also be used with the GitLab Container Registry for that project, allowing pulling and pushing Docker images from within the CI job. @@ -123,7 +123,7 @@ Using single token had multiple security implications: - The token would be readable to anyone who had Developer access to a project that could run CI jobs, allowing the developer to register any specific - Runner for that project. + runner for that project. - The token would allow to access only the project's sources, forbidding from accessing any other projects. - The token was not expiring and was multi-purpose: used for checking out sources, @@ -205,7 +205,7 @@ Container Registries for private projects. > > - GitLab Runner versions prior to 1.8 don't incorporate the introduced changes > for permissions. This makes the `image:` directive not work with private -> projects automatically and it needs to be configured manually on Runner's host +> projects automatically and it needs to be configured manually on the GitLab Runner host > with a predefined account (for example administrator's personal account with > access token created explicitly for this purpose). This issue is resolved with > latest changes in GitLab Runner 1.8 which receives GitLab credentials with diff --git a/doc/user/project/operations/img/alert_list_v13_1.png b/doc/user/project/operations/img/alert_list_v13_1.png Binary files differdeleted file mode 100644 index 7a1a5f5191e..00000000000 --- a/doc/user/project/operations/img/alert_list_v13_1.png +++ /dev/null diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index e6912259bfa..badafa478ef 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -1,6 +1,4 @@ --- -last_updated: 2020-07-25 -type: reference, howto disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pages/getting_started_part_three.html' stage: Release group: Release Management diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index cabaf734d77..a7eb4c4019f 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -1,6 +1,4 @@ --- -last_updated: 2020-01-06 -type: reference, howto stage: Release group: Release Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers @@ -9,8 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Create a GitLab Pages website from scratch This tutorial shows you how to create a Pages site from scratch. You will start with -a blank project and create your own CI file, which gives instruction to the -[GitLab Runner](https://docs.gitlab.com/runner/). When your CI/CD +a blank project and create your own CI file, which gives instruction to +a [runner](https://docs.gitlab.com/runner/). When your CI/CD [pipeline](../../../../ci/pipelines/index.md) runs, the Pages site is created. This example uses the [Jekyll](https://jekyllrb.com/) Static Site Generator (SSG). @@ -50,7 +48,7 @@ Create three files in the root (top-level) directory. ## Choose a Docker image -In this example, the Runner uses a [Docker image](../../../../ci/docker/using_docker_images.md) +In this example, the runner uses a [Docker image](../../../../ci/docker/using_docker_images.md) to run scripts and deploy the site. This specific Ruby image is maintained on [DockerHub](https://hub.docker.com/_/ruby). @@ -95,7 +93,7 @@ job: ``` For GitLab Pages, this `job` has a specific name, called `pages`. -This setting tells the Runner you want the job to deploy your website +This setting tells the runner you want the job to deploy your website with GitLab Pages: ```yaml @@ -124,7 +122,7 @@ pages: ## Specify the `public` directory for artifacts Now that Jekyll has output the files to the `public` directory, -the Runner needs to know where to get them. The artifacts are stored +the runner needs to know where to get them. The artifacts are stored in the `public` directory: ```yaml @@ -190,7 +188,7 @@ pages: - public ``` -Then configure the pipeline to run the job for the master branch only. +Then configure the pipeline to run the job for the `master` branch only. ```yaml image: ruby:2.7 diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 949a6c16c1b..9272b1f9093 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -1,6 +1,4 @@ --- -last_updated: 2018-06-04 -type: concepts, reference stage: Release group: Release Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index eff80a4c9bd..6c3b911d033 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -59,6 +59,7 @@ To update a GitLab Pages website: | [Explore GitLab Pages](introduction.md) | Requirements, technical aspects, specific GitLab CI/CD configuration options, Access Control, custom 404 pages, limitations, FAQ. | | [Custom domains and SSL/TLS Certificates](custom_domains_ssl_tls_certification/index.md) | Custom domains and subdomains, DNS records, and SSL/TLS certificates. | | [Let's Encrypt integration](custom_domains_ssl_tls_certification/lets_encrypt_integration.md) | Secure your Pages sites with Let's Encrypt certificates, which are automatically obtained and renewed by GitLab. | +| [Redirects](redirects.md) | Set up HTTP redirects to forward one page to another. | Learn more and see examples: diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index a6923779f24..cea6bab1a50 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,6 +1,4 @@ --- -type: reference -last_updated: 2020-01-06 stage: Release group: Release Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers @@ -36,7 +34,7 @@ If you are using [GitLab Pages on GitLab.com](#gitlab-pages-on-gitlabcom) to hos - The domain name for GitLab Pages on GitLab.com is `gitlab.io`. - Custom domains and TLS support are enabled. - Shared runners are enabled by default, provided for free and can be used to - build your website. If you want you can still bring your own Runner. + build your website. If you want you can still bring your own runner. ## Example projects @@ -62,13 +60,8 @@ If the case of `404.html`, there are different scenarios. For example: ## Redirects in GitLab Pages -Since you cannot use any custom server configuration files, like `.htaccess` or -any `.conf` file, if you want to redirect a page to another -location, you can use the [HTTP meta refresh tag](https://en.wikipedia.org/wiki/Meta_refresh). - -Some static site generators provide plugins for that functionality so that you -don't have to create and edit HTML files manually. For example, Jekyll has the -[redirect-from plugin](https://github.com/jekyll/jekyll-redirect-from). +You can configure redirects for your site using a `_redirects` file. To learn more, read +the [redirects documentation](redirects.md). ## GitLab Pages Access Control **(CORE)** diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md index d86704eb703..708d886b352 100644 --- a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -1,7 +1,5 @@ --- description: "How to secure GitLab Pages websites with Let's Encrypt (manual process, deprecated)." -type: howto -last_updated: 2019-07-15 --- # Let's Encrypt for GitLab Pages (manual process, deprecated) diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index 6fcad0a5357..b6b881b961e 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -10,7 +10,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/33422) in GitLab 11.5. > - Available on GitLab.com in GitLab 12.4. -You can enable Pages access control on your project, so that only +You can enable Pages access control on your project +if your administrator has [enabled the access control feature](../../../administration/pages/index.md#access-control) +on your GitLab instance. When enabled, only [members of your project](../../permissions.md#project-members-permissions) (at least Guest) can access your website: diff --git a/doc/user/project/pages/redirects.md b/doc/user/project/pages/redirects.md new file mode 100644 index 00000000000..ae7b1b4fa6e --- /dev/null +++ b/doc/user/project/pages/redirects.md @@ -0,0 +1,130 @@ +--- +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Create redirects for GitLab Pages + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/24) in GitLab Pages 1.25.0 and GitLab 13.4. +> - It's [deployed behind a feature flag](#enable-or-disable-redirects), disabled by default. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-redirects). + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +In GitLab Pages, you can [enable](#enable-or-disable-redirects) the redirects feature to configure rules to forward one URL to another using HTTP redirects. GitLab Pages uses +[Netlify style redirects](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file). + +## Supported features + +GitLab Pages only supports the +[`_redirects` plain text file syntax](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file), +and `.toml` files are not supported. + +Redirects are only supported at a basic level, and GitLab Pages doesn't support all +[special options offered by Netlify](https://docs.netlify.com/routing/redirects/redirect-options/): + +| Feature | Supported | Example | +| ------- | --------- | ------- | +| Redirects (`301`, `302`) | **{check-circle}** Yes | `/wardrobe.html /narnia.html 302` +| Rewrites (other status codes) | **{dotted-circle}** No | `/en/* /en/404.html 404` | +| [Splats](https://docs.netlify.com/routing/redirects/redirect-options/#splats) | **{dotted-circle}** No | `/news/* /blog/:splat` | +| Placeholders | **{dotted-circle}** No | `/news/:year/:month/:date/:slug /blog/:year/:month/:date/:slug` | +| Query parameters | **{dotted-circle}** No | `/store id=:id /blog/:id 301` | +| Force ([shadowing](https://docs.netlify.com/routing/redirects/rewrites-proxies/#shadowing)) | **{dotted-circle}** No | `/app/ /app/index.html 200!` | +| Domain-level redirects | **{dotted-circle}** No | `http://blog.example.com/* https://www.example.com/blog/:splat 301` | +| Redirect by country or language | **{dotted-circle}** No | `/ /anz 302 Country=au,nz` | +| Redirect by role | **{dotted-circle}** No | `/admin/* 200! Role=admin` | + +NOTE: **Note:** +Supported paths must start with a forward slash `/`. + +## Create redirects + +To create redirects after [enabling](#enable-or-disable-redirects) the feature, +create a configuration file named `_redirects` in the `public/` directory of your +GitLab Pages site. + +If your GitLab Pages site uses the default domain name (such as +`namespace.gitlab.io/projectname`) you must prefix every rule with the project name: + +```plaintext +/projectname/redirect-portal.html /projectname/magic-land.html 301 +/projectname/cake-portal.html /projectname/still-alive.html 302 +/projectname/wardrobe.html /projectname/narnia.html 302 +/projectname/pit.html /projectname/spikes.html 302 +``` + +If your GitLab Pages site uses [custom domains](custom_domains_ssl_tls_certification/index.md), +no project name prefix is needed. For example, if your custom domain is `example.com`, +your `_redirect` file would look like: + +```plaintext +/redirect-portal.html /magic-land.html 301 +/cake-portal.html /still-alive.html 302 +/wardrobe.html /narnia.html 302 +/pit.html /spikes.html 302 +``` + +## Files override redirects + +Files take priority over redirects. If a file exists on disk, GitLab Pages serves +the file instead of your redirect. For example, if the files `hello.html` and +`world.html` exist, and the `_redirects` file contains the following line, the redirect +is ignored because `hello.html` exists: + +```plaintext +/projectname/hello.html /projectname/world.html 302 +``` + +NOTE: **Note:** +GitLab does not support Netlify's +[force option](https://docs.netlify.com/routing/redirects/rewrites-proxies/#shadowing) +to change this behavior. + +## Debug redirect rules + +If a redirect isn't working as expected, or you want to check your redirect syntax, visit +`https://[namespace.gitlab.io]/projectname/_redirects`, replacing `[namespace.gitlab.io]` with +your domain name. The `_redirects` file isn't served directly, but your browser +displays a numbered list of your redirect rules, and whether the rule is valid or invalid: + +```plaintext +11 rules +rule 1: valid +rule 2: valid +rule 3: error: splats are not supported +rule 4: valid +rule 5: error: placeholders are not supported +rule 6: valid +rule 7: error: no domain-level redirects to outside sites +rule 8: error: url path must start with forward slash / +rule 9: error: no domain-level redirects to outside sites +rule 10: valid +rule 11: valid +``` + +## Enable or disable redirects + +Redirects in GitLab Pages is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. + +For [Omnibus installations](../../../administration/pages/index.md), define the +`FF_ENABLE_REDIRECTS` environment variable in the +[global settings](../../../administration/pages/index.md#global-settings). +Add the following line to `/etc/gitlab/gitlab.rb` and +[reconfigure the instance](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + +```ruby +gitlab_pages['env']['FF_ENABLE_REDIRECTS'] = 'true' +``` + +For [source installations](../../../administration/pages/source.md), define the +`FF_ENABLE_REDIRECTS` environment variable, then +[restart GitLab](../../../administration/restart_gitlab.md#installations-from-source): + +```shell +export FF_ENABLE_REDIRECTS="true" +/path/to/pages/bin/gitlab-pages -config gitlab-pages.conf +``` diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 20880d0c4fa..6c8aacd12b3 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -100,7 +100,7 @@ To edit the details of a release: 1. Click **Save changes**. You can edit the release title, notes, associated milestones, and asset links. -To change other release information, such as the tag or release date, use the +To change the release date use the [Releases API](../../../api/releases/index.md#update-a-release). ## Add release notes to Git tags @@ -322,13 +322,14 @@ The four types of links are "Runbook," "Package," "Image," and "Other." > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26019) in GitLab 12.6. Each time a release is created, GitLab takes a snapshot of data that's related to it. -This data is saved in a JSON file and called *release evidence*. It includes linked milestones -and issues and can facilitate internal processes like external audits. +This data is saved in a JSON file and called *release evidence*. The feature currently +includes test artifacts and linked milestones (and will include issues) to facilitate +internal processes, like external audits. To access the release evidence, on the Releases page, click the link to the JSON file that's listed under the **Evidence collection** heading. -You can also [use the API](../../../api/releases/index.md#collect-release-evidence-premium-only) to +You can also [use the API](../../../api/releases/index.md#collect-release-evidence) to generate release evidence for an existing release. Because of this, each release can have multiple release evidence snapshots. You can view the release evidence and its details on the Releases page. @@ -400,7 +401,7 @@ Here is an example of a release evidence object: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199065) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. -When a release is created, release evidence is automatically collected. To initiate evidence collection any other time, use an [API call](../../../api/releases/index.md#collect-release-evidence-premium-only). You can collect release evidence multiple times for one release. +When a release is created, release evidence is automatically collected. To initiate evidence collection any other time, use an [API call](../../../api/releases/index.md#collect-release-evidence). You can collect release evidence multiple times for one release. Evidence collection snapshots are visible on the Releases page, along with the timestamp the evidence was collected. diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index 54979d1c4ce..a937b6ed959 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -60,7 +60,7 @@ against accidental deletion and forced pushes. > - It's enabled on GitLab.com. > - It cannot be enabled or disabled per-project. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-custom-initial-branch-name-core-only). **(CORE ONLY)** +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-custom-initial-branch-name). **(CORE ONLY)** By default, when you create a new project in GitLab, the initial branch is called `master`. For self-managed instances, a GitLab administrator can customize the initial branch name to something diff --git a/doc/user/project/repository/git_blame.md b/doc/user/project/repository/git_blame.md index 6636463722e..a423f58ba21 100644 --- a/doc/user/project/repository/git_blame.md +++ b/doc/user/project/repository/git_blame.md @@ -8,7 +8,7 @@ description: "Documentation on Git file blame." # Git file blame -> [Introduced](https://git.sphere.ly/staff/publicgitlab/commit/39c657930625ddc3ac8a921f01ffc83acadce68f) in GitLab 2.5 +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/commit/39c657930625ddc3ac8a921f01ffc83acadce68f) in GitLab 2.5. [Git blame](https://git-scm.com/docs/git-blame) provides more information about every line in a file, including the last modified time, author, and diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 5526828c969..646d708d896 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -44,7 +44,7 @@ If you don't already have a GPG key, the following steps will help you get started: 1. [Install GPG](https://www.gnupg.org/download/index.html) for your operating system. - If your Operating System has `gpg2` installed, replace `gpg` with `gpg2` in + If your operating system has `gpg2` installed, replace `gpg` with `gpg2` in the following commands. 1. Generate the private/public key pair with the following command, which will spawn a series of questions: diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index a57806cf3ff..536cae263b8 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -190,7 +190,7 @@ updated every 15 minutes at most, so may not reflect recent activity. The displa The project size may differ slightly from one instance to another due to compression, housekeeping, and other factors. [Repository size limit](../../admin_area/settings/account_and_limit_settings.md) may be set by admins. -GitLab.com's repository size limit [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). +GitLab.com's repository size limit [is set by GitLab](../../gitlab_com/index.md#account-and-limit-settings). ## Contributors diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 8247f69c61a..28fdda07b05 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -14,11 +14,9 @@ Git repositories become larger over time. When large files are added to a Git re - Git repository storage limits [can be reached](#storage-limits). Rewriting a repository can remove unwanted history to make the repository smaller. -[`git filter-repo`](https://github.com/newren/git-filter-repo) is a tool for quickly rewriting Git -repository history, and is recommended over both: - -- [`git filter-branch`](https://git-scm.com/docs/git-filter-branch). -- [BFG](https://rtyley.github.io/bfg-repo-cleaner/). +We **recommend [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/README.md)** +over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and +[BFG](https://rtyley.github.io/bfg-repo-cleaner/). DANGER: **Danger:** Rewriting repository history is a destructive operation. Make sure to backup your repository before @@ -37,7 +35,7 @@ other internal references (refs) that are automatically created by GitLab. These - `refs/merge-requests/*` for merge requests. - `refs/pipelines/*` for - [pipelines](../../../ci/pipelines/index.md#troubleshooting-fatal-reference-is-not-a-tree). + [pipelines](../../../ci/troubleshooting.md#fatal-reference-is-not-a-tree-error). - `refs/environments/*` for environments. Git doesn't usually download these refs to make cloning and fetch faster, but we can use the `--mirror` option to @@ -49,7 +47,7 @@ download all the advertised refs. 1. Clone a fresh copy of the repository using `--bare` and `--mirror`: ```shell - git clone --bare --mirror https://example.gitlab.com/my/project.git + git clone --bare --mirror https://gitlab.example.com/my/project.git ``` 1. Using `git filter-repo`, purge any files from the history of your repository. @@ -252,9 +250,9 @@ When using repository cleanup, note: Repository size limits: -- Can [be set by an administrator](../../admin_area/settings/account_and_limit_settings.md#repository-size-limit-starter-only) +- Can [be set by an administrator](../../admin_area/settings/account_and_limit_settings.md#account-and-limit-settings) on self-managed instances. **(STARTER ONLY)** -- Are [set for GitLab.com](../../gitlab_com/index.md#repository-size-limit). +- Are [set for GitLab.com](../../gitlab_com/index.md#account-and-limit-settings). When a project has reached its size limit, you cannot: @@ -303,14 +301,9 @@ This process is not suitable for removing sensitive data like password or keys f Information about commits, including file content, is cached in the database, and will remain visible even after they have been removed from the repository. -<!-- ## Troubleshooting +## Troubleshooting -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. +### Incorrect repository statistics shown in the GUI -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +If the displayed size or commit number is different from the exported `.tar.gz` or local repository, +you can ask a GitLab administrator to [force an update](../../../administration/troubleshooting/gitlab_rails_cheat_sheet.md#incorrect-repository-statistics-shown-in-the-gui). diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index bbb673b74b0..e1d2c20850b 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -11,7 +11,8 @@ Repository mirroring allows for mirroring of repositories to and from external s used to mirror branches, tags, and commits between repositories. A repository mirror at GitLab will be updated automatically. You can also manually trigger an update -at most once every 5 minutes. +at most once every 5 minutes. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/237891) +for discussions on how to potentially reduce the delay. ## Overview @@ -45,6 +46,11 @@ The following are some possible use cases for repository mirroring: - You have old projects in another source that you don't use actively anymore, but don't want to remove for archiving purposes. In that case, you can create a push mirror so that your active GitLab repository can push its changes to the old location. +- You are a GitLab self-managed user for privacy reasons and your instance is closed to the public, + but you still have certain software components that you want open sourced. In this case, utilizing + GitLab to be your primary repository which is closed from the public, and using push mirroring to a + GitLab.com repository that's public, allows you to open source specific projects and contribute back + to the open source community. ## Pushing to a remote repository **(CORE)** @@ -67,7 +73,7 @@ When push mirroring is enabled, only push commits directly to the mirrored repos mirror diverging. All changes will end up in the mirrored repository whenever: - Commits are pushed to GitLab. -- A [forced update](#forcing-an-update-core) is initiated. +- A [forced update](#forcing-an-update) is initiated. Changes pushed to files in the repository are automatically pushed to the remote mirror at least: @@ -247,7 +253,7 @@ directly to the repository on GitLab. Instead, any commits should be pushed to t Changes pushed to the upstream repository will be pulled into the GitLab repository, either: - Automatically within a certain period of time. -- When a [forced update](#forcing-an-update-core) is initiated. +- When a [forced update](#forcing-an-update) is initiated. CAUTION: **Caution:** If you do manually update a branch in the GitLab repository, the branch will become diverged from @@ -393,17 +399,17 @@ failed. This will become visible in either the: - Pull mirror settings page. When a project is hard failed, it will no longer get picked up for mirroring. A user can resume the -project mirroring again by [Forcing an update](#forcing-an-update-core). +project mirroring again by [Forcing an update](#forcing-an-update). ### Trigger update using API **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3453) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. Pull mirroring uses polling to detect new branches and commits added upstream, often minutes -afterwards. If you notify GitLab by [API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter), +afterwards. If you notify GitLab by [API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project), updates will be pulled immediately. -For more information, see [Start the pull mirroring process for a Project](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter). +For more information, see [Start the pull mirroring process for a Project](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project). ## Forcing an update **(CORE)** @@ -425,8 +431,8 @@ them and how they will be resolved. Rewriting any mirrored commit on either remote will cause conflicts and mirroring to fail. This can be prevented by: -- [Pulling only protected branches](#only-mirror-protected-branches-starter). -- [Pushing only protected branches](#push-only-protected-branches-core). +- [Pulling only protected branches](#only-mirror-protected-branches). +- [Pushing only protected branches](#push-only-protected-branches). You should [protect the branches](../protected_branches.md) you wish to mirror on both remotes to prevent conflicts caused by rewriting history. @@ -439,13 +445,13 @@ protected branches. ### Configure a webhook to trigger an immediate pull to GitLab -Assuming you have already configured the [push](#setting-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) and [pull](#pulling-from-a-remote-repository-starter) mirrors in the upstream GitLab instance, to trigger an immediate pull as suggested above, you will need to configure a [Push Event Web Hook](../integrations/webhooks.md#push-events) in the downstream instance. +Assuming you have already configured the [push](#setting-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) and [pull](#pulling-from-a-remote-repository) mirrors in the upstream GitLab instance, to trigger an immediate pull as suggested above, you will need to configure a [Push Event Web Hook](../integrations/webhooks.md#push-events) in the downstream instance. To do this: - Create a [personal access token](../../profile/personal_access_tokens.md) with `API` scope. - Navigate to **Settings > Webhooks** -- Add the webhook URL which in this case will use the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project-starter) request to trigger an immediate pull after updates to the repository. +- Add the webhook URL which in this case will use the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) request to trigger an immediate pull after updates to the repository. ```plaintext https://gitlab.example.com/api/v4/projects/:id/mirror/pull?private_token=<your_access_token> diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index 452955b327c..af0daaaeca2 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -8,8 +8,8 @@ type: howto # GitLab Web Editor Sometimes it's easier to make quick changes directly from the GitLab interface -than to clone the project and use the Git command line tool. In this feature -highlight we look at how you can create a new file, directory, branch or +than to clone the project and use the Git command-line tool. In this feature +highlight, we look at how you can create a new file, directory, branch, or tag from the file browser. All of these actions are available from a single dropdown menu. @@ -17,13 +17,12 @@ dropdown menu. From a project's files page, click the '+' button to the right of the branch selector. Choose **New file** from the dropdown. -  -Enter a file name in the **File name** box. Then, add file content in the editor +Enter a file name in the **Filename** box. Then, add file content in the editor area. Add a descriptive commit message and choose a branch. The branch field will default to the branch you were viewing in the file browser. If you enter -a new branch name, a checkbox will appear allowing you to start a new merge +a new branch name, a checkbox will appear, allowing you to start a new merge request after you commit the changes. When you are satisfied with your new file, click **Commit Changes** at the bottom. @@ -32,14 +31,14 @@ When you are satisfied with your new file, click **Commit Changes** at the botto ### Template dropdowns -When starting a new project, there are some common files which the new project +When starting a new project, there are some common files that the new project might need too. Therefore a message will be displayed by GitLab to make this easy for you.  -When clicking on either `LICENSE` or `.gitignore`, etc., a dropdown will be displayed -to provide you with a template which might be suitable for your project. +When clicking on either `LICENSE` or `.gitignore` and so on, a dropdown will be displayed +to provide you with a template that might be suitable for your project.  @@ -56,16 +55,16 @@ least add a file in order for the button to show up. ## Upload a file The ability to create a file is great when the content is text. However, this -doesn't work well for binary data such as images, PDFs or other file types. In -this case you need to upload a file. +doesn't work well for binary data such as images, PDFs, or other file types. In +this case, you need to upload a file. From a project's files page, click the '+' button to the right of the branch selector. Choose **Upload file** from the dropdown.  -Once the upload dialog pops up there are two ways to upload your file. Either -drag and drop a file on the pop up or use the **click to upload** link. A file +Once the upload dialog pops up, there are two ways to upload your file. Either +drag and drop a file on the popup or use the **click to upload** link. A file preview will appear once you have selected a file to upload. Enter a commit message, choose a branch, and click **Upload file** when you are @@ -83,7 +82,7 @@ Choose **New directory** from the dropdown.  -In the new directory dialog enter a directory name, a commit message and choose +In the new directory dialog, enter a directory name, a commit message, and choose the target branch. Click **Create directory** to finish.  @@ -108,7 +107,7 @@ name or a referenced merge request or your project has an active fork relationship. If you would like to make this button appear, a possible workaround is to [remove your project's fork relationship](../settings/index.md#removing-a-fork-relationship). Once removed, the fork -relationship cannot be restored and you will no longer be able to send merge requests to the source. +relationship cannot be restored, and you will no longer be able to send merge requests to the source.  @@ -117,9 +116,9 @@ This dropdown contains the options **Create merge request and branch** and **Cre  Once you choose one of these options, a new branch or branch and merge request -will be created, based on the default -branch of your project, by default `master`. The branch name will be based on -the title of the issue and as a prefix, it will have its internal ID. Thus, the example +will be created based on the default +branch of your project (by default, `master`). The branch name will be based on +the title of the issue, and as a prefix, it will have its internal ID. Thus, the example screenshot above will create a branch named `2-make-static-site-auto-deploy-and-serve`. @@ -141,13 +140,13 @@ merge request is merged. ### Create a new branch from a project's dashboard If you want to make changes to several files before creating a new merge -request, you can create a new branch up front. From a project's files page, +request, you can create a new branch upfront. From a project's files page, choose **New branch** from the dropdown.  Enter a new **Branch name**. Optionally, change the **Create from** field -to choose which branch, tag or commit SHA this new branch will originate from. +to choose which branch, tag, or commit SHA this new branch will originate from. This field will autocomplete if you start typing an existing branch or tag. Click **Create branch** and you will be returned to the file browser on this new branch. @@ -155,7 +154,7 @@ branch.  You can now make changes to any files, as needed. When you're ready to merge -the changes back to master you can use the widget at the top of the screen. +the changes back to master, you can use the widget at the top of the screen. This widget only appears for a period of time after you create the branch or modify files. @@ -172,15 +171,15 @@ SHA. From a project's files page, choose **New tag** from the dropdown. Give the tag a name such as `v1.0.0`. Choose the branch or SHA from which you would like to create this new tag. You can optionally add a message and release notes. The release notes section supports Markdown format and you can -also upload an attachment. Click **Create tag** and you will be taken to the tag +also upload an attachment. Click **Create tag**, and you will be taken to the tag list page.  ## Tips -When creating or uploading a new file, or creating a new directory, you can -trigger a new merge request rather than committing directly to master. Enter +When creating or uploading a new file or creating a new directory, you can +trigger a new merge request rather than committing directly to `master`. Enter a new branch name in the **Target branch** field. You will notice a checkbox appear that is labeled **Start a new merge request with these changes**. After you commit the changes you will be taken to a new merge request form. diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index ae22dbc7e72..9d7d3914905 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -95,7 +95,7 @@ You can also sort the requirements list by: > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/215514) ability to specify individual requirements and their statuses in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. GitLab supports [requirements test -reports](../../../ci/pipelines/job_artifacts.md#artifactsreportsrequirements-ultimate) now. +reports](../../../ci/pipelines/job_artifacts.md#artifactsreportsrequirements) now. You can add a job to your CI pipeline that, when triggered, marks all existing requirements as Satisfied. diff --git a/doc/user/project/settings/img/cve_id_request_toggle.png b/doc/user/project/settings/img/cve_id_request_toggle.png Binary files differnew file mode 100644 index 00000000000..53ec804922c --- /dev/null +++ b/doc/user/project/settings/img/cve_id_request_toggle.png diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index a65e48d5d56..395d4bf30c5 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -37,6 +37,9 @@ You can select a framework label to identify that your project has certain compl - SOC 2 - Service Organization Control 2 - SOX - Sarbanes-Oxley +NOTE: **Note:** +Compliance framework labels do not affect your project settings. + ### Sharing and permissions For your repository, you can set up features such as public access, repository features, @@ -75,7 +78,7 @@ Some features depend on others: - If you disable the **Issues** option, GitLab also removes the following features: - **Issue Boards** - - [**Service Desk**](#service-desk-starter) + - [**Service Desk**](#service-desk) NOTE: **Note:** When the **Issues** option is disabled, you can still access **Milestones** @@ -97,6 +100,16 @@ Some features depend on others: - Metrics dashboard access requires reading both project environments and deployments. Users with access to the metrics dashboard can also access environments and deployments. +#### Disabling the CVE ID request button + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41203) in GitLab 13.4, only for public projects on GitLab.com. + +In applicable environments, a [**Create CVE ID Request** button](../../application_security/cve_id_request.md) +is present in the issue sidebar. The button may be disabled on a per-project basis by toggling the +setting **Enable CVE ID requests in the issue sidebar**. + + + #### Disabling email notifications Project owners can disable all [email notifications](../../profile/notifications.md#gitlab-notification-emails) @@ -234,10 +247,10 @@ This action: - Deletes a project including all associated resources (issues, merge requests etc). - From [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) on [Premium or Silver](https://about.gitlab.com/pricing/) or higher tiers, -group admins can [configure](../../group/index.md#enabling-delayed-project-removal-premium) projects within a group +group admins can [configure](../../group/index.md#enabling-delayed-project-removal) projects within a group to be deleted after a delayed period. When enabled, actual deletion happens after number of days -specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay-premium-only). +specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). CAUTION: **Warning:** The default behavior of [Delayed Project deletion](https://gitlab.com/gitlab-org/gitlab/-/issues/32935) in GitLab 12.6 was changed to diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index cbc4895f014..57cb610a2e9 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -15,10 +15,7 @@ type: reference, howto > - It's recommended for production use. > - For GitLab self-managed instances, GitLab administrators can [disable it](#enable-or-disable-project-access-tokens). -Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). - -<!-- Commented out until https://gitlab.com/gitlab-org/gitlab/-/issues/219551 is fixed --> -<!-- You can also use project access tokens with Git to authenticate over HTTP or SSH. --> +Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). You can also use project access tokens with Git to authenticate over HTTP or SSH. Project access tokens expire on the date you define, at midnight UTC. @@ -43,7 +40,8 @@ For each project access token created, a bot user will also be created and added For the bot: - The name is set to the name of the token. -- The username is set to `project_{project_id}_bot`, such as `project_123_bot`. +- The username is set to `project_{project_id}_bot` for the first access token, such as `project_123_bot`. +- The username is set to `project_{project_id}_bot{bot_count}` for further access tokens, such as `project_123_bot1`. API calls made with a project access token are associated with the corresponding bot user. @@ -54,7 +52,8 @@ When the project access token is [revoked](#revoking-a-project-access-token) the records will be moved to a system-wide user with the username "Ghost User". For more information, see [Associated Records](../../profile/account/delete_account.md#associated-records). -Project bot users are a [GitLab-created service account](../../../subscriptions/index.md#self-managed) and do not count as a licensed seat. +Project bot users are a [GitLab-created service account](../../../subscriptions/self_managed/index.md#choose-the-number-of-users), but count as a licensed seat. +These users will not count against your licensed seat in the future when [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/223695) is resolved. ## Revoking a project access token diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index 4e401014122..ce14cefba92 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -82,7 +82,7 @@ or [create a new project from a template](../../../gitlab-basics/create-project. 1. Visit your website and look at the bottom-left corner of the screen to see the new **Edit this page** button. Anyone satisfying the [requirements](#requirements) will be able to edit the -content of the pages without prior knowledge of Git nor of your site's +content of the pages without prior knowledge of Git or of your site's codebase. NOTE: **Note:** diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 12ba55cafdc..821b42af049 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -53,21 +53,42 @@ If you are missing Syntax Highlighting support for any language, we prepared a s NOTE: **Note:** Single file editing is based on the [Ace Editor](https://ace.c9.io). -### Schema based validation +### Themes + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in GitLab in 13.0. +> - Full Solarized Dark Theme [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219228) in GitLab 13.1. + +All the themes GitLab supports for syntax highlighting are added to the Web IDE's code editor. +You can pick a theme from your [profile preferences](../../profile/preferences.md). + +The themes are available only in the Web IDE file editor, except for the [dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/209808) and +the [solarized dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/219228), +which apply to the entire Web IDE screen. + +| Solarized Light Theme | Solarized Dark Theme | Dark Theme | +|---------------------------------------------------------------|-------------------------------------------------------------|-----------------------------------------| +|  |  |  | + +## Schema based validation -> - Support for `.gitlab-ci.yml` validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218472) in GitLab 13.2. +> - Support for validation based on predefined schemas [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218472) in GitLab 13.2. > - It was deployed behind a feature flag, disabled by default. > - It's enabled on GitLab.com. > - It cannot be enabled or disabled per-project. -> - For GitLab self-managed instances, GitLab administrators can opt to [enable it](#enable-or-disable-schema-based-validation-core-only). +> - For GitLab self-managed instances, GitLab administrators can opt to [enable it](#enable-or-disable-validation-based-on-predefined-schemas). +> - Support for validation based on custom schemas [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/226982) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. The Web IDE provides validation support for certain JSON and YAML files using schemas -based on the [JSON Schema Store](https://www.schemastore.org/json/). This feature is -only supported for the `.gitlab-ci.yml` file. +based on the [JSON Schema Store](https://www.schemastore.org/json/). -#### Enable or disable Schema based validation **(CORE ONLY)** +### Predefined schemas -Schema based validation is under development and not ready for production use. It is +The Web IDE has validation for certain files built in. This feature is only supported for +the `*.gitlab-ci.yml` files. + +#### Enable or disable validation based on predefined schemas **(CORE ONLY)** + +Validation based on predefined schemas is under development and not ready for production use. It is deployed behind a feature flag that is **disabled by default** for self-managed instances, [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can enable it for your instance. @@ -84,21 +105,35 @@ To disable it: Feature.disable(:schema_linting) ``` -### Themes +### Custom schemas **(PREMIUM)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in GitLab in 13.0. -> - Full Solarized Dark Theme [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219228) in GitLab 13.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/226982) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. -All the themes GitLab supports for syntax highlighting are added to the Web IDE's code editor. -You can pick a theme from your [profile preferences](../../profile/preferences.md). +The Web IDE also allows you to define custom schemas for certain JSON/YAML files in your project. +You can do so by defining a `schemas` entry in the `.gitlab/.gitlab-webide.yml` file inside the +repository's root. Here is an example configuration: -The themes are available only in the Web IDE file editor, except for the [dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/209808) and -the [solarized dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/219228), -which apply to the entire Web IDE screen. +```yaml +schemas: + - uri: https://json.schemastore.org/package + match: + - package.json + - uri: https://somewebsite.com/first/raw/url + match: + - data/release_posts/unreleased/*.{yml,yaml} + - uri: https://somewebsite.com/second/raw/url + match: + - "*.meta.json" +``` -| Solarized Light Theme | Solarized Dark Theme | Dark Theme | -|---------------------------------------------------------------|-------------------------------------------------------------|-----------------------------------------| -|  |  |  | +Each schema entry supports two properties: + +- `uri`: please provide an absolute URL for the schema definition file here. The schema from this URL +is loaded when a matching file is open. +- `match`: a list of matching paths or glob expressions. If a schema matches a particular path pattern, +it will be applied to that file. Please enclose the pattern in quotes if it begins with an asterisk (`*`), +it's be applied to that file. If a pattern begins with an asterisk (`*`), enclose it in quotation +marks. Otherwise, the configuration file is not valid YAML. ## Configure the Web IDE @@ -236,12 +271,12 @@ below. CAUTION: **Warning:** Interactive Web Terminals for the Web IDE is currently in **Beta**. -Shared Runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab/-/issues/24674), -so you would need to use your own private Runner(s) to make use of this feature. +Shared runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab/-/issues/24674), +so you would need to use your own private runner to make use of this feature. [Interactive Web Terminals](../../../ci/interactive_web_terminal/index.md) give the project [Maintainers](../../permissions.md#project-members-permissions) -user access to a terminal to interact with the Runner directly from +user access to a terminal to interact with the runner directly from GitLab, including through the Web IDE. ### Runner configuration @@ -249,7 +284,7 @@ GitLab, including through the Web IDE. Some things need to be configured in the runner for the interactive web terminal to work: -- The Runner needs to have +- The runner needs to have [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section). This section requires at least a `session_timeout` value (which defaults to 1800 seconds) and a `listen_address` value. If `advertise_address` is not defined, `listen_address` is used. @@ -346,7 +381,7 @@ environment. NOTE: **Note:** Only file changes in the Web IDE are synced to the terminal. Changes made in the terminal are **not** synced to the Web IDE. -This feature is only available for Kubernetes Runners. +This feature is only available for Kubernetes runners. To enable file syncing to the web terminal, the `.gitlab/.gitlab-webide.yml` file needs to have a `webide-file-sync` service configured. Here is an example @@ -373,7 +408,7 @@ terminal: more information. - `$CI_PROJECT_DIR` is a [predefined environment variable](../../../ci/variables/predefined_variables.md) - for GitLab Runners. This is where your project's repository will be. + for GitLab Runner. This is where your project's repository will be. Once you have configured the web terminal for file syncing, then when the web terminal is started, a **Terminal** status will be visible in the status bar. diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 5503cd97628..40ef5e216fd 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -165,7 +165,7 @@ Similar to versioned diff file views, you can see the changes made in a given Wi > - Git events were [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216014) in **GitLab 13.0.** > - It's enabled on GitLab.com. > - Git access activity creation is managed by a feature flag. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-in-git-core-only). **(CORE ONLY)** +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-in-git). **(CORE ONLY)** Wiki events (creation, deletion, and updates) are tracked by GitLab and displayed on the [user profile](../../profile/index.md#user-profile), diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index 820d66e4cd6..53ec8b35631 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -5,12 +5,12 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Advanced Global Search **(STARTER)** +# Advanced Search **(STARTER)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109) in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4. NOTE: **GitLab.com availability:** -Advanced Global Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. +Advanced Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. Leverage Elasticsearch for faster, more advanced code search across your entire GitLab instance. @@ -20,7 +20,7 @@ visit the [administrator documentation](../../integration/elasticsearch.md). ## Overview -The Advanced Global Search in GitLab is a powerful search service that saves +The Advanced Search in GitLab is a powerful search service that saves you time. Instead of creating duplicate code and wasting time, you can now search for code within other projects that can help your own project. @@ -39,12 +39,15 @@ searching in: ## Use cases -The Advanced Global Search can be useful in various scenarios. +The Advanced Search can be useful in various scenarios. ### Faster searches If you are dealing with huge amount of data and want to keep GitLab's search -fast, the Advanced Global Search will help you achieve that. +fast, Advanced Search will help you achieve that. + +NOTE: **Note:** +Between versions 12.10 and 13.4, Advanced Search response times have improved by 80%. ### Promote innersourcing @@ -58,13 +61,13 @@ throughout the GitLab instance and find the code they search for. Just use the search as before and GitLab will show you matching code from each project you have access to. - + -You can also use the [Advanced Syntax Search](advanced_search_syntax.md) which +You can also use the [Advanced Search Syntax](advanced_search_syntax.md) which provides some useful queries. NOTE: **Note:** Elasticsearch has only data for the default branch. That means that if you go to the repository tree and switch the branch from the default to something else, -then the "Code" tab in the search result page will be served by the regular +then the "Code" tab in the search result page will be served by the basic search even if Elasticsearch is enabled. diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index d8fce3223ed..804d4c540ac 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -5,12 +5,12 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference --- -# Advanced Syntax Search **(STARTER)** +# Advanced Search Syntax **(STARTER)** > - Introduced in [GitLab Enterprise Starter](https://about.gitlab.com/pricing/) 9.2 NOTE: **GitLab.com availability:** -Advanced Global Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. +Advanced Search (powered by Elasticsearch) is enabled for Bronze and above on GitLab.com since 2020-07-10. Use advanced queries for more targeted search results. @@ -19,11 +19,11 @@ visit the [administrator documentation](../../integration/elasticsearch.md). ## Overview -The Advanced Syntax Search is a subset of the -[Advanced Global Search](advanced_global_search.md), which you can use if you +The Advanced Search Syntax is a subset of the +[Advanced Search](advanced_global_search.md), which you can use if you want to have more specific search results. -Advanced Global Search only supports searching the [default branch](../project/repository/branches/index.md#default-branch). +Advanced Search only supports searching the [default branch](../project/repository/branches/index.md#default-branch). ## Use cases @@ -38,26 +38,26 @@ not so sure. In that case, using the advanced search syntax in your query will yield much better results. -## Using the Advanced Syntax Search +## Using the Advanced Search Syntax -The Advanced Syntax Search supports fuzzy or exact search queries with prefixes, +The Advanced Search Syntax supports fuzzy or exact search queries with prefixes, boolean operators, and much more. Full details can be found in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/5.3/query-dsl-simple-query-string-query.html#_simple_query_string_syntax), but here's a quick guide: - Searches look for all the words in a query, in any order - e.g.: searching - issues for `display bug` will return all issues matching both those words, in any order. -- To find the exact phrase (stemming still applies), use double quotes: `"display bug"` -- To find bugs not mentioning display, use `-`: `bug -display` -- To find a bug in display or sound, use `|`: `bug display | sound` -- To group terms together, use parentheses: `bug | (display +sound)` -- To match a partial word, use `*`: `bug find_by_*` -- To find a term containing one of these symbols, use `\`: `argument \-last` + issues for [`display bug`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=display+bug&group_id=9970&project_id=278964) and [`bug display`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=bug+Display&group_id=9970&project_id=278964) will return the same results. +- To find the exact phrase (stemming still applies), use double quotes: [`"display bug"`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=%22display+bug%22&group_id=9970&project_id=278964) +- To find bugs not mentioning display, use `-`: [`bug -display`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=bug+-display&group_id=9970&project_id=278964) +- To find a bug in display or banner, use `|`: [`bug display | banner`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=bug+display+%7C+banner&group_id=9970&project_id=278964) +- To group terms together, use parentheses: [`bug | (display +banner)`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=bug+%7C+%28display+%2Bbanner%29&group_id=9970&project_id=278964) +- To match a partial word, use `*`. In this example, I want to find bugs with any 500 errors. : [`bug error 50*`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=bug+error+50*&group_id=9970&project_id=278964) +- To use one of symbols above literally, escape the symbol with a preceding `\`: [`argument \-last`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=argument+%5C-last&group_id=9970&project_id=278964) ### Syntax search filters -The Advanced Syntax Search also supports the use of filters. The available filters are: +The Advanced Search Syntax also supports the use of filters. The available filters are: - filename: Filters by filename. You can use the glob (`*`) operator for fuzzy matching. - path: Filters by path. You can use the glob (`*`) operator for fuzzy matching. @@ -68,11 +68,11 @@ To use them, simply add them to your query in the format `<filter_name>:<value>` Examples: -- Finding a file with any content named `hello_world.rb`: `* filename:hello_world.rb` -- Finding a file named `hello_world` with the text `whatever` inside of it: `whatever filename:hello_world` -- Finding the text 'def create' inside files with the `.rb` extension: `def create extension:rb` -- Finding the text `sha` inside files in a folder called `encryption`: `sha path:encryption` -- Finding any file starting with `hello` containing `world` and with the `.js` extension: `world filename:hello* extension:js` +- Finding a file with any content named `search_results.rb`: [`* filename:search_results.rb`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=*+filename%3Asearch_results.rb&group_id=9970&project_id=278964) +- Finding a file named `found_blob_spec.rb` with the text `CHANGELOG` inside of it: [`CHANGELOG filename:found_blob_spec.rb](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=CHANGELOG+filename%3Afound_blob_spec.rb&group_id=9970&project_id=278964) +- Finding the text `EpicLinks` inside files with the `.rb` extension: [`EpicLinks extension:rb`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=EpicLinks+extension%3Arb&group_id=9970&project_id=278964) +- Finding the text `Sidekiq` in a file, when that file is in a path that includes `elastic`: [`Sidekiq path:elastic`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=Sidekiq+path%3Aelastic&group_id=9970&project_id=278964) +- Syntax filters can be combined for complex filtering. Finding any file starting with `search` containing `eventHub` and with the `.js` extension: [`eventHub filename:search* extension:js`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=eventHub+filename%3Asearch*+extension%3Ajs&group_id=9970&project_id=278964) #### Excluding filters @@ -86,7 +86,7 @@ Filters can be inversed to **filter out** results from the result set, by prefix Examples: -- Finding `rails` in all files but `Gemfile.lock`: `rails -filename:Gemfile.lock` -- Finding `success` in all files excluding `.po|pot` files: `success -filename:*.po*` -- Finding `import` excluding minified JavaScript (`.min.js`) files: `import -extension:min.js` -- Finding `docs` for all files outside the `docs/` folder: `docs -path:docs/` +- Finding `rails` in all files but `Gemfile.lock`: [`rails -filename:Gemfile.lock`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=rails+-filename%3AGemfile.lock&group_id=9970&project_id=278964) +- Finding `success` in all files excluding `.po|pot` files: [`success -filename:*.po*`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=success+-filename%3A*.po*&group_id=9970&project_id=278964) +- Finding `import` excluding minified JavaScript (`.min.js`) files: [`import -extension:min.js`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=import+-extension%3Amin.js&group_id=9970&project_id=278964) +- Finding `docs` for all files outside the `docs/` folder: [`docs -path:docs/`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=docs+-path%3Adocs%2F&group_id=9970&project_id=278964) diff --git a/doc/user/search/img/project_code_search.png b/doc/user/search/img/project_code_search.png Binary files differnew file mode 100644 index 00000000000..5412f614a74 --- /dev/null +++ b/doc/user/search/img/project_code_search.png diff --git a/doc/user/search/img/project_search_dropdown.png b/doc/user/search/img/project_search_dropdown.png Binary files differnew file mode 100644 index 00000000000..e0b922a186b --- /dev/null +++ b/doc/user/search/img/project_search_dropdown.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 98b34fe8383..475a72385ac 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -49,7 +49,7 @@ groups: - My-reaction - Confidential - Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in GitLab 12.9), - including [child epic](../group/epics/index.md#multi-level-child-epics-ultimate) + including [child epic](../group/epics/index.md#multi-level-child-epics) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0) - Search for this text @@ -129,6 +129,14 @@ characters to begin your search. For example, if you want to search for issues that have the assignee "Simone Presley", you'll need to type at least "Sim" before autocomplete gives any relevant results. +## Code search + +To search through code or other documents in a single project, you can use +the search field on the top-right of your screen while the project page is open. + + + + ## Search history You can view recent searches by clicking on the little arrow-clock icon, which is to the left of the search input. Click the search entry to run that search again. This feature is available for issues and merge requests. Searches are stored locally in your browser. @@ -154,6 +162,17 @@ quickly access issues and merge requests created or assigned to you within that  +### Autocomplete suggestions + +You can also type in this search bar to see autocomplete suggestions for: + +- Projects and groups +- Various help pages (try and type **API help**) +- Project feature pages (try and type **milestones**) +- Various settings pages (try and type **user settings**) +- Recently viewed issues (try and type some word from the title of a recently viewed issue) +- Recently viewed merge requests (try and type some word from the title of a recently merge request) + ## To-Do List Your [To-Do List](../todos.md#gitlab-to-do-list) can be searched by "to do" and "done". @@ -200,15 +219,15 @@ and **Labels**, select multiple issues to add to a list of your choice:  -## Advanced Global Search **(STARTER)** +## Advanced Search **(STARTER)** Leverage Elasticsearch for faster, more advanced code search across your entire GitLab instance. -[Learn how to use the Advanced Global Search.](advanced_global_search.md) +[Learn how to use the Advanced Search.](advanced_global_search.md) -## Advanced Syntax Search **(STARTER)** +## Advanced Search Syntax **(STARTER)** Use advanced queries for more targeted search results. -[Learn how to use the Advanced Syntax Search.](advanced_search_syntax.md) +[Learn how to use the Advanced Search Syntax.](advanced_search_syntax.md) diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index 314fe367ca6..c34d5be5899 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -27,7 +27,7 @@ These shortcuts are available in most areas of GitLab | <kbd>Shift</kbd> + <kbd>a</kbd> | Go to your Activity page. | | <kbd>Shift</kbd> + <kbd>l</kbd> | Go to your Milestones page. | | <kbd>Shift</kbd> + <kbd>s</kbd> | Go to your Snippets page. | -| <kbd>s</kbd> | Put cursor in the issues/merge requests search. | +| <kbd>s</kbd> / <kbd>/</kbd> | Put cursor in the search bar. | | <kbd>Shift</kbd> + <kbd>i</kbd> | Go to your Issues page. | | <kbd>Shift</kbd> + <kbd>m</kbd> | Go to your Merge requests page.| | <kbd>Shift</kbd> + <kbd>t</kbd> | Go to your To-Do List page. | @@ -40,6 +40,13 @@ for example comments, replies, issue descriptions, and merge request description | ---------------------------------------------------------------------- | ----------- | | <kbd>↑</kbd> | Edit your last comment. You must be in a blank text field below a thread, and you must already have at least one comment in the thread. | | <kbd>⌘</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | Toggle Markdown preview, when editing text in a text field that has **Write** and **Preview** tabs at the top. | +| <kbd>⌘</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>b</kbd> | Bold the selected text (surround it with `**`). | +| <kbd>⌘</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>i</kbd> | Italicize the selected text (surround it with `_`). | +| <kbd>⌘</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>k</kbd> | Add a link (surround the selected text with `[]()`). | + +NOTE: **Note:** +The shortcuts for editing in text fields are always enabled, even when +other keyboard shortcuts are disabled as explained above. ## Project diff --git a/doc/user/todos.md b/doc/user/todos.md index 02fef07a89a..1fca3c0ab64 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -5,67 +5,75 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab To-Do List +# GitLab To-Do List **(CORE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/2817) in GitLab 8.5. -When you log into GitLab, you normally want to see where you should spend your -time, take some action, or know what you need to keep an eye on without -a huge pile of e-mail notifications. GitLab is where you do your work, -so being able to get started quickly is important. +When you sign in to GitLab, you normally want to determine where you should +spend your time. This can include taking an action, or keeping track of things +(without having to read lots of email notifications). Because GitLab is where you +do your work, being able to get started quickly is important. -Your To-Do List offers a chronological list of items that are waiting for your input, all -in a simple dashboard. +Your *To-Do List* offers a chronological list of items waiting for your input +(known as *to-dos*) in a single dashboard. - +The To-Do List supports tracking [actions](#what-triggers-a-to-do) related to +the following: -You can quickly access your To-Do List by clicking the checkmark icon next to the -search bar in the top navigation. If the count is: +- Issues +- Merge Requests +- Epics **(ULTIMATE)** -- Less than 100, the number in blue is the number of To-Do items. -- 100 or more, the number displays as 99+. The exact number displays - on the To-Do List. -you still have open. Otherwise, the number displays as 99+. The exact number -displays on the To-Do List. + + +You can access your To-Do List by clicking the **{task-done}** To-Do List icon +next to the search bar in the top navigation. If the to-do item count is: + +- *Less than 100*, the number in blue is the number of to-do items. +- *100 or more*, the number displays as 99+. The exact number displays in the + To-Do List.  -## What triggers a To Do +## What triggers a to-do -A To Do appears on your To-Do List when: +A to-do item appears on your To-Do List when: -- An issue or merge request is assigned to you -- You are `@mentioned` in the description or comment of an: - - Issue - - Merge Request - - Epic **(ULTIMATE)** +- An issue or merge request is assigned to you. +- You're `@mentioned` in the description or comment of an issue or merge request + (or epic **(ULTIMATE)**). - You are `@mentioned` in a comment on a: - Commit - Design -- The CI/CD pipeline for your merge request failed -- An open merge request becomes unmergeable due to conflict, and one of the following is true: - - You are the author - - You are the user that set it to automatically merge once the pipeline succeeds -- [Since GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12136), a merge request - is removed from a [merge train](../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md) - and you are the user that added it. **(PREMIUM)** - -When multiple trigger actions occur for the same user on the same object (for example, an issue) -only the first is displayed as a single to-do on their To-Do List. - -To-do triggers are not affected by [GitLab Notification Email settings](profile/notifications.md). +- The CI/CD pipeline for your merge request failed. +- An open merge request becomes unmergeable due to conflict, and one of the + following is true: + - You're the author. + - You're the user that set the merge request to automatically merge after a + pipeline succeeds. +- [Since GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12136), a + merge request is removed from a + [merge train](../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md), + and you're the user that added it. **(PREMIUM)** + +When several trigger actions occur for the same user on the same object (for +example, an issue), GitLab displays only the first action as a single to-do +item. + +To-do triggers aren't affected by [GitLab notification email settings](profile/notifications.md). NOTE: **Note:** -When a user no longer has access to a resource related to a To Do (like an issue, merge request, -project, or group) the related To-Do items are deleted within the next hour for security reasons. -The delete is delayed to prevent data loss, in case the user's access was revoked by mistake. +When a user no longer has access to a resource related to a to-do (such as an +issue, merge request, project, or group), for security reasons GitLab deletes +any related to-do items within the next hour. Deletion is delayed to prevent +data loss, in the case where a user's access is accidentally revoked. -### Directly addressing a To Do +### Directly addressing a to-do > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7926) in GitLab 9.0. -If you are mentioned at the start of a line, the To Do you receive will be listed -as 'directly addressed'. For example, in this comment: +If you're mentioned at the start of a line, the to-do you receive will be listed +as *directly addressed*. For example, in the following comment: ```markdown @alice What do you think? cc: @bob @@ -79,81 +87,71 @@ as 'directly addressed'. For example, in this comment: @erin @frank thank you! ``` -The people receiving directly addressed To-Do items are `@alice`, `@erin`, and -`@frank`. Directly addressed To-Do items only differ from mentions in their type +The people receiving directly addressed to-do items are `@alice`, `@erin`, and +`@frank`. Directly addressed to-do items only differ from mentions in their type for filtering purposes; otherwise, they appear as normal. -### Manually creating a To Do +### Manually creating a to-do -You can also add the following to your To-Do List by clicking the **Add a To Do** button on an: - -- Issue -- Merge Request -- Epic **(ULTIMATE)** +You can add an issue or merge request (or epic **(ULTIMATE)**) to your +To-Do List by clicking its **Add a To Do** button.  -## Marking a To Do as done - -Any action to the following will mark the corresponding To Do as done: +## Marking a to-do as done -- Issue -- Merge Request -- Epic **(ULTIMATE)** +Any action to an issue or merge request (or epic **(ULTIMATE)**) will mark its +corresponding to-do as done. -Actions that dismiss To-Do items include: +Actions that dismiss to-do items include: - Changing the assignee - Changing the milestone - Adding/removing a label - Commenting on the issue -Your To-Do List is personal, and items are only marked as done if the action comes from -you. If you close the issue or merge request, your To Do is automatically -marked as done. - -To prevent other users from closing issues without you being notified, if someone else closes, merges, or takes action on the any of the following, your To Do will remain pending: - -- Issue -- Merge request -- Epic **(ULTIMATE)** +Your To-Do List is personal, and items are only marked as done if you take +action. If you close the issue or merge request, your to-do is marked as done. -There is just one To Do for each of these, so mentioning a user a hundred times in an issue will only trigger one To Do. +To prevent other users from closing issues without you being notified, if +someone else closes, merges, or takes action on an issue or merge request (or +epic **(ULTIMATE)**), your to-do will remain pending. -If no action is needed, you can manually mark the To Do as done by clicking the -corresponding **Done** button, and it will disappear from your To-Do List. +There's just one to-do for each of these, so mentioning a user many times in an +issue will only trigger one to-do item. - +If no action is needed, you can manually mark the to-do as done by clicking its +corresponding **Done** button to have GitLab remove the item from your +To-Do List. -You can also mark a To Do as done by clicking the **Mark as done** button in the sidebar of the following: + -- Issue -- Merge Request -- Epic **(ULTIMATE)** +You can also mark a to-do as done by clicking the **Mark as done** button in the +sidebar of an issue or merge request (or epic **(ULTIMATE)**).  -You can mark all your To-Do items as done at once by clicking the **Mark all as -done** button. +You can mark all your to-do items as done at once by clicking the +**Mark all as done** button. ## Filtering your To-Do List -There are four kinds of filters you can use on your To-Do List. +You can use the following types of filters with your To-Do List: -| Filter | Description | -| ------- | ----------- | -| Project | Filter by project | -| Group | Filter by group | -| Author | Filter by the author that triggered the To Do | -| Type | Filter by issue, merge request, design, or epic **(ULTIMATE)** | -| Action | Filter by the action that triggered the To Do | +| Filter | Description | +| ------- | ---------------------------------------------------------------- | +| Project | Filter by project. | +| Group | Filter by group. | +| Author | Filter by the author that triggered the To Do. | +| Type | Filter by issue, merge request, design, or epic. **(ULTIMATE)** | +| Action | Filter by the action that triggered the To Do. | -You can also filter by more than one of these at the same time. The possible Actions are -[described above](#what-triggers-a-to-do) and include: +You can also filter by more than one of these at the same time. The previously +described [triggering actions](#what-triggers-a-to-do) include: -- Any Action +- Any action - Assigned - Mentioned - Added - Pipelines -- Directly Addressed +- Directly addressed |
