diff options
Diffstat (limited to 'doc/user')
239 files changed, 3843 insertions, 2723 deletions
diff --git a/doc/user/admin_area/abuse_reports.md b/doc/user/admin_area/abuse_reports.md index 0283f1a9eff..4c346d7dfe8 100644 --- a/doc/user/admin_area/abuse_reports.md +++ b/doc/user/admin_area/abuse_reports.md @@ -20,6 +20,9 @@ To receive notifications of new abuse reports by e-mail, follow these steps: 1. Expand the **Abuse reports** section. 1. Provide an email address. +The notification email address can also be set and retrieved +[using the API](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). + ## Reporting abuse To find out more about reporting abuse, see [abuse reports user @@ -31,14 +34,14 @@ To access abuse reports, go to **Admin Area > Abuse Reports**. There are 3 ways to resolve an abuse report, with a button for each method: -- Remove user & report. This will: - - [Delete the reported user](../profile/account/delete_account.md) from the +- Remove user & report. This: + - [Deletes the reported user](../profile/account/delete_account.md) from the instance. - - Remove the abuse report from the list. + - Removes the abuse report from the list. - [Block user](#blocking-users). -- Remove report. This will: - - Remove the abuse report from the list. - - Remove access restrictions for the reported user. +- Remove report. This: + - Removes the abuse report from the list. + - Removes access restrictions for the reported user. The following is an example of the **Abuse Reports** page: @@ -54,8 +57,7 @@ Blocking a user: - Leaves them in the abuse report list. - Changes the **Block user** button to a disabled **Already blocked** button. -The user will be notified with the -[following message](https://gitlab.com/gitlab-org/gitlab/blob/master/app/workers/email_receiver_worker.rb#L38): +The user is notified with the following message: ```plaintext Your account has been blocked. If you believe this is in error, contact a staff member. diff --git a/doc/user/admin_area/activating_deactivating_users.md b/doc/user/admin_area/activating_deactivating_users.md index 29f162616bf..8b3a7682841 100644 --- a/doc/user/admin_area/activating_deactivating_users.md +++ b/doc/user/admin_area/activating_deactivating_users.md @@ -39,7 +39,7 @@ A user can be deactivated from the Admin Area. To do this: Please note that for the deactivation option to be visible to an admin, the user: - Must be currently active. -- Must not have signed in, or have any activity, in the last 180 days. +- Must not have signed in, or have any activity, in the last 90 days. Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). diff --git a/doc/user/admin_area/analytics/img/cohorts_v13_4.png b/doc/user/admin_area/analytics/img/cohorts_v13_4.png Binary files differindex 4af1841a033..6f7dd5101f2 100644 --- a/doc/user/admin_area/analytics/img/cohorts_v13_4.png +++ b/doc/user/admin_area/analytics/img/cohorts_v13_4.png diff --git a/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png b/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png Binary files differindex 1fa070a6915..d47d86cd514 100644 --- a/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png +++ b/doc/user/admin_area/analytics/img/dev_ops_report_v13_4.png diff --git a/doc/user/admin_area/analytics/index.md b/doc/user/admin_area/analytics/index.md index b3336b471f8..f79245c7325 100644 --- a/doc/user/admin_area/analytics/index.md +++ b/doc/user/admin_area/analytics/index.md @@ -4,7 +4,8 @@ Administrators have access to instance-wide analytics, as shown in **Admin Area > Analytics**. -There are two kinds of statistics: +There are several kinds of statistics: - [DevOps Report](dev_ops_report.md): Provides an overview of your entire instance's feature usage. +- [Instance Statistics](instance_statistics.md): Shows how much data your instance contains, and how that is changing. - [User Cohorts](user_cohorts.md): Display the monthly cohorts of new users and their activities over time. diff --git a/doc/user/admin_area/analytics/instance_statistics.md b/doc/user/admin_area/analytics/instance_statistics.md new file mode 100644 index 00000000000..bac0e845d2c --- /dev/null +++ b/doc/user/admin_area/analytics/instance_statistics.md @@ -0,0 +1,18 @@ +# Instance Statistics + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235754) in GitLab 13.4. + +Instance Statistics gives you an overview of how much data your instance contains, and how quickly this volume is changing over time. + +## Total counts + +At the top of the page, Instance Statistics shows total counts for: + +- Users +- Projects +- Groups +- Issues +- Merge Requests +- Pipelines + +These figures can be useful for understanding how much data your instance contains in total. diff --git a/doc/user/admin_area/approving_users.md b/doc/user/admin_area/approving_users.md new file mode 100644 index 00000000000..486d0b6a25d --- /dev/null +++ b/doc/user/admin_area/approving_users.md @@ -0,0 +1,36 @@ +--- +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: howto +--- + +# Users pending approval + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4491) in GitLab 13.5. + +When [Require admin approval for new sign-ups](settings/sign_up_restrictions.md#require-admin-approval-for-new-sign-ups) is enabled, any user that signs up for an account using the registration form is placed under a **Pending approval** state. + +A user pending approval is functionally identical to a [blocked](blocking_unblocking_users.md) user. + +A user pending approval: + +- Will not be able to sign in. +- Cannot access Git repositories or the API. +- Will not receive any notifications from GitLab. +- Does not consume a [seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). + +## Approving a user + +A user that is pending approval can be approved from the Admin Area. To do this: + +1. Navigate to **Admin Area > Overview > Users**. +1. Click on the **Pending approval** tab. +1. Select a user. +1. Under the **Account** tab, click **Approve user**. + +Approving a user: + +1. Activates their account. +1. Changes the user's state to active and it consumes a +[seat](../../subscriptions/self_managed/index.md#choose-the-number-of-users). diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 7f2d49dafea..b17d0ab3dd5 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -9,11 +9,9 @@ type: howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20912) in GitLab 12.6. -## Overview - GitLab administrators are responsible for the overall security of their instance. To assist, GitLab provides a Credentials inventory to keep track of all the credentials that can be used to access their self-managed instance. -Using Credentials inventory, you can see all the personal access tokens (PAT) and SSH keys that exist in your GitLab instance. In addition, you can [revoke them](#revoke-a-users-personal-access-token) and see: +Using Credentials inventory, you can see all the personal access tokens (PAT) and SSH keys that exist in your GitLab instance. In addition, you can [revoke](#revoke-a-users-personal-access-token) and [delete](#delete-a-users-ssh-key) and see: - Who they belong to. - Their access scope. @@ -29,7 +27,7 @@ The following is an example of the Credentials inventory page: ## Revoke a user's personal access token -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214811) in GitLab 13.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214811) in GitLab 13.4. If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: @@ -37,7 +35,15 @@ If you see a **Revoke** button, you can revoke that user's PAT. Whether you see |-------------|------------------------|--------------------|----------------------------------------------------------------------------| | Active | Yes | Yes | Allows administrators to revoke the PAT, such as for a compromised account | | Active | No | Yes | Allows administrators to revoke the PAT, such as for a compromised account | -| Expired | Yes | No | PAT expires automatically | -| Expired | No | Yes | The administrator may revoke the PAT to prevent indefinite use | -| Revoked | Yes | No | Not applicable; token is already revoked | -| Revoked | No | No | Not applicable; token is already revoked | +| Expired | Yes | No | PAT expires automatically | +| Expired | No | Yes | The administrator may revoke the PAT to prevent indefinite use | +| Revoked | Yes | No | Not applicable; token is already revoked | +| Revoked | No | No | Not applicable; token is already revoked | + +## Delete a user's SSH key + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225248) in GitLab 13.5. + +You can **Delete** a user's SSH key by navigating to the credentials inventory's SSH Keys tab. + + diff --git a/doc/user/admin_area/img/admin_wrench.png b/doc/user/admin_area/img/admin_wrench.png Binary files differdeleted file mode 100644 index 17eee143e87..00000000000 --- a/doc/user/admin_area/img/admin_wrench.png +++ /dev/null diff --git a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png b/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png Binary files differnew file mode 100644 index 00000000000..f5edf513bbf --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 58430ab615b..0ddbe17580a 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -118,8 +118,15 @@ To list users matching a specific criteria, click on one of the following tabs o - **[Deactivated](activating_deactivating_users.md)** - **Without projects** -For each user, their username, email address, are listed, also the date their account was -created and the date of last activity. To edit a user, click the **Edit** button in that user's +For each user, the following are listed: + +1. Username +1. Email address +1. Project membership count +1. Date of account creation +1. Date of last activity + +To edit a user, click the **Edit** button in that user's row. To delete the user, or delete the user and their contributions, click the cog dropdown in that user's row, and select the desired option. diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index ecbc615f56a..c37a61d6748 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -8,130 +8,126 @@ type: howto # Activate GitLab EE with a license **(STARTER ONLY)** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload -a license. Once you've received your license from GitLab Inc., you can upload it -by **signing into your GitLab instance as an admin** or add it at +a license. After you've received your license from GitLab Inc., you can upload it +by **signing into your GitLab instance as an admin** or adding it at installation time. -The license has the form of a base64 encoded ASCII text with a `.gitlab-license` -extension and can be obtained when you [purchase one](https://about.gitlab.com/pricing/) or when you sign -up for a [free trial](https://about.gitlab.com/free-trial/). +The license is a base64-encoded ASCII text file with a `.gitlab-license` +extension. You can obtain the file by [purchasing a license](https://about.gitlab.com/pricing/) +or by signing up for a [free trial](https://about.gitlab.com/free-trial/). -NOTE: **Note:** As of GitLab Enterprise Edition 9.4.0, a newly-installed instance without an -uploaded license will only have the Core features active. A trial license will -activate all Ultimate features, but after +uploaded license only has the Core features active. A trial license +activates all Ultimate features, but after [the trial expires](#what-happens-when-your-license-expires), some functionality -will be locked. +is locked. ## Uploading your license The very first time you visit your GitLab EE installation signed in as an admin, you should see a note urging you to upload a license with a link that takes you -straight to **Admin Area > License**. +to **Admin Area > License**. Otherwise, you can: -1. Navigate manually to the **Admin Area** by clicking the wrench icon in the menu bar. +1. Navigate manually to the **Admin Area** by clicking the wrench (**{admin}**) icon in the menu bar. -  - -1. And then going to the **License** tab and click on **Upload New License**. +1. Navigate to the **License** tab, and click **Upload New License**.  -1. If you've received a `.gitlab-license` file, you should have already downloaded - it in your local machine. You can then upload it directly by choosing the - license file and clicking the **Upload license** button. In the image below, - you can see that the selected license file is named `GitLab.gitlab-license`. + - *If you've received a `.gitlab-license` file,* you should have already downloaded + it in your local machine. You can then upload it directly by choosing the + license file and clicking the **Upload license** button. In the image below, + the selected license file is named `GitLab.gitlab-license`. -  +  - If you've received your license as plain text, you need to select the - "Enter license key" option, copy the license, paste it into the "License key" - field and click **Upload license**. + - *If you've received your license as plain text,* select the + **Enter license key** option, copy the license, paste it into the **License key** + field, and click **Upload license**. ## Add your license at install time -A license can be automatically imported at install time, by placing a file named -`Gitlab.gitlab-license` in `/etc/gitlab/` for Omnibus, or `config/` for source installations. +A license can be automatically imported at install time by placing a file named +`Gitlab.gitlab-license` in `/etc/gitlab/` for Omnibus GitLab, or `config/` for source installations. -It is also possible to specify a custom location and filename for the license. +You can also specify a custom location and filename for the license: -Source installations should set the `GITLAB_LICENSE_FILE` environment -variable with the path to a valid GitLab Enterprise Edition license. +- Source installations should set the `GITLAB_LICENSE_FILE` environment + variable with the path to a valid GitLab Enterprise Edition license. -```shell -export GITLAB_LICENSE_FILE="/path/to/license/file" -``` + ```shell + export GITLAB_LICENSE_FILE="/path/to/license/file" + ``` -Omnibus installations should add this entry to `gitlab.rb`: +- Omnibus GitLab installations should add this entry to `gitlab.rb`: -```ruby -gitlab_rails['initial_license_file'] = "/path/to/license/file" -``` + ```ruby + gitlab_rails['initial_license_file'] = "/path/to/license/file" + ``` CAUTION: **Caution:** -These methods will only add a license at the time of installation. Use the -Admin Area in the web user interface to renew or upgrade licenses. +These methods only add a license at the time of installation. Use the +**{admin}** **Admin Area** in the web user interface to renew or upgrade licenses. --- -Once the license is uploaded, all GitLab Enterprise Edition functionality -will be active until the end of the license period. When that period ends, the +After the license is uploaded, all GitLab Enterprise Edition functionality +is active until the end of the license period. When that period ends, the instance will [fall back](#what-happens-when-your-license-expires) to Core-only functionality. -You can review the license details at any time in the License section of the -Admin Area. +You can review the license details at any time in the **License** section of the +**Admin Area**.  ## Notification before the license expires -One month before the license expires, a message informing when the expiration -is due to, will start appearing to GitLab admins. Make sure that you update your -license, otherwise you will miss all the paid features if it expires. +One month before the license expires, a message informing about the expiration +date is displayed to GitLab admins. Make sure that you update your +license, otherwise you miss all the paid features if your license expires. ## What happens when your license expires -In case your license expires, GitLab will lock down some features like Git pushes, -issue creation, etc., and a message to inform of the expired license will be -presented to all admins. +In case your license expires, GitLab locks down some features like Git pushes, +and issue creation, and displays a message to all admins to inform of the expired license. -In order to get back all the previous functionality, a new license must be uploaded. -To fall back to having only the Core features active, you'll need to delete the +To get back all the previous functionality, you must upload a new license. +To fall back to having only the Core features active, you must delete the expired license(s). ### Remove a license To remove a license from a self-managed instance: -1. Go to the [Admin Area](index.md) (click the wrench in the top navigation bar). +1. In the top navigation bar, click the **{admin}** wrench icon to navigate to the [Admin Area](index.md). 1. Click **License** in the left sidebar. 1. Click **Remove License**. ## License history -It's possible to upload and view more than one license, -but only the latest license will be used as the active license. +You can upload and view more than one license, +but only the latest license is used as the active license. ## Troubleshooting ### There is no License tab in the Admin Area -If you originally installed Community Edition rather than Enterprise Edition you will need to +If you originally installed Community Edition rather than Enterprise Edition you must [upgrade to Enterprise Edition](../../update/README.md#community-to-enterprise-edition) before uploading your license. -GitLab.com users cannot upload and use a self-managed license. If you -wish to use paid features on GitLab.com, a separate subscription may be -[purchased](../../subscriptions/gitlab_com/index.md). +GitLab.com users can't upload and use a self-managed license. If you +want to use paid features on GitLab.com, you can +[purchase a separate subscription](../../subscriptions/gitlab_com/index.md). ### Users exceed license limit upon renewal -If you've added new users to your GitLab instance prior to renewal you may need to -purchase additional seats to cover those users. If this is the case and a license -without enough users is uploaded a message is displayed prompting you to purchase +If you've added new users to your GitLab instance prior to renewal, you may need to +purchase additional seats to cover those users. If this is the case, and a license +without enough users is uploaded, GitLab displays a message prompting you to purchase additional users. More information on how to determine the required number of users and how to add additional seats can be found in the [licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). diff --git a/doc/user/admin_area/settings/gitaly_timeouts.md b/doc/user/admin_area/settings/gitaly_timeouts.md index 2003d02c9b3..cac05678a1a 100644 --- a/doc/user/admin_area/settings/gitaly_timeouts.md +++ b/doc/user/admin_area/settings/gitaly_timeouts.md @@ -3,36 +3,28 @@ stage: Create group: Gitaly info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference -type: reference --- -# Gitaly timeouts - - - -3 timeout types can be configured to make sure that long running -Gitaly calls don't needlessly take up resources. - -- Default timeout - -This timeout is the default for most Gitaly calls. -It should be shorter than the worker timeout that can be configured -for -[Puma](https://docs.gitlab.com/omnibus/settings/puma.html#puma-settings) -or [Unicorn](https://docs.gitlab.com/omnibus/settings/unicorn.html). -This makes sure that Gitaly calls made within a web request cannot -exceed these the entire request timeout. +# Gitaly timeouts **(CORE ONLY)** -The default for this timeout is 55 seconds. +[Gitaly](../../../administration/gitaly/index.md) timeouts are configurable. The timeouts can be +configured to make sure that long running Gitaly calls don't needlessly take up resources. -- Fast timeout +To access Gitaly timeout settings: -This is the timeout for very short Gitaly calls. +1. Go to **Admin Area > Settings > Preferences**. +1. Expand the **Gitaly** section. -The default for this timeout is 10 seconds. +## Available timeouts -- Medium timeout +The following timeouts can be modified: -This timeout should be between the default and the fast timeout +- **Default Timeout Period**. This timeout is the default for most Gitaly calls. It should be shorter than the + worker timeout that can be configured for [Puma](https://docs.gitlab.com/omnibus/settings/puma.html#puma-settings) + or [Unicorn](https://docs.gitlab.com/omnibus/settings/unicorn.html). Used to make sure that Gitaly + calls made within a web request cannot exceed the entire request timeout. + Defaults to 55 seconds. -The default for this timeout is 30 seconds. +- **Fast Timeout Period**. This is the timeout for very short Gitaly calls. Defaults to 10 seconds. +- **Medium Timeout Period**. This timeout should be between the default and the fast timeout. + Defaults to 30 seconds. diff --git a/doc/user/admin_area/settings/img/email_confirmation_v12_7.png b/doc/user/admin_area/settings/img/email_confirmation_v12_7.png Binary files differdeleted file mode 100644 index 6bcadb63b9a..00000000000 --- a/doc/user/admin_area/settings/img/email_confirmation_v12_7.png +++ /dev/null diff --git a/doc/user/admin_area/settings/img/gitaly_timeouts.png b/doc/user/admin_area/settings/img/gitaly_timeouts.png Binary files differdeleted file mode 100644 index 28394d238f7..00000000000 --- a/doc/user/admin_area/settings/img/gitaly_timeouts.png +++ /dev/null diff --git a/doc/user/admin_area/settings/img/sign_up_restrictions_v13_5.png b/doc/user/admin_area/settings/img/sign_up_restrictions_v13_5.png Binary files differnew file mode 100644 index 00000000000..ebbfad77e69 --- /dev/null +++ b/doc/user/admin_area/settings/img/sign_up_restrictions_v13_5.png diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index ba9bccbf3e7..bc8df63e33f 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -30,11 +30,11 @@ Access the default page for admin area settings by navigating to **Admin Area > | Option | Description | | ------ | ----------- | -| [Elasticsearch](../../../integration/elasticsearch.md#enabling-elasticsearch) | Elasticsearch integration. Elasticsearch AWS IAM. | +| [Elasticsearch](../../../integration/elasticsearch.md#enabling-advanced-search) | Elasticsearch integration. Elasticsearch AWS IAM. | | [PlantUML](../../../administration/integration/plantuml.md#gitlab) | Allow rendering of PlantUML diagrams in AsciiDoc documents. | | [Slack application](../../../user/project/integrations/gitlab_slack_application.md#configuration) **(FREE ONLY)** | Slack integration allows you to interact with GitLab via slash commands in a chat window. This option is only available on GitLab.com, though it may be [available for self-managed instances in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/28164). | | [Third party offers](third_party_offers.md) | Control the display of third party offers. | -| [Snowplow](../../../development/telemetry/snowplow.md) | Configure the Snowplow integration. | +| [Snowplow](../../../development/product_analytics/snowplow.md) | Configure the Snowplow integration. | | [Google GKE](../../project/clusters/add_gke_clusters.md) | Google GKE integration allows you to provision GKE clusters from GitLab. | | [Amazon EKS](../../project/clusters/add_eks_clusters.md) | Amazon EKS integration allows you to provision EKS clusters from GitLab. | @@ -61,7 +61,7 @@ Access the default page for admin area settings by navigating to **Admin Area > | ------ | ----------- | | [Continuous Integration and Deployment](continuous_integration.md) | Auto DevOps, runners and job artifacts. | | [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration) **(PREMIUM ONLY)** | Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/README.md). This pipeline configuration will be run after the project's own configuration. | -| [Package Registry](continuous_integration.md#package-registry-configuration) | Settings related to the use and experience of using GitLab's Package Registry. Note there are [risks involved](./../../packages/container_registry/index.md#use-with-external-container-registries) in enabling some of these settings. | +| [Package Registry](continuous_integration.md#package-registry-configuration) | Settings related to the use and experience of using GitLab's Package Registry. Note there are [risks involved](../../packages/container_registry/index.md#use-with-external-container-registries) in enabling some of these settings. | ## Reporting @@ -102,7 +102,7 @@ Access the default page for admin area settings by navigating to **Admin Area > | Option | Description | | ------ | ----------- | | [Email](email.md) | Various email settings. | -| [Help page](../../../customization/help_message.md) | Help page text and support page URL. | +| [Help page](help_page.md) | Help page text and support page URL. | | [Pages](../../../administration/pages/index.md#custom-domain-verification) | Size and domain settings for static websites | | [Real-time features](../../../administration/polling.md) | Change this value to influence how frequently the GitLab UI polls for updates. | | [Gitaly timeouts](gitaly_timeouts.md) | Configure Gitaly timeouts. | diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index 97380b93295..20f2812bc39 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -9,8 +9,6 @@ type: reference > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5986) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.3. -## Overview - In hosted systems, enterprises often have a need to share their own templates across teams. This feature allows an administrator to pick a project to be the instance-wide collection of file templates. These templates are then exposed to diff --git a/doc/user/admin_area/settings/project_integration_management.md b/doc/user/admin_area/settings/project_integration_management.md index e4fe7e36139..748d608676d 100644 --- a/doc/user/admin_area/settings/project_integration_management.md +++ b/doc/user/admin_area/settings/project_integration_management.md @@ -4,38 +4,53 @@ group: Ecosystem info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Project integration management **(CORE ONLY)** +# Project integration management -> [Introduced in](https://gitlab.com/groups/gitlab-org/-/epics/2137) GitLab 13.3. +Project integrations can be configured and enabled by project administrators. As a GitLab instance +administrator, you can set default configuration parameters for a given integration that all projects +can inherit and use. This enables the integration for all projects that are not already using custom +settings. -Project integrations can be configured and enabled by project administrators. As a GitLab instance administrator, you can set default configuration parameters for a given integration that all projects can inherit and use, enabling the integration for all projects that are not already using custom settings. +You can update these default settings at any time, changing the settings used for all projects that +are set to use instance-level defaults. Updating the default settings also enables the integration +for all projects that didn't have it already enabled. -You can update these default settings at any time, changing the settings in use for all projects that are set to use instance-level defaults. This also enables the integration for all projects on which it was not already enabled. +Only the complete settings for an integration can be inherited. Per-field inheritance is +[planned](https://gitlab.com/groups/gitlab-org/-/epics/2137) as is +[group-level management](https://gitlab.com/groups/gitlab-org/-/epics/2543) of integration settings. -It is only possible to inherit the complete settings for an integration. Per-field inheritance is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137), as well as [group-level management](https://gitlab.com/groups/gitlab-org/-/epics/2543) of integration settings. +## Manage instance-level default settings for a project integration **(CORE ONLY)** -## Manage default settings for a project integration +> [Introduced in](https://gitlab.com/groups/gitlab-org/-/epics/2137) GitLab 13.3. 1. Navigate to **Admin Area > Settings > Integrations**. 1. Select a project integration. 1. Enter configuration details and click **Save changes**. CAUTION: **Caution:** -This may affect all or most of the projects on your GitLab instance. Please review the details below. +This may affect all or most of the projects on your GitLab instance. Please review the details +below. If this is the first time you are setting up instance-level settings for an integration: -- The integration is enabled for all projects that do not already have this integration configured if you have the **Enable integration** toggle turned on in the instance-level settings. -- Projects that already have the integration configured are not affected, but can choose to use the inherited settings at any time. +- The integration is enabled for all projects that don't already have this integration configured, + if you have the **Enable integration** toggle turned on in the instance-level settings. +- Projects that already have the integration configured are not affected, but can choose to use the + inherited settings at any time. When you make further changes to the instance defaults: -- They are immediately applied to all projects that have the integration set to use instance-level default settings. -- They are immediately applied to newer projects, created since you last saved defaults for the integration. - - If your instance-level default setting has the **Enable integration** toggle turned on, the integration is automatically enabled for all such projects. -- Projects with custom settings selected for the integration are not immediately affected and may choose to use the latest instance-level defaults at any time. +- They are immediately applied to all projects that have the integration set to use default settings. +- They are immediately applied to newer projects, created since you last saved defaults for the + integration. If your instance-level default setting has the **Enable integration** toggle turned + on, the integration is automatically enabled for all such projects. +- Projects with custom settings selected for the integration are not immediately affected and may + choose to use the latest defaults at any time. -It is only possible to inherit the complete settings for an integration. Per-field inheritance is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow instance administrators to update settings inherited by projects without enabling the integration on all non-configured projects by default. +Only the complete settings for an integration can be inherited. Per-field inheritance +is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow +administrators to update settings inherited by projects without enabling the +integration on all non-configured projects by default. ## Use instance-level default settings for a project integration @@ -47,7 +62,7 @@ It is only possible to inherit the complete settings for an integration. Per-fie ## Use custom settings for a project integration -1. Navigate to **Project > Settings > Integrations**. +1. Navigate to project's **Settings > Integrations**. 1. Choose the integration you want to enable or update. 1. From the drop-down, select **Use custom settings**. 1. Ensure the toggle is set to **Enable integration** and enter all required settings. diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 80092102091..f57cf7c2045 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -7,6 +7,7 @@ type: reference You can use sign-up restrictions to: - Disable new sign-ups. +- Require admin approval for new sign-ups. - Require user email confirmation. - Denylist or allowlist email addresses belonging to specific domains. @@ -32,12 +33,20 @@ Alternatively, you could also consider setting up a [allowlist](#allowlist-email-domains) or [denylist](#denylist-email-domains) on email domains to prevent malicious users from creating accounts. +## Require admin approval for new sign-ups + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4491) in GitLab 13.5. + +When this setting is enabled, any user visiting your GitLab domain and signing up for a new account will have to be explicitly [approved](../approving_users.md#approving-a-user) by an administrator before they can start using their account. + + + ## Require email confirmation You can send confirmation emails during sign-up and require that users confirm their email address before they are allowed to sign in. - + ## Minimum password length limit diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 55bbcfbe1d8..140d149555a 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -58,7 +58,7 @@ sequenceDiagram ## Usage Ping **(CORE ONLY)** -See [Usage Ping guide](../../../development/telemetry/usage_ping.md). +See [Usage Ping guide](../../../development/product_analytics/usage_ping.md). ## Instance-level statistics **(CORE ONLY)** diff --git a/doc/user/analytics/img/mr_throughput_chart_v13_3.png b/doc/user/analytics/img/mr_throughput_chart_v13_3.png Binary files differindex 04fa54f323c..100c9a8557c 100644 --- a/doc/user/analytics/img/mr_throughput_chart_v13_3.png +++ b/doc/user/analytics/img/mr_throughput_chart_v13_3.png diff --git a/doc/user/analytics/img/mr_throughput_table_v13_3.png b/doc/user/analytics/img/mr_throughput_table_v13_3.png Binary files differindex 63ffb9389f4..bb63770dc3f 100644 --- a/doc/user/analytics/img/mr_throughput_table_v13_3.png +++ b/doc/user/analytics/img/mr_throughput_table_v13_3.png diff --git a/doc/user/analytics/img/new_value_stream_v13_3.png b/doc/user/analytics/img/new_value_stream_v13_3.png Binary files differindex 4284b8ab194..bc8502e85a6 100644 --- a/doc/user/analytics/img/new_value_stream_v13_3.png +++ b/doc/user/analytics/img/new_value_stream_v13_3.png diff --git a/doc/user/analytics/img/vsa_filter_bar_v13.3.png b/doc/user/analytics/img/vsa_filter_bar_v13.3.png Binary files differindex 71e59892434..506765f63cb 100644 --- a/doc/user/analytics/img/vsa_filter_bar_v13.3.png +++ b/doc/user/analytics/img/vsa_filter_bar_v13.3.png diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index 044b9eb3e64..54b4c702c5c 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -25,10 +25,8 @@ The following analytics features are available at the group level: - [Contribution](../group/contribution_analytics/index.md). **(STARTER)** - [Insights](../group/insights/index.md). **(ULTIMATE)** - [Issue](../group/issues_analytics/index.md). **(PREMIUM)** -- [Productivity](productivity_analytics.md), enabled with the `productivity_analytics` - [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(PREMIUM)** -- [Value Stream](value_stream_analytics.md), enabled with the `cycle_analytics` - [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(PREMIUM)** +- [Productivity](productivity_analytics.md) **(PREMIUM)** +- [Value Stream](value_stream_analytics.md). **(PREMIUM)** ## Project-level analytics @@ -40,6 +38,5 @@ The following analytics features are available at the project level: - [Issue](../group/issues_analytics/index.md). **(PREMIUM)** - [Merge Request](merge_request_analytics.md), enabled with the `project_merge_request_analytics` [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(STARTER)** -- [Repository](repository_analytics.md). -- [Value Stream](value_stream_analytics.md), enabled with the `cycle_analytics` - [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development). **(STARTER)** +- [Repository](repository_analytics.md). **(CORE)** +- [Value Stream](value_stream_analytics.md). **(CORE)** diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index 6a18d46fd1a..04a5fa71e19 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -73,18 +73,3 @@ The **Merge Request Analytics** feature can be accessed only: - On [Starter](https://about.gitlab.com/pricing/) and above. - By users with [Reporter access](../permissions.md) and above. - -## Enable and disable related feature flags - -Merge Request Analytics is disabled by default but can be enabled using the following -[feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-locally-in-development): - -- `project_merge_request_analytics` - -A GitLab administrator can: - -- Enable this feature by running the following command in a Rails console: - - ```ruby - Feature.enable(:project_merge_request_analytics) - ``` diff --git a/doc/user/analytics/productivity_analytics.md b/doc/user/analytics/productivity_analytics.md index 653836de8be..951be1c550b 100644 --- a/doc/user/analytics/productivity_analytics.md +++ b/doc/user/analytics/productivity_analytics.md @@ -13,7 +13,7 @@ Track development velocity with Productivity Analytics. For many companies, the development cycle is a black box and getting an estimate of how long, on average, it takes to deliver features is an enormous endeavor. -While [Value Stream Analytics](../project/cycle_analytics.md) focuses on the entire +While [Value Stream Analytics](../analytics/value_stream_analytics.md) focuses on the entire Software Development Life Cycle (SDLC) process, Productivity Analytics provides a way for Engineering Management to drill down in a systematic way to uncover patterns and causes for success or failure at an individual, project, or group level. Productivity can slow down for many reasons ranging from degrading code base to quickly growing teams. In order to investigate, department or team leaders can start by visualizing the time it takes for merge requests to be merged. @@ -62,29 +62,3 @@ The **Productivity Analytics** dashboard can be accessed only: - On [Premium or Silver tier](https://about.gitlab.com/pricing/) and above. - By users with [Reporter access](../permissions.md) and above. - -## Enabling and disabling using feature flags - -Productivity Analytics is: - -- [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18754) from GitLab 12.4, - but can be disabled using the following feature flags: - - `productivity_analytics`. - - `productivity_analytics_scatterplot_enabled`. -- Disabled by default in GitLab 12.3, but can be enabled using the following feature flag: - - `productivity_analytics`. - -A GitLab administrator can: - -- Disable this feature from GitLab 12.4 by running the follow in a Rails console: - - ```ruby - Feature.disable(:productivity_analytics) - Feature.disable(:productivity_analytics_scatterplot_enabled) - ``` - -- Enable this feature in GitLab 12.3 by running the following in a Rails console: - - ```ruby - Feature.enable(:productivity_analytics) - ``` diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 14012d4a28d..86525587b30 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -12,26 +12,26 @@ info: To determine the technical writer assigned to the Stage/Group associated w Value Stream Analytics measures the time spent to go from an [idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) -(also known as cycle time) for each of your projects. Value Stream Analytics displays the median time +(also known as cycle time) for each of your projects or groups. Value Stream Analytics displays the median time spent in each stage defined in the process. -For information on how to contribute to the development of Value Stream Analytics, see our [contributor documentation](../../development/value_stream_analytics.md). - Value Stream Analytics is useful in order to quickly determine the velocity of a given project. It points to bottlenecks in the development process, enabling management to uncover, triage, and identify the root cause of slowdowns in the software development life cycle. -Value Stream Analytics is tightly coupled with the [GitLab flow](../../topics/gitlab_flow.md) and -calculates a separate median for each stage. +For information on how to contribute to the development of Value Stream Analytics, see our [contributor documentation](../../development/value_stream_analytics.md). + +## Project Level Value Stream Analytics **(CORE)** + +Project Level Value Stream Analytics is available via **Project > Analytics > Value Stream**. -## Overview +## Group Level Value Stream Analytics **(PREMIUM)** -Value Stream Analytics is available: +From GitLab 12.9, group level Value Stream Analytics is available via **Group > Analytics > Value Stream**. -- From GitLab 12.9, at the group level via **Group > Analytics > Value Stream**. **(PREMIUM)** -- At the project level via **Project > Analytics > Value Stream**. +## Default stages -There are seven stages that are tracked as part of the Value Stream Analytics calculations. +The stages tracked by Value Stream Analytics by default represent the [GitLab flow](../../topics/gitlab_flow.md). These stages can be customized in Group Level Value Stream Analytics. - **Issue** (Tracker) - Time to schedule an issue (by milestone or by adding it to an issue board) @@ -96,8 +96,7 @@ Value Stream Analytics records stage time and data based on the project issues w exception of the staging stage, where only data deployed to production are measured. -Specifically, if your CI is not set up and you have not defined a `production` -or `production/*` [environment](../../ci/yaml/README.md#environment), then you will not have any +Specifically, if your CI is not set up and you have not defined a [production environment](#how-the-production-environment-is-identified), then you will not have any data for this stage. Each stage of Value Stream Analytics is further described in the table below. @@ -109,7 +108,7 @@ Each stage of Value Stream Analytics is further described in the table below. | Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. | | Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. | | Review | Measures the median time taken to review the merge request that has a closing issue pattern, between its creation and until it's merged. | -| Staging | Measures the median time between merging the merge request with a closing issue pattern until the very first deployment to production. It's tracked by the environment set to `production` or matching `production/*` (case-sensitive, `Production` won't work) in your GitLab CI/CD configuration. If there isn't a production environment, this is not tracked. | +| Staging | Measures the median time between merging the merge request with a closing issue pattern until the very first deployment to a [production environment](#how-the-production-environment-is-identified). If there isn't a production environment, this is not tracked. | How this works, behind the scenes: @@ -123,13 +122,23 @@ How this works, behind the scenes: we need for the stages, like issue creation date, merge request merge time, etc. -To sum up, anything that doesn't follow [GitLab flow](../../workflow/gitlab_flow.md) will not be tracked and the +To sum up, anything that doesn't follow [GitLab flow](../../topics/gitlab_flow.md) will not be tracked and the Value Stream Analytics dashboard will not present any data for: - Merge requests that do not close an issue. - Issues not labeled with a label present in the Issue Board or for issues not assigned a milestone. -- Staging stage, if the project has no `production` or `production/*` - environment. +- Staging stage, if the project has no [production environment](#how-the-production-environment-is-identified). + +## How the production environment is identified + +Value Stream Analytics identifies production environments by looking for project [environments](../../ci/yaml/README.md#environment) with a name matching any of these patterns: + +- `prod` or `prod/*` +- `production` or `production/*` + +These patterns are not case-sensitive. + +You can change the name of a project environment in your GitLab CI/CD configuration. ## Example workflow @@ -345,7 +354,7 @@ administrator can open a Rails console and disable it with the following command Feature.disable(:cycle_analytics_scatterplot_enabled) ``` -## Type of work - Tasks by type chart **(PREMIUM)** +## Type of work - Tasks by type chart > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32421) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. @@ -368,11 +377,8 @@ The current permissions on the Project Value Stream Analytics dashboard are: You can [read more about permissions](../../user/permissions.md) in general. -For Value Stream Analytics functionality introduced in GitLab 12.3 and later: - -- Users must have Reporter access or above. -- Features are available only on - [Premium or Silver tiers](https://about.gitlab.com/pricing/) and above. +For Value Stream Analytics functionality introduced in GitLab 12.3 and later, +users must have Reporter access or above. ## More resources diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index ae22655e30b..145422f8736 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -8,8 +8,8 @@ type: reference, howto # Web API Fuzz Testing **(ULTIMATE)** You can add web API fuzzing to your [GitLab CI/CD](../../../ci/README.md) -pipelines. This helps you discover bugs and potential security issues that other QA processes may miss. -API fuzzing performs fuzz testing of API operation parameters. +pipelines. This helps you discover bugs and potential security issues that other QA processes may miss. +API fuzzing performs fuzz testing of API operation parameters. Fuzz testing sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API backend. We recommend that you use fuzz testing in addition to [GitLab Secure](../index.md)'s @@ -443,7 +443,7 @@ Example usage for setting a single header: ```json { "headers": { - "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" } } ``` @@ -453,7 +453,7 @@ Example usage for setting both a header and cookie: ```json { "headers": { - "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" }, "cookies": { "flags": "677" diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index a6ad701360e..ead34ca227e 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -19,8 +19,9 @@ then in the left sidebar go to **Security & Compliance > Configuration**. For each security control the page displays: -- **Status** - Status of the security control: enabled, not enabled, or available. -- **Manage** - A management option or a link to the documentation. +- **Security Control:** Name, description, and a documentation link. +- **Status:** The security control's status (enabled, not enabled, or available). +- **Manage:** A management option or a documentation link. ## Status @@ -29,12 +30,11 @@ The status of each security control is determined by the project's latest defaul If a job with the expected security report artifact exists in the pipeline, the feature's status is _enabled_. -For SAST, click **View history** to see the `.gitlab-ci.yml` file’s history. - -NOTE: **Note:** If the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), all security features are configured by default. +For SAST, click **View history** to see the `.gitlab-ci.yml` file's history. + ## Manage You can configure the following security controls: diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 880e5a3875a..9e7f98dd4fc 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -9,8 +9,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3672) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. -## Overview - Your application's Docker image may itself be based on Docker images that contain known vulnerabilities. By including an extra job in your pipeline that scans for those vulnerabilities and displays them in a merge request, you can use GitLab to audit your Docker-based apps. @@ -19,7 +17,6 @@ By default, container scanning in GitLab is based on [Clair](https://github.com/ containers. [GitLab's Klar analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar/) scans the containers and serves as a wrapper for Clair. -NOTE: **Note:** To integrate security scanners other than Clair and Klar into GitLab, see [Security scanner integration](../../../development/integrations/secure.md). @@ -46,7 +43,7 @@ To enable container scanning in your pipeline, you need the following: or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. - Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the shared runners on GitLab.com, then this is already the case. -- [Build and push](../../packages/container_registry/index.md#container-registry-examples-with-gitlab-cicd) +- [Build and push](../../packages/container_registry/index.md#build-and-push-by-using-gitlab-cicd) your Docker image to your project's container registry. The name of the Docker image should use the following [predefined environment variables](../../../ci/variables/predefined_variables.md): @@ -65,7 +62,7 @@ To enable container scanning in your pipeline, you need the following: variables: IMAGE_TAG: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - docker build -t $IMAGE_TAG . - docker push $IMAGE_TAG ``` @@ -119,7 +116,7 @@ build: IMAGE: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA script: - docker info - - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - docker build -t $IMAGE . - docker push $IMAGE @@ -219,14 +216,21 @@ To use container scanning in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - To configure a local Docker container registry with copies of the container scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [container scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). -NOTE: **Note:** -GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. +##### Support for Custom Certificate Authorities + +Support for custom certificate authorities was introduced in the following versions: + +| Analyzer | Version | +| -------- | ------- | +| `klar` | [v2.3.0](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/releases/v2.3.0) | + #### Make GitLab container scanning analyzer images available inside your Docker registry For container scanning, import the following default images from `registry.gitlab.com` into your @@ -287,7 +291,7 @@ build_latest_vulnerabilities: script: - docker pull arminc/clair-db:latest - docker tag arminc/clair-db:latest $CI_REGISTRY/namespace/clair-vulnerabilities-db - - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - docker push $CI_REGISTRY/namespace/clair-vulnerabilities-db ``` @@ -433,3 +437,7 @@ This is a result of a bug in Docker which is now [fixed](https://github.com/cont To prevent the error, ensure the Docker version that the runner is using is `18.09.03` or higher. For more information, see [issue #10241](https://gitlab.com/gitlab-org/gitlab/-/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). + +### Getting warning message `gl-container-scanning-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index dff71cb9445..9508407ccae 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -175,6 +175,52 @@ To use coverage fuzzing in an offline environment, follow these steps: `NEW_URL_GITLAB_COV_FUZ` is the URL of the private `gitlab-cov-fuzz` clone that you set up in the first step. +### Continuous fuzzing (long-running async fuzzing jobs) + +It's also possible to run the fuzzing jobs longer and without blocking your main pipeline. This +configuration uses the GitLab [parent-child pipelines](../../../ci/parent_child_pipelines.md). +The full example is available in the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing#running-go-fuzz-from-ci). +This example uses Go, but is applicable for any other supported languages. + +The suggested workflow in this scenario is to have long-running, async fuzzing jobs on a +main/development branch, and short, blocking sync fuzzing jobs on all other branches and MRs. This +is a good way to balance the needs of letting a developer's per-commit pipeline complete quickly, +and also giving the fuzzer a large amount of time to fully explore and test the app. + +Long-running fuzzing jobs are usually necessary for the coverage guided fuzzer to find deeper bugs +in your latest code base. THe following is an example of what `.gitlab-ci.yml` looks like in this +workflow (for the full example, see the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing)): + +```yaml + +sync_fuzzing: + variables: + COVFUZZ_ADDITIONAL_ARGS: '-max_total_time=300' + trigger: + include: .covfuzz-ci.yml + strategy: depend + rules: + - if: $CI_COMMIT_BRANCH != 'continuous_fuzzing' && $CI_PIPELINE_SOURCE != 'merge_request_event' + +async_fuzzing: + variables: + COVFUZZ_ADDITIONAL_ARGS: '-max_total_time=3600' + trigger: + include: .covfuzz-ci.yml + rules: + - if: $CI_COMMIT_BRANCH == 'continuous_fuzzing' && $CI_PIPELINE_SOURCE != 'merge_request_event' +``` + +This essentially creates two steps: + +1. `sync_fuzzing`: Runs all your fuzz targets for a short period of time in a blocking + configuration. This finds simple bugs and allows you to be confident that your MRs aren't + introducing new bugs or causing old bugs to reappear. +1. `async_fuzzing`: Runs on your branch and finds deep bugs in your code without blocking your + development cycle and MRs. + +The `covfuzz-ci.yml` is the same as that in the [original synchronous example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example#running-go-fuzz-from-ci). + ### Glossary - Seed corpus: The set of test cases given as initial input to the fuzz target. This usually speeds diff --git a/doc/user/application_security/dast/img/dast_v13_2.png b/doc/user/application_security/dast/img/dast_v13_2.png Binary files differdeleted file mode 100644 index bbf7944eb40..00000000000 --- a/doc/user/application_security/dast/img/dast_v13_2.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_v13_4.png b/doc/user/application_security/dast/img/dast_v13_4.png Binary files differnew file mode 100644 index 00000000000..d9c1d1b5c66 --- /dev/null +++ b/doc/user/application_security/dast/img/dast_v13_4.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 73a8e727389..fffaf4ad26b 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -9,17 +9,17 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4348) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. -NOTE: **Note:** -The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) -explains how **4 of the top 6 attacks were application based**. Download it -to learn how to protect your organization. - Running [static checks](../sast/index.md) on your code is the first step to detect vulnerabilities that can put the security of your code at risk. Yet, once deployed, your application is exposed to a new category of possible attacks, such as cross-site scripting or broken authentication flaws. This is where Dynamic Application Security Testing (DAST) comes into place. +NOTE: **Note:** +The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) +explains how 4 of the top 6 attacks were application based. Download it to learn how to protect your +organization. + ## Overview If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web applications @@ -32,11 +32,10 @@ provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the DAST report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. -NOTE: **Note:** -This comparison logic uses only the latest pipeline executed for the target branch's base commit. -Running the pipeline on any other commit has no effect on the merge request. +Note that this comparison logic uses only the latest pipeline executed for the target branch's base +commit. Running the pipeline on any other commit has no effect on the merge request. - + By clicking on one of the detected linked vulnerabilities, you can see the details and the URL(s) affected. @@ -53,12 +52,11 @@ However, DAST can be [configured](#full-scan) to also perform an *active scan*: attack your application and produce a more extensive security report. It can be very useful combined with [Review Apps](../../../ci/review_apps/index.md). -NOTE: **Note:** -A pipeline may consist of multiple jobs, including SAST and DAST scanning. If any -job fails to finish for any reason, the security dashboard doesn't show DAST scanner -output. For example, if the DAST job finishes but the SAST job fails, the security -dashboard doesn't show DAST results. The analyzer outputs an -[exit code](../../../development/integrations/secure.md#exit-code) on failure. +Note that a pipeline may consist of multiple jobs, including SAST and DAST scanning. If any job +fails to finish for any reason, the security dashboard doesn't show DAST scanner output. For +example, if the DAST job finishes but the SAST job fails, the security dashboard doesn't show DAST +results. On failure, the analyzer outputs an +[exit code](../../../development/integrations/secure.md#exit-code). ## Use cases @@ -206,8 +204,8 @@ variables: DAST_FULL_SCAN_ENABLED: "true" ``` -NOTE: **Note:** -If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). +If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some +tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). #### Domain validation @@ -398,11 +396,9 @@ variables: DAST_API_HOST_OVERRIDE: api-test.host.com ``` -NOTE: **Note:** -Using a host override is ONLY supported when importing the API -specification from a URL. It does not work and will be ignored when importing -the specification from a file. This is due to a limitation in the ZAP OpenAPI -extension. +Note that using a host override is ONLY supported when importing the API specification from a URL. +It doesn't work and is ignored when importing the specification from a file. This is due to a +limitation in the ZAP OpenAPI extension. #### Authentication using headers @@ -427,7 +423,8 @@ A URL scan allows you to specify which parts of a website are scanned by DAST. #### Define the URLs to scan -To specify the paths to be scanned, add a comma-separated list of the paths to the `DAST_PATHS` environment variable. Note that you can only scan paths of a single host. +To specify the paths to scan, add a comma-separated list of the paths to the `DAST_PATHS` +environment variable. Note that you can only scan paths of a single host. ```yaml include: @@ -437,8 +434,10 @@ variables: DAST_PATHS=/page1.html,/category1/page1.html,/page3.html ``` -NOTE: **Note:** -`DAST_AUTH_EXCLUDE_URLS` are ignored when `DAST_PATHS` is set. +When using `DAST_PATHS`, note the following: + +- The `DAST_PATHS` environment variable has a limit of about 130kb. If you have a list or paths + greater than this, you should create multiple DAST jobs and split the paths over each job. #### Full Scan @@ -590,8 +589,7 @@ To use DAST in an offline environment, you need: [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). -NOTE: **Note:** -GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we @@ -672,11 +670,6 @@ To delete an existing site profile: ## Scanner profile > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4. -> - [Deployed behind a feature flag](../../feature_flags.md), enabled by default. -> - Enabled on GitLab.com. -> - Can be enabled or disabled per-project. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can [disable this feature](#enable-or-disable-dast-scanner-profiles). A scanner profile defines the scanner settings used to run an on-demand scan: @@ -684,6 +677,11 @@ A scanner profile defines the scanner settings used to run an on-demand scan: - **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site. - **Target timeout:** The maximum number of seconds DAST waits for the site to be available before starting the scan. +- **Scan mode:** A passive scan monitors all HTTP messages (requests and responses) sent to the target. An active scan attacks the target to find potential vulnerabilities. +- **AJAX spider:** Run the AJAX spider, in addition to the traditional spider, to crawl the target site. +- **Debug messages:** Include debug messages in the DAST console output. + +Scan mode, AJAX spider, Debug messages are [added in GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/225804) ### Create a scanner profile @@ -711,29 +709,6 @@ To delete a scanner profile: 1. Click **Manage** in the **DAST Profiles** row. 1. Click **{remove}** in the scanner profile's row. -### Enable or disable DAST scanner profiles - -The scanner profile feature is ready for production use. It's deployed behind a feature flag that -is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to disable it. - -To disable it: - -```ruby -# Instance-wide -Feature.disable(:security_on_demand_scans_scanner_profiles) -# or by project -Feature.disable(:security_on_demand_scans_scanner_profiles, Project.find(<project id>)) -``` - -To enable it: - -```ruby -# Instance-wide -Feature.enable(:security_on_demand_scans_scanner_profiles) -# or by project -Feature.enable(:security_on_demand_scans_scanner_profiles, Project.find(<project ID>)) -``` - ## On-demand scans > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. @@ -756,7 +731,8 @@ An on-demand DAST scan: NOTE: **Note:** You must have permission to run an on-demand DAST scan against a protected branch. -The default branch is automatically protected. For more details, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). +The default branch is automatically protected. For more information, see +[Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). To run an on-demand DAST scan, you need: @@ -765,8 +741,8 @@ To run an on-demand DAST scan, you need: 1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left sidebar. 1. Click **Create new DAST scan**. -1. In **Scanner settings**, select a scanner profile from the dropdown. -1. In **Site profiles**, select a site profile from the dropdown. +1. In **Scanner profile**, select a scanner profile from the dropdown. +1. In **Site profile**, select a site profile from the dropdown. 1. Click **Run scan**. The on-demand DAST scan runs and the project's dashboard shows the results. @@ -866,7 +842,7 @@ include: template: DAST.gitlab-ci.yml variables: - DAST_INCLUDE_ALPHA_VULNERABILITIES: true + DAST_INCLUDE_ALPHA_VULNERABILITIES: "true" ``` ## Interacting with the vulnerabilities @@ -923,6 +899,10 @@ Change the number after `-Xmx` to the required memory amount. If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). +### Getting warning message `gl-dast-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 5cce336d04c..b90bb37c60f 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -9,25 +9,26 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5105) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.7. -Dependency Scanning helps to find security vulnerabilities in your dependencies automatically -while you're developing and testing your applications, such as when your -application is using an external (open source) library that is known to be vulnerable. +GitLab's Dependency Scanning feature can automatically find security vulnerabilities in your +dependencies while you're developing and testing your applications. For example, dependency scanning +lets you know if your application uses an external (open source) library that is known to be +vulnerable. You can then take action to protect your application. ## Overview -If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your dependencies for known -vulnerabilities using Dependency Scanning. -All dependencies are scanned, including the transitive dependencies (also known as nested dependencies). -You can take advantage of Dependency Scanning by either [including the Dependency Scanning template](#configuration) -in your existing `.gitlab-ci.yml` file or by implicitly using -the [Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning) +If you're using [GitLab CI/CD](../../../ci/README.md), you can use dependency scanning to analyze +your dependencies for known vulnerabilities. GitLab scans all dependencies, including transitive +dependencies (also known as nested dependencies). You can take advantage of dependency scanning by +either [including the dependency scanning template](#configuration) +in your existing `.gitlab-ci.yml` file, or by implicitly using +the [auto dependency scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning) provided by [Auto DevOps](../../../topics/autodevops/index.md). -GitLab checks the Dependency Scanning report, compares the found vulnerabilities +GitLab checks the dependency scanning report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. - + The results are sorted by the severity of the vulnerability: @@ -40,7 +41,7 @@ The results are sorted by the severity of the vulnerability: ## Requirements -To run Dependency Scanning jobs, by default, you need GitLab Runner with the +To run dependency scanning jobs, by default, you need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. If you're using the shared runners on GitLab.com, this is enabled by default. @@ -56,24 +57,25 @@ The current detection logic limits the maximum search depth to two levels. For e The following languages and dependency managers are supported: -| Language (package managers) | Supported files | Scan tool(s) | -|----------------------------- | --------------- | ------------ | -| C# .NET ([NuGet](https://www.nuget.org/) 4.9+) | [`packages.lock.json`](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#enabling-lock-file) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| C/C++ ([Conan](https://conan.io/)) | [`conan.lock`](https://docs.conan.io/en/latest/versioning/lockfiles.html) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Java ([Gradle](https://gradle.org/), [Maven](https://maven.apache.org/)) | `build.gradle`, `build.gradle.kts`, `pom.xml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | -| Go ([Golang](https://golang.org/)) | `go.sum` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| PHP ([Composer](https://getcomposer.org/)) | `composer.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Python ([setuptools](https://setuptools.readthedocs.io/en/latest/), [pip](https://pip.pypa.io/en/stable/), [Pipenv](https://pipenv.pypa.io/en/latest/)) | `setup.py`, `requirements.txt`, `requirements.pip`, `requires.txt`, `Pipfile` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Ruby ([Bundler](https://bundler.io/)) | `Gemfile.lock`, `gems.locked` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | -| Scala ([sbt](https://www.scala-sbt.org/)) | `build.sbt` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Package Managers | Languages | Supported files | Scan tools | +| ------------------- | --------- | --------------- | ------------ | +| [Bundler](https://bundler.io/) | Ruby | `Gemfile.lock`, `gems.locked` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | +| [Composer](https://getcomposer.org/) | PHP | `composer.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [Conan](https://conan.io/) | C, C++ | [`conan.lock`](https://docs.conan.io/en/latest/versioning/lockfiles.html) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [Golang](https://golang.org/) | Go | `go.sum` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) | Java | `build.gradle`, `build.gradle.kts`, `pom.xml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/) | JavaScript | `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | +| [NuGet](https://www.nuget.org/) 4.9+ | .NET, C# | [`packages.lock.json`](https://docs.microsoft.com/en-us/nuget/consume-packages/package-references-in-project-files#enabling-lock-file) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [setuptools](https://setuptools.readthedocs.io/en/latest/), [pip](https://pip.pypa.io/en/stable/), [Pipenv](https://pipenv.pypa.io/en/latest/) | Python | `setup.py`, `requirements.txt`, `requirements.pip`, `requires.txt`, `Pipfile` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| [sbt](https://www.scala-sbt.org/) 1.2 and below ([Ivy](http://ant.apache.org/ivy/)) | Scala | `build.sbt` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | Plans are underway for supporting the following languages, dependency managers, and dependency files. For details, see the issue link for each. -| Language (package managers) | Supported files | Scan tool(s) | Issue | -|----------------------------- | --------------- | ------------ | ----- | -| Python ([Poetry](https://python-poetry.org/)) | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | -| Python ([Pipenv](https://pipenv.pypa.io/en/latest/)) | `Pipfile.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#11756](https://gitlab.com/gitlab-org/gitlab/-/issues/11756) | +| Package Managers | Languages | Supported files | Scan tools | +| ------------------- | --------- | --------------- | ------------ | +| [Pipenv](https://pipenv.pypa.io/en/latest/) | Python | `Pipfile.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#11756](https://gitlab.com/gitlab-org/gitlab/-/issues/11756) | +| [Poetry](https://python-poetry.org/) | Python | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | +| [sbt](https://www.scala-sbt.org/) 1.3+ ([Coursier](https://get-coursier.io/))| Scala | `build.sbt` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | [GitLab#249526](https://gitlab.com/gitlab-org/gitlab/-/issues/249526) | ## Contribute your scanner @@ -81,7 +83,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Configuration -To enable Dependency Scanning for GitLab 11.9 and later, you must +To enable dependency scanning for GitLab 11.9 and later, you must [include](../../../ci/yaml/README.md#includetemplate) the [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) that is provided as a part of your GitLab installation. @@ -95,16 +97,16 @@ include: - template: Dependency-Scanning.gitlab-ci.yml ``` -The included template creates Dependency Scanning jobs in your CI/CD +The included template creates dependency scanning jobs in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[Dependency Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning) +[dependency scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning) that you can later download and analyze. Due to implementation limitations, we -always take the latest Dependency Scanning artifact available. +always take the latest dependency scanning artifact available. -### Customizing the Dependency Scanning settings +### Customizing the dependency scanning settings -The Dependency Scanning settings can be changed through [environment variables](#available-variables) by using the +The dependency scanning settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. For example: @@ -119,7 +121,7 @@ variables: Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable takes precedence. -### Overriding Dependency Scanning jobs +### Overriding dependency scanning jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) @@ -141,10 +143,10 @@ gemnasium-dependency_scanning: ### Available variables -Dependency Scanning can be [configured](#customizing-the-dependency-scanning-settings) +Dependency scanning can be [configured](#customizing-the-dependency-scanning-settings) using environment variables. -#### Configuring Dependency Scanning +#### Configuring dependency scanning The following variables allow configuration of global dependency scanning settings. @@ -156,7 +158,7 @@ The following variables allow configuration of global dependency scanning settin | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"` | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info` | -#### Configuring specific analyzers used by Dependency Scanning +#### Configuring specific analyzers used by dependency scanning The following variables are used for configuring specific analyzers (used for a specific language/framework). @@ -176,7 +178,7 @@ The following variables are used for configuring specific analyzers (used for a | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repos). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | -| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| +| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running dependency scanning in an offline, air-gapped environment.| | `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | | `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` environment variable. | @@ -214,16 +216,16 @@ For more information about the vulnerabilities database update, check the ## Dependency List -An additional benefit of Dependency Scanning is the ability to view your +An additional benefit of dependency scanning is the ability to view your project's dependencies and their known vulnerabilities. Read more about the [Dependency List](../dependency_list/index.md). ## Reports JSON format -The Dependency Scanning tool emits a JSON report file. For more information, see the +The dependency scanning tool emits a JSON report file. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/dependency-scanning-report-format.json). -Here's an example Dependency Scanning report: +Here's an example dependency scanning report: ```json-doc { @@ -342,36 +344,35 @@ You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-product to find a vulnerability in the Gemnasium database. You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md). -## Running Dependency Scanning in an offline environment +## Running dependency scanning in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for Dependency Scanning +to external resources through the internet, some adjustments are required for dependency scanning jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). -### Requirements for offline Dependency Scanning +### Requirements for offline dependency scanning -Here are the requirements for using Dependency Scanning in an offline environment: +Here are the requirements for using dependency scanning in an offline environment: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- Docker Container Registry with locally available copies of Dependency Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. +- Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/). This is required because, in an offline environment, the Gemnasium analyzer can't fetch the latest advisories from the online repository. - _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). - _Only if scanning npm/yarn projects_: Host an offline copy of the [retire.js](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [js](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. -NOTE: **Note:** -GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. -### Make GitLab Dependency Scanning analyzer images available inside your Docker registry +### Make GitLab dependency scanning analyzer images available inside your Docker registry -For Dependency Scanning with all [supported languages and frameworks](#supported-languages-and-package-managers), -import the following default Dependency Scanning analyzer images from `registry.gitlab.com` into +For dependency scanning with all [supported languages and frameworks](#supported-languages-and-package-managers), +import the following default dependency scanning analyzer images from `registry.gitlab.com` into your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext @@ -392,7 +393,19 @@ For details on saving and transporting Docker images as a file, see Docker's doc [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). -### Set Dependency Scanning CI job variables to use local Dependency Scanning analyzers +#### Support for Custom Certificate Authorities + +Support for custom certificate authorities was introduced in the following versions. + +| Analyzer | Version | +| -------- | ------- | +| `gemnasium` | [v2.8.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/releases/v2.8.0) | +| `gemnasium-maven` | [v2.9.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/releases/v2.9.0) | +| `gemnasium-python` | [v2.7.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/releases/v2.7.0) | +| `retire.js` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js/-/releases/v2.4.0) | +| `bundler-audit` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/releases/v2.4.0) | + +### Set dependency scanning CI job variables to use local dependency scanning analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must change the value of `SECURE_ANALYZERS_PREFIX` to refer to your local Docker container registry. You must also change the @@ -479,7 +492,19 @@ As a workaround, remove the [`retire.js`](analyzers.md#selecting-specific-analyz ### `Error response from daemon: error processing tar file: docker-tar: relocation error` -This error occurs when the Docker version that runs the Dependency Scanning job is `19.03.00`. +This error occurs when the Docker version that runs the dependency scanning job is `19.03.00`. Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/13830#note_211354992 "Current SAST container fails"). + +### Getting warning message `gl-dependency-scanning-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + +### Limitation when using rules:exists + +The [dependency scanning CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) +uses the [`rules:exists`](../../../ci/yaml/README.md#rulesexists) +syntax. This directive is limited to 10000 checks and always returns `true` after reaching this +number. Because of this, and depending on the number of files in your repository, a dependency +scanning job might be triggered even if the scanner doesn't support your project. diff --git a/doc/user/application_security/img/cve_request_communication.png b/doc/user/application_security/img/cve_request_communication.png Binary files differindex 0766b371c11..5c58df463ef 100644 --- a/doc/user/application_security/img/cve_request_communication.png +++ b/doc/user/application_security/img/cve_request_communication.png diff --git a/doc/user/application_security/img/cve_request_communication_publication.png b/doc/user/application_security/img/cve_request_communication_publication.png Binary files differindex 9e34c217e13..9eb6f2f8d9f 100644 --- a/doc/user/application_security/img/cve_request_communication_publication.png +++ b/doc/user/application_security/img/cve_request_communication_publication.png diff --git a/doc/user/application_security/img/new_cve_request_issue.png b/doc/user/application_security/img/new_cve_request_issue.png Binary files differindex a342c73992e..6ea7ca4a2ab 100644 --- a/doc/user/application_security/img/new_cve_request_issue.png +++ b/doc/user/application_security/img/new_cve_request_issue.png diff --git a/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png b/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png Binary files differindex f497b0fbc4e..7b04988afdb 100644 --- a/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png +++ b/doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png diff --git a/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png b/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png Binary files differindex fc847b578f5..b9b6dd13294 100644 --- a/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png +++ b/doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png diff --git a/doc/user/application_security/img/vulnerability-check_v13_4.png b/doc/user/application_security/img/vulnerability-check_v13_4.png Binary files differindex e0b53059b45..3e38f6eebe7 100644 --- a/doc/user/application_security/img/vulnerability-check_v13_4.png +++ b/doc/user/application_security/img/vulnerability-check_v13_4.png diff --git a/doc/user/application_security/img/vulnerability_solution.png b/doc/user/application_security/img/vulnerability_solution.png Binary files differindex d86b89a5f99..63e9c1473b6 100644 --- a/doc/user/application_security/img/vulnerability_solution.png +++ b/doc/user/application_security/img/vulnerability_solution.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index d509176f2b2..413a9f894e2 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -22,10 +22,10 @@ Testing (SAST), and Secret Detection by adding the following to your `.gitlab-ci ```yaml include: - - template: Dependency-Scanning.gitlab-ci.yml - - template: License-Scanning.gitlab-ci.yml - - template: SAST.gitlab-ci.yml - - template: Secret-Detection.gitlab-ci.yml + - template: Security/Dependency-Scanning.gitlab-ci.yml + - template: Security/License-Scanning.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml + - template: Security/Secret-Detection.gitlab-ci.yml ``` To add Dynamic Application Security Testing (DAST) scanning, add the following to your @@ -33,7 +33,7 @@ To add Dynamic Application Security Testing (DAST) scanning, add the following t ```yaml include: - - template: DAST.gitlab-ci.yml + - template: Security/DAST.gitlab-ci.yml variables: DAST_WEBSITE: https://staging.example.com @@ -449,7 +449,7 @@ To fix this issue, you can either: ```yaml include: - template: SAST.gitlab-ci.yml + template: Security/SAST.gitlab-ci.yml spotbugs-sast: stage: unit-tests @@ -458,6 +458,15 @@ To fix this issue, you can either: [Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). All the security scanning tools define their stage, so this error can occur with all of them. +### Getting warning messages `… report.json: no matching files` + +This is often followed by the [error `No files to upload`](../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload), +and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Please +check the entire job log for such messages. If you don't find these messages, retry the failed job +after setting `SECURE_LOG_LEVEL: "debug"` as a +[custom environment variable](../../ci/variables/README.md#custom-environment-variables). +This provides useful information to investigate further. + ### Getting error message `sast job: config key may not be used with 'rules': only/except` When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template @@ -490,7 +499,7 @@ would look similar to: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml # Ensure that the scanning is only executed on master or merge requests spotbugs-sast: @@ -505,7 +514,7 @@ would be written as follows: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml # Ensure that the scanning is only executed on master or merge requests spotbugs-sast: @@ -519,7 +528,7 @@ it would look similar to: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml # Ensure that the scanning is not executed on tags spotbugs-sast: @@ -531,7 +540,7 @@ To transition to the new `rules` syntax, the override would be rewritten as: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml # Ensure that the scanning is not executed on tags spotbugs-sast: diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 727f077aa09..6167c0445f9 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -25,6 +25,7 @@ SAST supports the following official analyzers: - [`flawfinder`](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) (Flawfinder) - [`gosec`](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Gosec) - [`kubesec`](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) (Kubesec) +- [`mobsf`](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf) (MobSF (beta)) - [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) (NodeJsScan) - [`phpcs-security-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit) (PHP CS security-audit) - [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) (PMD (Apex only)) @@ -53,7 +54,7 @@ In `.gitlab-ci.yml` define: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images @@ -70,7 +71,7 @@ In `.gitlab-ci.yml` define: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SAST_DEFAULT_ANALYZERS: "bandit,flawfinder" @@ -86,7 +87,7 @@ default analyzers. In `.gitlab-ci.yml` define: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SAST_DEFAULT_ANALYZERS: "" @@ -118,24 +119,24 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Analyzers Data -| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | -| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | -| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | -| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | -| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | x | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | -| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | +| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Sobelow | +| --------------------------------------- | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :------------------: | :---------------------: | :-------------------------: | :----------------: | +| Severity | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Description | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | ✓ | ✓ | ✓ | +| End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | +| End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Affected item (e.g. class or package) | ✓ | 𐄂 | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Confidence | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | x | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | +| Source code extract | 𐄂 | ✓ | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | - ✓ => we have that data - ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a4fc3c9e638..9e4d4112ae8 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -11,8 +11,8 @@ type: reference, howto NOTE: **Note:** The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab.com/resources/whitepaper-seismic-shift-application-security/) -explains how **4 of the top 6 attacks were application based**. Download it -to learn how to protect your organization. +explains how 4 of the top 6 attacks were application based. Download it to learn how to protect your +organization. If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known vulnerabilities using Static Application Security Testing (SAST). GitLab checks the SAST report and @@ -31,8 +31,10 @@ The results are sorted by the priority of the vulnerability: 1. Unknown 1. Everything else -NOTE: **Note:** -A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard doesn't show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard doesn't show SAST results. The analyzer outputs an [exit code](../../../development/integrations/secure.md#exit-code) on failure. +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish +for any reason, the security dashboard doesn't show SAST scanner output. For example, if the SAST +job finishes but the DAST job fails, the security dashboard doesn't show SAST results. On failure, +the analyzer outputs an [exit code](../../../development/integrations/secure.md#exit-code). ## Use cases @@ -59,39 +61,41 @@ is **not** `19.03.0`. See [troubleshooting information](#error-response-from-dae GitLab SAST supports a variety of languages, package managers, and frameworks. Our SAST security scanners also feature automatic language detection which works even for mixed-language projects. If any supported language is detected in project source code we will automatically run the appropriate SAST analyzers. -You can also [view our language roadmap](https://about.gitlab.com/direction/secure/static-analysis/sast/#language-support) and [request other language support by opening an issue](https://gitlab.com/groups/gitlab-org/-/epics/297). +You can also [view our language roadmap](https://about.gitlab.com/direction/secure/static-analysis/sast/#language-support) and [request other language support by opening an issue](https://gitlab.com/groups/gitlab-org/-/epics/297). | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |--------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Go | [Gosec](https://github.com/securego/gosec) | 10.7, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT), [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Java ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT), [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2 | -| Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | -| React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2 | -| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3, [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.1 | -| Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven), [moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.3 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | +| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | +| C/C++ | [Flawfinder](https://github.com/david-a-wheeler/flawfinder) | 10.7 | +| Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.1 | +| Go | [Gosec](https://github.com/securego/gosec) | 10.7 | +| Groovy ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.3 (Gradle) & 11.9 (Ant, Maven, SBT) | +| Helm Charts | [Kubesec](https://github.com/controlplaneio/kubesec) | 13.1 | +| Java ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 10.6 (Maven), 10.8 (Gradle) & 11.9 (Ant, SBT) | +| Java (Android) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | +| JavaScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | +| Kotlin (Android) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | +| Kubernetes manifests | [Kubesec](https://github.com/controlplaneio/kubesec) | 12.6 | +| Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | +| Objective-C (iOS) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | +| PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | +| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | +| React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 | +| Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | +| Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | +| Swift (iOS) | [MobSF (beta)](https://github.com/MobSF/Mobile-Security-Framework-MobSF) | 13.5 | | TypeScript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.9, [merged](https://gitlab.com/gitlab-org/gitlab/-/issues/36059) with ESLint in 13.2 | -NOTE: **Note:** -The Java analyzers can also be used for variants like the +Note that the Java analyzers can also be used for variants like the [Gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html), -[Grails](https://grails.org/) and the [Maven wrapper](https://github.com/takari/maven-wrapper). +[Grails](https://grails.org/), +and the [Maven wrapper](https://github.com/takari/maven-wrapper). ### Making SAST analyzers available to all GitLab tiers -All open source (OSS) analyzers have been moved to the GitLab Core tier. Progress can be -tracked in the corresponding -[epic](https://gitlab.com/groups/gitlab-org/-/epics/2098). +All open source (OSS) analyzers have been moved to the GitLab Core tier as of GitLab 13.3. #### Summary of features per tier @@ -147,16 +151,19 @@ always take the latest SAST artifact available. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab Ultimate 13.3. > - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in GitLab Ultimate 13.4. +> - [Improved](https://gitlab.com/groups/gitlab-org/-/epics/3635) in GitLab Ultimate 13.5. You can enable and configure SAST with a basic configuration using the **SAST Configuration** page: 1. From the project's home page, go to **Security & Compliance** > **Configuration** in the left sidebar. -1. If the project does not have a `gitlab-ci.yml` file, click **Enable** in the Static Application Security Testing (SAST) row, otherwise click **Configure**. -1. Enter the custom SAST values, then click **Create Merge Request**. +1. If the project does not have a `.gitlab-ci.yml` file, click **Enable** in the Static Application Security Testing (SAST) row, otherwise click **Configure**. +1. Enter the custom SAST values. Custom values are stored in the `.gitlab-ci.yml` file. For variables not in the SAST Configuration page, their values are left unchanged. Default values are inherited from the GitLab SAST template. +1. Optionally, expand the **SAST analyzers** section, select individual [SAST analyzers](./analyzers.md) and enter custom analyzer values. +1. Click **Create Merge Request**. 1. Review and merge the merge request. ### Customizing the SAST settings @@ -169,7 +176,7 @@ set the `SAST_GOSEC_LEVEL` variable to `2`: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SAST_GOSEC_LEVEL: 2 @@ -191,13 +198,78 @@ inclusion and specify any additional keys under it. For example, this enables `F ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml spotbugs-sast: variables: FAIL_NEVER: 1 ``` +### Custom rulesets + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235382) in GitLab 13.5. + +You can customize the default scanning rules provided with SAST's NodeJS-Scan and Gosec analyzers. +Customization allows you to exclude rules and modify the behavior of existing rules. + +To customize the default scanning rules, create a file containing custom rules. These rules +are passed through to the analyzer's underlying scanner tool. + +To create a custom ruleset: + +1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. +1. Create a custom ruleset file named `sast-ruleset.toml` in the `.gitlab` directory. +1. In the `sast-ruleset.toml` file, do one of the following: + + - Define a custom analyzer configuration. In this example, customized rules are defined for the + `nodejs-scan` scanner: + + ```toml + [nodejs-scan] + description = 'custom ruleset for nodejs-scan' + + [[nodejs-scan.passthrough]] + type = "raw" + value = ''' + - nodejs-extensions: + - .js + + template-extensions: + - .new + - .hbs + - '' + + ignore-filenames: + - skip.js + + ignore-paths: + - __MACOSX + - skip_dir + - node_modules + + ignore-extensions: + - .hbs + + ignore-rules: + - regex_injection_dos + - pug_jade_template + - express_xss + + ''' + ``` + + - Provide the name of the file containing a custom analyzer configuration. In this example, + customized rules for the `gosec` scanner are contained in the file `gosec-config.json`: + + ```toml + [gosec] + description = 'custom ruleset for gosec' + + [[gosec.passthrough]] + type = "file" + value = "gosec-config.json" + ``` + ### Using environment variables to pass credentials for private repositories Some analyzers require downloading the project's dependencies in order to @@ -222,7 +294,7 @@ Kubesec analyzer. In `.gitlab-ci.yml`, define: ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SCAN_KUBERNETES_MANIFESTS: "true" @@ -248,7 +320,7 @@ stages: - test include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml build: stage: build @@ -264,17 +336,16 @@ spotbugs-sast: - build variables: MAVEN_REPO_PATH: ./.m2/repository - COMPILE: false + COMPILE: "false" artifacts: reports: sast: gl-sast-report.json ``` -NOTE: **Note:** -The path to the vendored directory must be specified explicitly to allow -the analyzer to recognize the compiled artifacts. This configuration can vary per -analyzer but in the case of Java above, `MAVEN_REPO_PATH` can be used. -See [Analyzer settings](#analyzer-settings) for the complete list of available options. +To allow the analyzer to recognize the compiled artifacts, you must explicitly specify the path to +the vendored directory. This configuration can vary per analyzer but in the case of Java above, you +can use `MAVEN_REPO_PATH`. See +[Analyzer settings](#analyzer-settings) for the complete list of available options. ### Available variables @@ -358,13 +429,29 @@ CAUTION: **Caution:** Variables having names starting with these prefixes will **not** be propagated to the SAST Docker container and/or analyzer containers: `DOCKER_`, `CI`, `GITLAB_`, `FF_`, `HOME`, `PWD`, `OLDPWD`, `PATH`, `SHLVL`, `HOSTNAME`. +### Experimental features + +Receive early access to experimental features. + +Currently, this will enable scanning of iOS and Android apps via the [MobSF analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf/). + +To enable experimental features, add the following to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: Security/SAST.gitlab-ci.yml + +variables: + SAST_EXPERIMENTAL_FEATURES: "true" +``` + ## Reports JSON format The SAST tool emits a JSON report file. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). -The JSON report file can be downloaded from the CI pipelines page, for more -information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). +The JSON report file can be downloaded from the CI pipelines page, or the +pipelines tab on merge requests. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). Here's an example SAST report: @@ -480,7 +567,6 @@ To use SAST in an offline environment, you need: - A Docker Container Registry with locally available copies of SAST [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Configure certificate checking of packages (optional). -NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) @@ -518,6 +604,25 @@ For details on saving and transporting Docker images as a file, see Docker's doc [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). +#### If support for Custom Certificate Authorities are needed + +Support for custom certificate authorities was introduced in the following versions. + +| Analyzer | Version | +| -------- | ------- | +| `bandit` | [v2.3.0](https://gitlab.com/gitlab-org/security-products/analyzers/bandit/-/releases/v2.3.0) | +| `brakeman` | [v2.1.0](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman/-/releases/v2.1.0) | +| `eslint` | [v2.9.2](https://gitlab.com/gitlab-org/security-products/analyzers/eslint/-/releases/v2.9.2) | +| `flawfinder` | [v2.3.0](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder/-/releases/v2.3.0) | +| `gosec` | [v2.5.0](https://gitlab.com/gitlab-org/security-products/analyzers/gosec/-/releases/v2.5.0) | +| `kubesec` | [v2.1.0](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec/-/releases/v2.1.0) | +| `nodejs-scan` | [v2.9.5](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan/-/releases/v2.9.5) | +| `phpcs-security-audit` | [v2.8.2](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit/-/releases/v2.8.2) | +| `pmd-apex` | [v2.1.0](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex/-/releases/v2.1.0) | +| `security-code-scan` | [v2.7.3](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/releases/v2.7.3) | +| `sobelow` | [v2.2.0](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow/-/releases/v2.2.0) | +| `spotbugs` | [v2.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs/-/releases/v2.7.1) | + ### Set SAST CI job variables to use local SAST analyzers Add the following configuration to your `.gitlab-ci.yml` file. You must replace @@ -525,7 +630,7 @@ Add the following configuration to your `.gitlab-ci.yml` file. You must replace ```yaml include: - - template: SAST.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml variables: SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" @@ -549,3 +654,16 @@ This error occurs when the Docker version that runs the SAST job is `19.03.0`. Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/13830#note_211354992 "Current SAST container fails"). + +### Getting warning message `gl-sast-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + +### Limitation when using rules:exists + +The [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) +uses the `rules:exists` parameter. For performance reasons, a maximum number of matches are made +against the given glob pattern. If the number of matches exceeds the maximum, the `rules:exists` +parameter returns `true`. Depending on the number of files in your repository, a SAST job might be +triggered even if the scanner doesn't support your project. For more details about this issue, see +the [`rules:exists` documentation](../../../ci/yaml/README.md#rulesexists). diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index f3e411cdc16..bb10e9d7315 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -9,8 +9,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://about.gitlab.com/releases/2019/03/22/gitlab-11-9-released/#detect-secrets-and-credentials-in-the-repository) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9. -## Overview - A recurring problem when developing applications is that developers may unintentionally commit secrets and credentials to their remote repositories. If other people have access to the source, or if the project is public, the sensitive information is then exposed and can be leveraged by @@ -40,7 +38,7 @@ To run Secret Detection jobs, by default, you need GitLab Runner with the If you're using the shared runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** -Our Secret Detection jobs currently expect a Linux container type. Windows containers are not yet supported. +Our Secret Detection jobs expect a Linux container type. Windows containers are not supported. CAUTION: **Caution:** If you use your own runners, make sure the Docker version installed @@ -67,26 +65,27 @@ as shown in the following table: ## Configuration -NOTE: **Note:** -With GitLab 13.1 Secret Detection was split into its own CI/CD template. +> GitLab 13.1 splits Secret Detection from the [SAST configuration](../sast#configuration) into its own CI/CD template. If you're using GitLab 13.0 or earlier and SAST is enabled, then Secret Detection is already enabled. Secret Detection is performed by a [specific analyzer](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) -during the `secret-detection` job. It runs regardless of the programming -language of your app. +during the `secret-detection` job. It runs regardless of your app's programming language. -The Secret Detection analyzer includes [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) checks. +The Secret Detection analyzer includes [Gitleaks](https://github.com/zricethezav/gitleaks) and +[TruffleHog](https://github.com/dxa4481/truffleHog) checks. -NOTE: **Note:** -The Secret Detection analyzer will ignore "Password in URL" vulnerabilities if the password begins -with a dollar sign (`$`) as this likely indicates the password being used is an environment -variable. For example, `https://username:$password@example.com/path/to/repo` won't be -detected, whereas `https://username:password@example.com/path/to/repo` would be detected. +Note that the Secret Detection analyzer ignores Password-in-URL vulnerabilities if the password +begins with a dollar sign (`$`), as this likely indicates the password is an environment variable. +For example, `https://username:$password@example.com/path/to/repo` isn't detected, while +`https://username:password@example.com/path/to/repo` is. NOTE: **Note:** -You don't have to configure Secret Detection manually as shown in this section if you're using [Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection) +You don't have to configure Secret Detection manually as shown in this section if you're using +[Auto Secret Detection](../../../topics/autodevops/stages.md#auto-secret-detection) provided by [Auto DevOps](../../../topics/autodevops/index.md). -To enable Secret Detection for GitLab 13.1 and later, you must include the `Secret-Detection.gitlab-ci.yml` template that’s provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined in that template. +To enable Secret Detection for GitLab 13.1 and later, you must include the +`Secret-Detection.gitlab-ci.yml` template that's provided as a part of your GitLab installation. For +GitLab versions earlier than 11.9, you can copy and use the job as defined in that template. Add the following to your `.gitlab-ci.yml` file: @@ -103,30 +102,6 @@ The results are saved as a that you can later download and analyze. Due to implementation limitations, we always take the latest Secret Detection artifact available. -### Using the SAST Template - -Prior to GitLab 13.1, Secret Detection was part of [SAST configuration](../sast#configuration). -If you already have SAST enabled for your app configured before GitLab 13.1, -you don't need to manually configure it. - -CAUTION: **Planned Deprecation:** -In a future GitLab release, configuring Secret Detection with the SAST template will be deprecated. Please begin using `Secret-Detection.gitlab-ci.yml` -to prevent future issues. We have made a -[video to guide you through the process of transitioning](https://www.youtube.com/watch?v=W2tjcQreDwQ) -to this new template. - -<div class="video-fallback"> - See the video: <a href="https://www.youtube.com/watch?v=W2tjcQreDwQ">Walkthrough of historical secret scan</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube.com/embed/W2tjcQreDwQ" frameborder="0" allowfullscreen="true"> </iframe> -</figure> - -When using the SAST template, Secret Detection is performed by a [specific analyzer](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L180) -during the `sast` job. It runs regardless of the programming -language of your app, and you don't need to change your -CI/CD configuration file to enable it. Results are available in the SAST report. - ### Customizing settings The Secret Detection scan settings can be changed through [environment variables](#available-variables) @@ -164,9 +139,52 @@ Secret Detection can be customized by defining available variables: |-------------------------|---------------|-------------| | `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | | `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | -| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories will also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | +| `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | +### Custom rulesets + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. + +You can customize the default secret detection rules provided with GitLab. +Customization allows you to exclude rules and add new rules. + +To create a custom ruleset: + +1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. +1. Create a custom ruleset file named `secret-detection-ruleset.toml` in the `.gitlab` directory. +1. In the `secret-detection-ruleset.toml` file, do one of the following: + + - Define a custom ruleset: + + ```toml + [secrets] + description = 'secrets custom rules configuration' + + [[secrets.passthrough]] + type = "raw" + target = "gitleaks.toml" + value = """\ + title = "gitleaks config" + # add regexes to the regex table + [[rules]] + description = "Test for Raw Custom Rulesets" + regex = '''Custom Raw Ruleset T[est]{3}''' + """ + ``` + + - Provide the name of the file containing a custom ruleset: + + ```toml + [secrets] + description = 'secrets custom rules configuration' + + [[secrets.passthrough]] + type = "file" + target = "gitleaks.toml" + value = "config/gitleaks.toml" + ``` + ### Logging level To control the verbosity of logs set the `SECURE_LOG_LEVEL` environment variable. Messages of this logging level or higher are output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. @@ -197,3 +215,35 @@ We have created a [short video walkthrough](https://youtu.be/wDtc_K00Y0A) showca <figure class="video-container"> <iframe src="https://www.youtube.com/embed/wDtc_K00Y0A" frameborder="0" allowfullscreen="true"> </iframe> </figure> + +### Make GitLab Secret Detection analyzer image available inside your Docker registry + +Import the following default Secret Detection analyzer images from `registry.gitlab.com` into your +[local Docker container registry](../../packages/container_registry/index.md): + +```plaintext +registry.gitlab.com/gitlab-org/security-products/analyzers/secrets:3 +``` + +The process for importing Docker images into a local offline Docker registry depends on +**your network security policy**. Please consult your IT staff to find an accepted and approved +process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you're able to make periodic updates yourself. + +For details on saving and transporting Docker images as a file, see Docker's documentation on +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). + +#### If support for Custom Certificate Authorities are needed + +Support for custom certificate authorities was introduced in the following versions. + +| Analyzer | Version | +| -------- | ------- | +| secrets | [v3.0.0](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/releases/v3.0.0) | + +## Troubleshooting + +### Getting warning message `gl-secret-detection-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png b/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png Binary files differindex 67a7bb5f368..0310ef3ea0a 100644 --- a/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png +++ b/doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_center_settings_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_center_settings_v13_4.png Binary files differnew file mode 100644 index 00000000000..4223494c294 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_center_settings_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png Binary files differindex 3c618090be8..5edceb32e5c 100644 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png Binary files differindex d010adcc90c..5379b5c6e5d 100644 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_dismissal_v13_4.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_dismissal_v13_4.png Binary files differnew file mode 100644 index 00000000000..eb1dfe6c6f4 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_dismissal_v13_4.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png Binary files differdeleted file mode 100644 index 878bb83c2a2..00000000000 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png Binary files differdeleted file mode 100644 index 7cab7b0a61f..00000000000 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_2.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png Binary files differdeleted file mode 100644 index 34c64f830ba..00000000000 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_5.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_5.png Binary files differnew file mode 100644 index 00000000000..c46a8295a53 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_5.png diff --git a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png Binary files differindex eb91cfc47ad..760942c3239 100644 --- a/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png +++ b/doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 51d9b4f45cd..5fa8ebb80e0 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -5,21 +5,26 @@ group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab Security Dashboard **(ULTIMATE)** +# GitLab Security Dashboard, Security Center, and Vulnerability Reports **(ULTIMATE)** -The Security Dashboard is a good place to get an overview of all the security -vulnerabilities in your groups, projects, and pipelines. +GitLab provides a comprehensive set of features for viewing and managing vulnerabilities: + +- Security dashboards: An overview of the security status in your instance, groups, and projects. +- Vulnerability reports: Detailed lists of all vulnerabilities for the instance, group, project, or + pipeline. This is where you triage and manage vulnerabilities. +- Security Center: A dedicated area for vulnerability management at the instance level. This + includes a security dashboard, vulnerability report, and settings. You can also drill down into a vulnerability and get extra information. This includes the project it comes from, any related file(s), and metadata that helps you analyze the risk it poses. You can also dismiss a vulnerability or create an issue for it. -To benefit from the Security Dashboard you must first configure one of the +To benefit from these features, you must first configure one of the [security scanners](../index.md). ## Supported reports -The Security Dashboard displays vulnerabilities detected by scanners such as: +The vulnerability report displays vulnerabilities detected by scanners such as: - [Container Scanning](../container_scanning/index.md) - [Dynamic Application Security Testing](../dast/index.md) @@ -29,7 +34,7 @@ The Security Dashboard displays vulnerabilities detected by scanners such as: ## Requirements -To use the instance, group, project, or pipeline security dashboard: +To use the security dashboards and vulnerability reports: 1. At least one project inside a group must be configured with at least one of the [supported reports](#supported-reports). @@ -41,15 +46,19 @@ To use the instance, group, project, or pipeline security dashboard: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13496) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -At the pipeline level, the Security section displays the vulnerabilities present in the branch of the project the pipeline was run against. +At the pipeline level, the Security section displays the vulnerabilities present in the branch of +the project the pipeline ran against.  Visit the page for any pipeline that ran any of the [supported reports](#supported-reports). To view the pipeline's security findings, select the **Security** tab when viewing the pipeline. -NOTE: **Note:** -A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish +for any reason, the security dashboard doesn't show SAST scanner output. For example, if the SAST +job finishes but the DAST job fails, the security dashboard doesn't show SAST results. On failure, +the analyzer outputs an +[exit code](../../../development/integrations/secure.md#exit-code). ## Project Security Dashboard @@ -60,12 +69,15 @@ At the project level, the Security Dashboard displays the vulnerabilities merged to **Security & Compliance > Security Dashboard**. By default, the Security Dashboard displays all detected and confirmed vulnerabilities. -The Security Dashboard first displays the total number of vulnerabilities by severity (for example, +The Security Dashboard first displays the time at which the last pipeline completed on the project's +default branch. There's also a link to view this in more detail. + +The Security Dashboard next displays the total number of vulnerabilities by severity (for example, Critical, High, Medium, Low, Info, Unknown). Below this, a table shows each vulnerability's status, severity, and description. Clicking a vulnerability takes you to its [Vulnerability Details](../vulnerabilities) page to view more information about that vulnerability. - + You can filter the vulnerabilities by one or more of the following: @@ -78,7 +90,7 @@ You can also dismiss vulnerabilities in the table: 1. Select the checkbox for each vulnerability you want to dismiss. 1. In the menu that appears, select the reason for dismissal and click **Dismiss Selected**. - + ## Group Security Dashboard @@ -86,79 +98,99 @@ You can also dismiss vulnerabilities in the table: The group Security Dashboard gives an overview of the vulnerabilities in the default branches of the projects in a group and its subgroups. Access it by navigating to **Security > Security Dashboard** -for your group. By default, the Security Dashboard displays all detected and confirmed -vulnerabilities. +after selecting your group. By default, the Security Dashboard displays all detected and confirmed +vulnerabilities. If you don't see the vulnerabilities over time graph, the likely cause is that you +have not selected a group. -NOTE: **Note:** -The Security Dashboard only shows projects with [security reports](#supported-reports) enabled in a -group. +Note that the Security Dashboard only shows projects with +[security reports](#supported-reports) +enabled in a group.  There is a timeline chart that shows how many open -vulnerabilities your projects had at various points in time. You can filter among 30, 60, and -90 days, with the default being 90. Hover over the chart to get more details about -the open vulnerabilities at a specific time. +vulnerabilities your projects had at various points in time. You can display the vulnerability +trends over a 30, 60, or 90-day time frame (the default is 90 days). Hover over the chart to get +more details about the open vulnerabilities at a specific time. Next to the timeline chart is a list of projects, grouped and sorted by the severity of the vulnerability found: -- F: 1 or more "critical" -- D: 1 or more "high" or "unknown" -- C: 1 or more "medium" -- B: 1 or more "low" -- A: 0 vulnerabilities +- F: One or more "critical" +- D: One or more "high" or "unknown" +- C: One or more "medium" +- B: One or more "low" +- A: Zero vulnerabilities Projects with no vulnerability tests configured will not appear in the list. Additionally, dismissed -vulnerabilities are not included either. +vulnerabilities are excluded. -Navigate to the group's [Vulnerability Report](#vulnerability-list) to view the vulnerabilities found. +Navigate to the group's [vulnerability report](#vulnerability-report) to view the vulnerabilities found. -## Instance Security Dashboard +## Instance Security Center -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6953) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.8. +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3426) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.4. -At the instance level, the Security Dashboard displays the vulnerabilities present in the default -branches of all the projects you configure to display on the dashboard. It includes all the -[group Security Dashboard's](#group-security-dashboard) -features. +The Security Center is where you manage vulnerabilities for your instance. It displays the +vulnerabilities present in the default branches of all the projects you configure. It includes the +following: + +- The [group security dashboard's](#group-security-dashboard) features. +- A [vulnerability report](#vulnerability-report). +- A dedicated settings area to configure which projects to display.  -You can access the Instance Security Dashboard from the menu +You can access the Instance Security Center from the menu bar at the top of the page. Under **More**, select **Security**. - + -The dashboard is empty before you add projects to it. +The dashboard and vulnerability report are empty before you add projects. - + -### Adding projects to the dashboard +### Adding projects to the Security Center -To add projects to the dashboard: +To add projects to the Security Center: 1. Click **Settings** in the left navigation bar or click the **Add projects** button. 1. Search for and add one or more projects using the **Search your projects** field. 1. Click the **Add projects** button. -After you add projects, the Security Dashboard displays the vulnerabilities found in those projects' -default branches. + + +After you add projects, the security dashboard and vulnerability report display the vulnerabilities +found in those projects' default branches. ## Export vulnerabilities > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. -You can export all your vulnerabilities in CSV format by clicking the **{upload}** **Export** -button located at top right of the **Security Dashboard**. After the report -is built, the CSV report downloads to your local machine. The report contains all -vulnerabilities for the projects defined in the **Security Dashboard**, -as filters don't apply to the export function. - - +You can export all your vulnerabilities in CSV (comma separated values) format by clicking the +**{upload}** **Export** button located at top right of the Security Dashboard. When the report is +ready, the CSV report downloads to your local machine. The report contains all vulnerabilities for +the projects defined in the Security Dashboard, as filters don't apply to the export function. NOTE: **Note:** It may take several minutes for the download to start if your project contains -thousands of vulnerabilities. Do not close the page until the download finishes. +thousands of vulnerabilities. Don't close the page until the download finishes. + +The fields in the export include: + +- Group Name +- Project Name +- Scanner Type +- Scanner Name +- Status +- Vulnerability +- Details +- Additional Info +- Severity +- [CVE](https://cve.mitre.org/) (Common Vulnerabilities and Exposures) +- [CWE](https://cwe.mitre.org/) (Common Weakness Enumeration) +- Other Identifiers + + ## Keeping the dashboards up to date @@ -191,14 +223,14 @@ When using [Auto DevOps](../../../topics/autodevops/index.md), use [special environment variables](../../../topics/autodevops/customize.md#environment-variables) to configure daily security scans. -## Vulnerability list +## Vulnerability report -Each dashboard's vulnerability list contains vulnerabilities from the latest scans that were merged +Each vulnerability report contains vulnerabilities from the latest scans that were merged into the default branch.  -You can filter which vulnerabilities the Security Dashboard displays by: +You can filter which vulnerabilities the vulnerability report displays by: - Status - Severity @@ -211,8 +243,10 @@ To create an issue associated with the vulnerability, click the **Create Issue**  -Once you create the issue, the vulnerability list contains a link to the issue and an icon whose -color indicates the issue's status (green for open issues, blue for closed issues). +Once you create the issue, the linked issue icon in the vulnerability list: + +- Indicates that an issue has been created for that vulnerability. +- Shows a tooltip that contains a link to the issue.  diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index 8006a49ba35..f975de213ef 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -13,7 +13,6 @@ This terminology list for GitLab Secure and Defend aims to: - Improve the effectiveness of communication regarding GitLab's application security features. - Get new contributors up to speed faster. -NOTE: **Note:** This document defines application security terms in the specific context of GitLab's Secure and Defend products. Terms may therefore have different meanings outside of GitLab Secure and Defend. diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index 5414800b290..391666a077e 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -105,11 +105,10 @@ disabled state. Once enabled,a predefined policy deploys to the selected environment's deployment platform and you can manage it like the regular policies. -NOTE: **Note:** -If you're using [Auto DevOps](../../../topics/autodevops/index.md) and -change a policy in this section, your `auto-deploy-values.yaml` file -doesn't update. Auto DevOps users must make changes by following -the [Container Network Policy documentation](../../../topics/autodevops/stages.md#network-policy). +Note that if you're using [Auto DevOps](../../../topics/autodevops/index.md) +and change a policy in this section, your `auto-deploy-values.yaml` file doesn't update. Auto DevOps +users must make changes by following the +[Container Network Policy documentation](../../../topics/autodevops/stages.md#network-policy). ### Changing enforcement status @@ -119,12 +118,9 @@ To change a network policy's enforcement status: - Click the **Enforcement status** toggle to update the selected policy. - Click the **Apply changes** button to deploy network policy changes. -NOTE: **Note:** -Disabled network policies have the -`network-policy.gitlab.com/disabled_by: gitlab` selector inside the -`podSelector` block. This narrows the scope of such a policy and as a -result it doesn't affect any pods. The policy itself is still deployed -to the corresponding deployment namespace. +Disabled network policies have the `network-policy.gitlab.com/disabled_by: gitlab` selector inside +the `podSelector` block. This narrows the scope of such a policy and as a result it doesn't affect +any pods. The policy itself is still deployed to the corresponding deployment namespace. ### Container Network Policy editor @@ -135,10 +131,8 @@ create a new policy click the **New policy** button located in the **Policy** tab's header. To edit an existing policy, click**Edit policy** in the selected policy drawer. -NOTE: **Note:** -The policy editor only supports the -[CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/policy/)specification. Regular -Kubernetes +Note that the policy editor only supports the +[CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/policy/)specification. Regular Kubernetes [NetworkPolicy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#networkpolicy-v1-networking-k8s-io) resources aren't supported. diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index ff383fdf553..ee3fd6c4dd4 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -14,6 +14,7 @@ Each security vulnerability in a project's [Security Dashboard](../security_dash - Details of the vulnerability. - The status of the vulnerability within the project. - Available actions for the vulnerability. +- Issues related to the vulnerability. On the vulnerability page, you can interact with the vulnerability in several different ways: @@ -23,6 +24,7 @@ several different ways: - [Create issue](#creating-an-issue-for-a-vulnerability) - Create a new issue with the title and description pre-populated with information from the vulnerability report. By default, such issues are [confidential](../../project/issues/confidential_issues.md). +- [Link issues](#link-issues-to-the-vulnerability) - Link existing issues to vulnerability. - [Solution](#automatic-remediation-for-vulnerabilities) - For some vulnerabilities, a solution is provided for how to fix the vulnerability. @@ -38,6 +40,9 @@ the following values: | Dismissed | A user has seen this vulnerability and dismissed it | | Resolved | The vulnerability has been fixed and is no longer in the codebase | +A timeline shows you when the vulnerability status has changed, +and allows you to comment on a change. + ## Creating an issue for a vulnerability You can create an issue for a vulnerability by selecting the **Create issue** button. @@ -47,6 +52,12 @@ project the vulnerability came from, and pre-populates it with useful informatio the vulnerability report. After the issue is created, GitLab redirects you to the issue page so you can edit, assign, or comment on the issue. +## Link issues to the vulnerability + +You can link one or more existing issues to the vulnerability. This allows you to +indicate that this vulnerability affects multiple issues. It also allows you to indicate +that the resolution of one issue would resolve multiple vulnerabilities. + ## Automatic remediation for vulnerabilities You can fix some vulnerabilities by applying the solution that GitLab automatically diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 6521b71e9e6..196c3e9fb43 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -6,18 +6,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Kubernetes Agent **(PREMIUM ONLY)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. +> - It's disabled on GitLab.com. Rolling this feature out to GitLab.com is [planned](https://gitlab.com/groups/gitlab-org/-/epics/3834). -## Goals +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. -The [GitLab Kubernetes Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) is an active in-cluster component for solving GitLab and Kubernetes integration tasks in a secure and cloud native way. +The [GitLab Kubernetes Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) +is an active in-cluster component for solving GitLab and Kubernetes integration +tasks in a secure and cloud-native way. It enables: -Features: +- Integrating GitLab with a Kubernetes cluster behind a firewall or NAT + (network address translation). +- Pull-based GitOps deployments by leveraging the + [GitOps Engine](https://github.com/argoproj/gitops-engine). +- Real-time access to API endpoints within a cluster. -1. Makes it possible to integrate GitLab with a Kubernetes cluster behind a firewall or NAT -1. Enables pull-based GitOps deployments by leveraging the [GitOps Engine](https://github.com/argoproj/gitops-engine) -1. Allows for real-time access to API endpoints within a cluster. -1. Many more features are planned. Please [review our roadmap](https://gitlab.com/groups/gitlab-org/-/epics/3329). +Many more features are planned. Please [review our roadmap](https://gitlab.com/groups/gitlab-org/-/epics/3329). ## Architecture @@ -39,34 +44,52 @@ sequenceDiagram end ``` -Please refer to our [full architecture documentation in the Agent project](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). +Please refer to our [full architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture) +in the Agent project. -## Getting started with GitOps using the GitLab Agent and the GitLab Cloud Native Helm chart +## Get started with GitOps and the GitLab Agent There are several components that work in concert for the Agent to accomplish GitOps deployments: -1. A Kubernetes cluster that is properly configured -1. A configuration repository that contains a `config.yaml` file. This `config.yaml` tells the Agent which repositories to synchronize with. -1. A manifest repository that contains a `manifest.yaml`. This `manifest.yaml` (which can be autogenerated) is tracked by the Agent and any changes to the file are automatically applied to the cluster. +- A properly-configured Kubernetes cluster. +- A configuration repository that contains a `config.yaml` file, which tells the + Agent which repositories to synchronize with. +- A manifest repository that contains a `manifest.yaml`, which is tracked by the + Agent and can be auto-generated. Any changes to `manifest.yaml` are applied to the cluster. -The setup process involves a few steps that, once completed, will enable GitOps deployments to work +The setup process involves a few steps to enable GitOps deployments: -1. Installing the Agent server via GitLab Helm chart -1. Defining a configuration directory -1. Creating an Agent record in GitLab -1. Generating and copying a Secret token used to connect to the Agent -1. Installing the Agent into the cluster -1. Creating a `manifest.yaml` +1. Installing the Agent server. +1. Defining a configuration directory. +1. Creating an Agent record in GitLab. +1. Generating and copying a Secret token used to connect to the Agent. +1. Installing the Agent into the cluster. +1. Creating a `manifest.yaml`. -### Installing the Agent server via Helm +### Install the Agent server -Currently the GitLab Kubernetes Agent can only be deployed via our [Helm chart](https://gitlab.com/gitlab-org/charts/gitlab). +The GitLab Kubernetes Agent can be deployed using [Omnibus +GitLab](https://docs.gitlab.com/omnibus/) or the [GitLab +chart](https://gitlab.com/gitlab-org/charts/gitlab). If you don't already have +GitLab installed, please refer to our [installation +documentation](https://docs.gitlab.com/ee/install/README.html). -NOTE: We are working quickly to [include the Agent in Official Linux Package](https://gitlab.com/gitlab-org/gitlab/-/issues/223060). +NOTE: **Note:** +GitLab plans to include the Agent on [GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/3834). -If you don't already have GitLab installed via Helm please refer to our [installation documentation](https://docs.gitlab.com/charts/installation/) +When using the [Omnibus GitLab](https://docs.gitlab.com/omnibus/) package: -When installing/upgrading the GitLab Helm chart please consider the following Helm 2 example (if using Helm 3 please modify): +1. Edit `/etc/gitlab/gitlab.rb`: + +```plaintext +gitlab_kas['enable'] = true +``` + +1. [Reconfigure GitLab](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). + +When installing or upgrading the GitLab Helm chart, consider the following Helm 2 example. +(If you're using Helm 3, you must modify this example.) You must set `global.kas.enabled=true` +for the Agent to be properly installed and configured: ```shell helm repo update @@ -79,15 +102,14 @@ helm upgrade --force --install gitlab gitlab/gitlab \ --set global.kas.enabled=true ``` -`global.kas.enabled=true` must be set in order for the Agent to be properly installed and configured. - -### Defining a configuration repository +### Define a configuration repository -Next you will need a GitLab repository that will contain your Agent configuration. +Next, you need a GitLab repository to contain your Agent configuration. The minimal +repository layout looks like this: -The minimal repository layout looks like this: - -`.gitlab/agents/<agent-name>/config.yaml` +```plaintext +.gitlab/agents/<agent-name>/config.yaml +``` The `config.yaml` file contents should look like this: @@ -97,31 +119,24 @@ gitops: - id: "path-to/your-awesome-project" ``` -### Creating an Agent record in GitLab - -Next you will need to create an GitLab Rails Agent record so that your GitLab project so that the Agent itself can associate with a GitLab project. This process will also yield a Secret that you will use to configure the Agent in subsequent steps. +### Create an Agent record in GitLab -There are two ways to accomplish this: +Next, create an GitLab Rails Agent record so the Agent can associate itself with +the configuration repository project. Creating this record also creates a Secret needed to configure +the Agent in subsequent steps. You can create an Agent record either: -1. Via the Rails console -1. Via GraphQL +- Through the Rails console, by running `rails c`: -To do this you could either run `rails c` or via GraphQL. From `rails c`: - -```ruby + ```ruby project = ::Project.find_by_full_path("path-to/your-awesome-project") agent = ::Clusters::Agent.create(name: "<agent-name>", project: project) token = ::Clusters::AgentToken.create(agent: agent) token.token # this will print out the token you need to use on the next step -``` + ``` -or using GraphQL: +- Through GraphQL: **(PREMIUM ONLY)** -with this approach, you'll need a premium license to use this feature. - -If you are new to using the GitLab GraphQL API please refer to the [Getting started with the GraphQL API page](../../../api/graphql/getting_started.md) or check out the [GraphQL Explorer](https://gitlab.com/-/graphql-explorer). - -```json + ```graphql mutation createAgent { createClusterAgent(input: { projectPath: "path-to/your-awesome-project", name: "<agent-name>" }) { clusterAgent { @@ -142,37 +157,77 @@ If you are new to using the GitLab GraphQL API please refer to the [Getting star errors } } -``` + ``` -Note that GraphQL will only show you the token once, after you've created it. + NOTE: **Note:** + GraphQL only displays the token once, after creating it. -### Creating the Kubernetes secret + If you are new to using the GitLab GraphQL API, refer to the + [Getting started with the GraphQL API page](../../../api/graphql/getting_started.md), + or the [GraphQL Explorer](https://gitlab.com/-/graphql-explorer). -Once the token has been generated it needs to be applied to the Kubernetes cluster. +### Create the Kubernetes secret -If you didn't previously define or create a namespace you need to do that first: +After generating the token, you must apply it to the Kubernetes cluster. -```shell -kubectl create namespace <YOUR-DESIRED-NAMESPACE> -``` +1. If you haven't previous defined or created a namespace, run the following command: + + ```shell + kubectl create namespace <YOUR-DESIRED-NAMESPACE> + ``` + +1. Run the following command to create your Secret: + + ```shell + kubectl create secret generic -n <YOUR-DESIRED-NAMESPACE> gitlab-agent-token --from-literal=token='YOUR_AGENT_TOKEN' + ``` -Run the following command to create your Secret: +### Install the Agent into the cluster + +Next, install the in-cluster component of the Agent. This example file contains the +Kubernetes resources required for the Agent to be installed. You can modify this +example [`resources.yml` file](#example-resourcesyml-file) in the following ways: + +- You can replace `gitlab-agent` with `<YOUR-DESIRED-NAMESPACE>`. +- For the `kas-address` (Kubernetes Agent Server), the agent can use the WebSockets + or gRPC protocols to connect to the Agent Server. Depending on your cluster + configuration and GitLab architecture, you may need to use one or the other. + For the `gitlab-kas` Helm chart, an Ingress is created for the Agent Server using + the `/-/kubernetes-agent` endpoint. This can be used for the WebSockets protocol connection. + - Specify the `grpc` scheme (such as `grpc://gitlab-kas:5005`) to use gRPC directly. + Encrypted gRPC is not supported yet. Follow the + [Support TLS for gRPC communication issue](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/7) + for progress updates. + - Specify the `ws` scheme (such as `ws://gitlab-kas-ingress:80/-/kubernetes-agent`) + to use an unencrypted WebSockets connection. + - Specify the `wss` scheme (such as `wss://gitlab-kas-ingress:443/-/kubernetes-agent`) + to use an encrypted WebSockets connection. This is the recommended option if + installing the Agent into a separate cluster from your Agent Server. +- If you defined your own secret name, replace `gitlab-agent-token` with your secret name. + +To apply this file, run the following command: ```shell -kubectl create secret generic -n <YOUR-DESIRED-NAMESPACE> gitlab-agent-token --from-literal=token='YOUR_AGENT_TOKEN' +kubectl apply -n gitlab-agent -f ./resources.yml ``` -### Installing the Agent into the cluster +To review your configuration, run the following command: -Next you are now ready to install the in-cluster component of the Agent. The below is an example YAML file of the Kubernetes resources required for the Agent to be installed. - -Let's highlight a few of the details in the example below: +```shell +$ kubectl get pods --all-namespaces -1. You can replace `gitlab-agent` with <YOUR-DESIRED-NAMESPACE> -1. For the `kas-address` (Kubernetes Agent Server), you can replace `grpc://host.docker.internal:5005` with the address of the kas agent that was initialized via your Helm install. -1. If you defined your own secret name, then replace `gitlab-agent-token` with your secret name. +NAMESPACE NAME READY STATUS RESTARTS AGE +gitlab-agent gitlab-agent-77689f7dcb-5skqk 1/1 Running 0 51s +kube-system coredns-f9fd979d6-n6wcw 1/1 Running 0 14m +kube-system etcd-minikube 1/1 Running 0 14m +kube-system kube-apiserver-minikube 1/1 Running 0 14m +kube-system kube-controller-manager-minikube 1/1 Running 0 14m +kube-system kube-proxy-j6zdh 1/1 Running 0 14m +kube-system kube-scheduler-minikube 1/1 Running 0 14m +kube-system storage-provisioner 1/1 Running 0 14m +``` -`./resources.yml` +#### Example `resources.yml` file ```yaml apiVersion: v1 @@ -201,7 +256,7 @@ spec: args: - --token-file=/config/token - --kas-address - - grpc://host.docker.internal:5005 # {"$openapi":"kas-address"} + - grpc://host.docker.internal:5005 # {"$openapi":"kas-address"} volumeMounts: - name: token-volume mountPath: /config @@ -269,31 +324,24 @@ subjects: - name: gitlab-agent kind: ServiceAccount namespace: gitlab-agent - ``` -```shell -kubectl apply -n gitlab-agent -f ./resources.yml -``` +### Create a `manifest.yaml` + +In a previous step, you configured a `config.yaml` to point to the GitLab projects +the Agent should synchronize. In each of those projects, you must create a `manifest.yaml` +file for the Agent to monitor. You can auto-generate this `manifest.yaml` with a +templating engine or other means. + +Each time you commit and push a change to this file, the Agent logs the change: ```plaintext -$ kubectl get pods --all-namespaces -NAMESPACE NAME READY STATUS RESTARTS AGE -gitlab-agent gitlab-agent-77689f7dcb-5skqk 1/1 Running 0 51s -kube-system coredns-f9fd979d6-n6wcw 1/1 Running 0 14m -kube-system etcd-minikube 1/1 Running 0 14m -kube-system kube-apiserver-minikube 1/1 Running 0 14m -kube-system kube-controller-manager-minikube 1/1 Running 0 14m -kube-system kube-proxy-j6zdh 1/1 Running 0 14m -kube-system kube-scheduler-minikube 1/1 Running 0 14m -kube-system storage-provisioner 1/1 Running 0 14m +2020-09-15_14:09:04.87946 gitlab-k8s-agent : time="2020-09-15T10:09:04-04:00" level=info msg="Config: new commit" agent_id=1 commit_id=e6a3651f1faa2e928fe6120e254c122451be4eea ``` -### Creating a `manifest.yaml` - -In the above step, you configured a `config.yaml` to point to which GitLab projects the Agent should synchronize. Within each one of those projects, you need to create a `manifest.yaml` file which the Agent will monitor. This `manifest.yaml` can be autogenerated by a templating engine or other means. +#### Example `manifest.yaml` file -Example `manifest.yaml`: +This file creates a simple NGINX deployment. ```yaml apiVersion: apps/v1 @@ -318,14 +366,9 @@ spec: - containerPort: 80 ``` -The above file creates a simple NGINX deployment. - -Each time you commit and push a change to the `manifest.yaml` the Agent will observe the change. Example log: - -```plaintext -2020-09-15_14:09:04.87946 gitlab-k8s-agent : time="2020-09-15T10:09:04-04:00" level=info msg="Config: new commit" agent_id=1 commit_id=e6a3651f1faa2e928fe6120e254c122451be4eea -``` - ## Example projects -Basic GitOps example deploying NGINX: [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent), [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) +This basic GitOps example deploys NGINX: + +- [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent) +- [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 2243ffa0cb1..4e0e2991acb 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -6,37 +6,30 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Managed Apps -GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can -be added directly to your configured cluster. +GitLab provides **GitLab Managed Apps** for various +applications which can be added directly to your configured cluster. These +applications are needed for [Review Apps](../../ci/review_apps/index.md) and +[deployments](../../ci/environments/index.md) when using [Auto DevOps](../../topics/autodevops/index.md). +You can install them after you [create a cluster](../project/clusters/add_remove_clusters.md). GitLab provides +GitLab Managed Apps that can installed with [one-click](#install-with-one-click) or [using CI/CD](#install-using-gitlab-cicd-alpha). -These applications are needed for [Review Apps](../../ci/review_apps/index.md) -and [deployments](../../ci/environments/index.md) when using [Auto DevOps](../../topics/autodevops/index.md). +## Install with one click -You can install them after you -[create a cluster](../project/clusters/add_remove_clusters.md). - -## Installing applications - -Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. - -This namespace: +Applications managed by GitLab are installed onto the `gitlab-managed-apps` +namespace. This namespace: - Is different from the namespace used for project deployments. - Is created once. - Has a non-configurable name. -To see a list of available applications to install. For a: +To view a list of available applications to install for a: - [Project-level cluster](../project/clusters/index.md), navigate to your project's **Operations > Kubernetes**. - [Group-level cluster](../group/clusters/index.md), navigate to your group's **Kubernetes** page. -NOTE: **Note:** -As of GitLab 11.6, Helm will be upgraded to the latest version supported -by GitLab before installing any of the applications. - -The following applications can be installed: +You can install the following applications with one click: - [Helm](#helm) - [Ingress](#ingress) @@ -49,27 +42,28 @@ The following applications can be installed: - [Elastic Stack](#elastic-stack) - [Fluentd](#fluentd) -With the exception of Knative, the applications will be installed in a dedicated +With the exception of Knative, the applications are installed in a dedicated namespace called `gitlab-managed-apps`. -NOTE: **Note:** Some applications are installable only for a project-level cluster. Support for installing these applications in a group-level cluster is planned for future releases. -For updates, see [the issue tracking -progress](https://gitlab.com/gitlab-org/gitlab/-/issues/24411). +For updates, see the [issue tracking progress](https://gitlab.com/gitlab-org/gitlab/-/issues/24411). CAUTION: **Caution:** If you have an existing Kubernetes cluster with Helm already installed, you should be careful as GitLab cannot detect it. In this case, installing -Helm via the applications will result in the cluster having it twice, which +Helm with the applications results in the cluster having it twice, which can lead to confusion during deployments. +In GitLab versions 11.6 and greater, Helm is upgraded to the latest version +supported by GitLab before installing any of the applications. + ### Helm > - Introduced in GitLab 10.2 for project-level clusters. > - Introduced in GitLab 11.6 for group-level clusters. -> - [Uses a local Tiller](https://gitlab.com/gitlab-org/gitlab/-/issues/209736) since GitLab 13.2. +> - [Uses a local Tiller](https://gitlab.com/gitlab-org/gitlab/-/issues/209736) in GitLab 13.2 and later. [Helm](https://helm.sh/docs/) is a package manager for Kubernetes and is used to install the GitLab-managed apps. GitLab runs each `helm` command @@ -81,7 +75,6 @@ applications. Prior to [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issu GitLab used an in-cluster Tiller server in the `gitlab-managed-apps` namespace. This server can now be safely removed. -NOTE: **Note:** GitLab's Helm integration does not support installing applications behind a proxy, but a [workaround](../../topics/autodevops/index.md#install-applications-behind-a-proxy) is available. @@ -90,26 +83,25 @@ is available. > Introduced in GitLab 11.6 for project- and group-level clusters. -[cert-manager](https://cert-manager.io/docs/) is a native -Kubernetes certificate management controller that helps with issuing -certificates. Installing cert-manager on your cluster will issue a -certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that -certificates are valid and up-to-date. +[cert-manager](https://cert-manager.io/docs/) is a native Kubernetes certificate +management controller that helps with issuing certificates. Installing +cert-manager on your cluster issues a certificate by [Let's Encrypt](https://letsencrypt.org/) +and ensures that certificates are valid and up-to-date. The chart used to install this application depends on the version of GitLab used. In: -- GitLab 12.3 and newer, the [jetstack/cert-manager](https://github.com/jetstack/cert-manager) +- GitLab 12.3 and newer, the [`jetstack/cert-manager`](https://github.com/jetstack/cert-manager) chart is used with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/cert_manager/values.yaml) file. -- GitLab 12.2 and older, the [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) +- GitLab 12.2 and older, the [`stable/cert-manager`](https://gi2wthub.com/helm/charts/tree/master/stable/cert-manager) chart was used. -If you have installed cert-manager prior to GitLab 12.3, Let's Encrypt will -[block requests from older versions of cert-manager](https://community.letsencrypt.org/t/blocking-old-cert-manager-versions/98753). - -To resolve this: +If you installed cert-manager prior to GitLab 12.3, Let's Encrypt +[blocks requests](https://community.letsencrypt.org/t/blocking-old-cert-manager-versions/98753) +from older versions of `cert-manager`. To resolve this: -1. Uninstall cert-manager (consider [backing up any additional configuration](https://cert-manager.io/docs/tutorials/backup/)). +1. [Back up any additional configuration](https://cert-manager.io/docs/tutorials/backup/). +1. Uninstall cert-manager. 1. Install cert-manager again. ### GitLab Runner @@ -117,52 +109,52 @@ To resolve this: > - Introduced in GitLab 10.6 for project-level clusters. > - Introduced in GitLab 11.10 for group-level clusters. -[GitLab Runner](https://docs.gitlab.com/runner/) is the open source -project that is used to run your jobs and send the results back to -GitLab. It is used in conjunction with [GitLab -CI/CD](../../ci/README.md), the open-source continuous integration -service included with GitLab that coordinates the jobs. - -If the project is on GitLab.com, shared runners are available -(the first 2000 minutes are free, you can -[buy more later](../../subscriptions/gitlab_com/index.md#purchase-additional-ci-minutes)) -and you do not have to deploy one if they are enough for your needs. If a -project-specific runner is desired, or there are no shared runners, it is easy -to deploy one. - -Note that the deployed runner will be set as **privileged**, which means it will essentially -have root access to the underlying machine. This is required to build Docker images, -so it is the default. Make sure you read the -[security implications](../project/clusters/index.md#security-implications) +[GitLab Runner](https://docs.gitlab.com/runner/) is the open source project that +is used to run your jobs and send the results back to GitLab. It's used in +conjunction with [GitLab CI/CD](../../ci/README.md), the open-source continuous +integration service included with GitLab that coordinates the jobs. + +If the project is on GitLab.com, [shared runners](../gitlab_com/index.md#shared-runners) +are available. You don't have to deploy one if they are enough for your +needs. If a project-specific runner is desired, or there are no shared runners, +you can deploy one. + +The deployed runner is set as **privileged**. Root access to the underlying +server is required to build Docker images, so it's the default. Be sure to read +the [security implications](../project/clusters/index.md#security-implications) before deploying one. -NOTE: **Note:** The [`runner/gitlab-runner`](https://gitlab.com/gitlab-org/charts/gitlab-runner) chart is used to install this application, using [a preconfigured `values.yaml`](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/master/values.yaml) -file. Customizing the installation by modifying this file is not supported. +file. Customizing the installation by modifying this file is not supported. This +also means you cannot modify `config.toml` file for this Runner. If you want to +have that possibility and still deploy Runner in Kubernetes, consider using the +[Cluster management project](management_project.md) or installing Runner manually +via [GitLab Runner Helm Chart](https://docs.gitlab.com/runner/install/kubernetes.html). ### Ingress > - Introduced in GitLab 10.2 for project-level clusters. > - Introduced in GitLab 11.6 for group-level clusters. -[Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) provides load balancing, SSL termination, and name-based virtual hosting +[Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) +provides load balancing, SSL termination, and name-based virtual hosting out of the box. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../topics/autodevops/index.md) or deploy your own web apps. -The Ingress Controller installed is [Ingress-NGINX](https://kubernetes.io/docs/concepts/services-networking/ingress/), +The Ingress Controller installed is +[Ingress-NGINX](https://kubernetes.io/docs/concepts/services-networking/ingress/), which is supported by the Kubernetes community. -NOTE: **Note:** With the following procedure, a load balancer must be installed in your cluster to obtain the endpoint. You can use either Ingress, or Knative's own load balancer ([Istio](https://istio.io)) if using Knative. -In order to publish your web application, you first need to find the endpoint which will be either an IP +To publish your web application, you first need to find the endpoint, which is either an IP address or a hostname associated with your load balancer. -To install it, click on the **Install** button for Ingress. GitLab will attempt +To install it, click on the **Install** button for Ingress. GitLab attempts to determine the external endpoint and it should be available within a few minutes. #### Determining the external endpoint automatically @@ -178,22 +170,25 @@ using the `KUBE_INGRESS_BASE_DOMAIN` environment variable. If the endpoint doesn't appear and your cluster runs on Google Kubernetes Engine: -1. Check your [Kubernetes cluster on Google Kubernetes Engine](https://console.cloud.google.com/kubernetes) to ensure there are no errors on its nodes. -1. Ensure you have enough [Quotas](https://console.cloud.google.com/iam-admin/quotas) on Google Kubernetes Engine. For more information, see [Resource Quotas](https://cloud.google.com/compute/quotas). -1. Check [Google Cloud's Status](https://status.cloud.google.com/) to ensure they are not having any disruptions. +1. [Examine your Kubernetes cluster](https://console.cloud.google.com/kubernetes) + on Google Kubernetes Engine to ensure there are no errors on its nodes. +1. Ensure you have enough [Quotas](https://console.cloud.google.com/iam-admin/quotas) + on Google Kubernetes Engine. For more information, see + [Resource Quotas](https://cloud.google.com/compute/quotas). +1. Review [Google Cloud's Status](https://status.cloud.google.com/) for service + disruptions. -Once installed, you may see a `?` for "Ingress IP Address" depending on the -cloud provider. For EKS specifically, this is because the ELB is created -with a DNS name, not an IP address. If GitLab is still unable to -determine the endpoint of your Ingress or Knative application, you can -[determine it manually](#determining-the-external-endpoint-manually). - -NOTE: **Note:** The [`stable/nginx-ingress`](https://github.com/helm/charts/tree/master/stable/nginx-ingress) chart is used to install this application with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/ingress/values.yaml) file. +After installing, you may see a `?` for **Ingress IP Address** depending on the +cloud provider. For EKS specifically, this is because the ELB is created +with a DNS name, not an IP address. If GitLab is still unable to +determine the endpoint of your Ingress or Knative application, you can +[determine it manually](#determining-the-external-endpoint-manually). + #### Determining the external endpoint manually If the cluster is on GKE, click the **Google Kubernetes Engine** link in the @@ -204,62 +199,62 @@ the `gcloud` command in a local terminal or using the **Cloud Shell**. If the cluster is not on GKE, follow the specific instructions for your Kubernetes provider to configure `kubectl` with the right credentials. -The output of the following examples will show the external endpoint of your +The output of the following examples show the external endpoint of your cluster. This information can then be used to set up DNS entries and forwarding rules that allow external access to your deployed applications. -If you installed Ingress via the **Applications**, run the following command: +- If you installed Ingress using the **Applications**, run the following + command: -```shell -kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' -``` + ```shell + kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' + ``` -Some Kubernetes clusters return a hostname instead, like [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: +- Some Kubernetes clusters return a hostname instead, like + [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: -```shell -kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' -``` + ```shell + kubectl get service --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' + ``` -For Istio/Knative, the command will be different: + If EKS is used, an [Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/) + is also created, which will incur additional AWS costs. -```shell -kubectl get svc --namespace=istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' -``` +- For Istio/Knative, the command is different: -Otherwise, you can list the IP addresses of all load balancers: + ```shell + kubectl get svc --namespace=istio-system istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} ' + ``` -```shell -kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' -``` +- Otherwise, you can list the IP addresses of all load balancers: -NOTE: **Note:** -If EKS is used, an [Elastic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/) -will also be created, which will incur additional AWS costs. + ```shell + kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' + ``` -NOTE: **Note:** -You may see a trailing `%` on some Kubernetes versions, **do not include it**. +You may see a trailing `%` on some Kubernetes versions. Do not include it. -The Ingress is now available at this address and will route incoming requests to -the proper service based on the DNS name in the request. To support this, a -wildcard DNS CNAME record should be created for the desired domain name. For example, +The Ingress is now available at this address, and routes incoming requests to +the proper service based on the DNS name in the request. To support this, create +a wildcard DNS CNAME record for the desired domain name. For example, `*.myekscluster.com` would point to the Ingress hostname obtained earlier. #### Using a static IP By default, an ephemeral external IP address is associated to the cluster's load balancer. If you associate the ephemeral IP with your DNS and the IP changes, -your apps will not be able to be reached, and you'd have to change the DNS -record again. In order to avoid that, you should change it into a static -reserved IP. +your apps won't be reachable, and you'd have to change the DNS record again. +To avoid that, change it into a static reserved IP. Read how to [promote an ephemeral external IP address in GKE](https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address#promote_ephemeral_ip). #### Pointing your DNS at the external endpoint -Once you've set up the external endpoint, you should associate it with a [wildcard DNS -record](https://en.wikipedia.org/wiki/Wildcard_DNS_record) such as `*.example.com.` -in order to be able to reach your apps. If your external endpoint is an IP address, -use an A record. If your external endpoint is a hostname, use a CNAME record. +After you have set up the external endpoint, associate it with a +[wildcard DNS record](https://en.wikipedia.org/wiki/Wildcard_DNS_record) (such +as `*.example.com.`) to reach your apps. If your external endpoint is an IP +address, use an A record. If your external endpoint is a hostname, use a CNAME +record. #### Web Application Firewall (ModSecurity) @@ -269,16 +264,15 @@ A Web Application Firewall (WAF) examines traffic being sent or received, and can block malicious traffic before it reaches your application. The benefits of a WAF are: -- Real-time security monitoring for your application -- Logging of all your HTTP traffic to the application -- Access control for your application -- Highly configurable logging and blocking rules +- Real-time security monitoring for your application. +- Logging of all your HTTP traffic to the application. +- Access control for your application. +- Highly configurable logging and blocking rules. -Out of the box, GitLab provides you with a WAF known as [`ModSecurity`](https://www.modsecurity.org/). - -ModSecurity is a toolkit for real-time web application monitoring, logging, -and access control. With GitLab's offering, the [OWASP's Core Rule Set](https://www.modsecurity.org/CRS/Documentation/), -which provides generic attack detection capabilities, is automatically applied. +By default, GitLab provides you with a WAF known as [`ModSecurity`](https://www.modsecurity.org/), +which is a toolkit for real-time web application monitoring, logging, and access +control. GitLab's offering applies the [OWASP's Core Rule Set](https://www.modsecurity.org/CRS/Documentation/), +which provides generic attack detection capabilities. This feature: @@ -299,57 +293,61 @@ There is a small performance overhead by enabling ModSecurity. If this is considered significant for your application, you can disable ModSecurity's rule engine for your deployed application in any of the following ways: -1. Setting [the deployment variable](../../topics/autodevops/index.md) -`AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` to `Off`. This will prevent ModSecurity -from processing any requests for the given application or environment. - -1. Switching its respective toggle to the disabled position and applying changes through the **Save changes** button. This will reinstall -Ingress with the recent changes. +1. Set the [deployment variable](../../topics/autodevops/index.md) + `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` to `Off` to prevent ModSecurity + from processing any requests for the given application or environment. +1. Switch its respective toggle to the disabled position, and then apply changes + by selecting **Save changes** to reinstall Ingress with the recent changes.  ##### Logging and blocking modes To help you tune your WAF rules, you can globally set your WAF to either -**Logging** or **Blocking** mode: +*Logging* or *Blocking* mode: -- **Logging mode** - Allows traffic matching the rule to pass, and logs the event. -- **Blocking mode** - Prevents traffic matching the rule from passing, and logs the event. +- *Logging mode*: Allows traffic matching the rule to pass, and logs the event. +- *Blocking mode*: Prevents traffic matching the rule from passing, and logs the event. To change your WAF's mode: -1. [Install ModSecurity](../../topics/web_application_firewall/quick_start_guide.md) if you have not already done so. +1. If you haven't already done so, [install ModSecurity](../../topics/web_application_firewall/quick_start_guide.md). 1. Navigate to **Operations > Kubernetes**. 1. In **Applications**, scroll to **Ingress**. 1. Under **Global default**, select your desired mode. -1. Click **Save changes**. +1. Select **Save changes**. ##### WAF version updates -Enabling, disabling, or changing the logging mode for **ModSecurity** is only allowed within same version of [Ingress](#ingress) due to limitations in [Helm](https://helm.sh/) which might be overcome in future releases. +Enabling, disabling, or changing the logging mode for **ModSecurity** is only +allowed within same version of [Ingress](#ingress) due to limitations in +[Helm](https://helm.sh/) which might be overcome in future releases. -**ModSecurity** UI controls are disabled if the version deployed differs from the one available in GitLab, while actions at the [Ingress](#ingress) level, such as uninstalling, can still be performed: +**ModSecurity** user interface controls are disabled if the version deployed +differs from the one available in GitLab, while actions at the [Ingress](#ingress) +level, such as uninstalling, can still be performed:  -Updating [Ingress](#ingress) to the most recent version enables you to take advantage of bug fixes, security fixes, and performance improvements. To update [Ingress application](#ingress), you must first uninstall it, and then re-install it as described in [Install ModSecurity](../../topics/web_application_firewall/quick_start_guide.md). +Update [Ingress](#ingress) to the most recent version to take advantage of bug +fixes, security fixes, and performance improvements. To update the +[Ingress application](#ingress), you must first uninstall it, and then re-install +it as described in [Install ModSecurity](../../topics/web_application_firewall/quick_start_guide.md). ##### Viewing Web Application Firewall traffic > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14707) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. You can view Web Application Firewall traffic by navigating to your project's -**Security & Compliance > Threat Monitoring** page. - -From there, you can see tracked over time: +**Security & Compliance > Threat Monitoring** page. From there, you can see +tracked over time: - The total amount of traffic to your application. -- The proportion of traffic that is considered anomalous by the Web Application +- The proportion of traffic that's considered anomalous by the Web Application Firewall's default [OWASP ruleset](https://www.modsecurity.org/CRS/Documentation/). -If a significant percentage of traffic is anomalous, it should be investigated -for potential threats, which can be done by -[examining the Web Application Firewall logs](#web-application-firewall-modsecurity). +If a significant percentage of traffic is anomalous, investigate it for potential threats +by [examining the Web Application Firewall logs](#web-application-firewall-modsecurity).  @@ -358,55 +356,52 @@ for potential threats, which can be done by > - Introduced in GitLab 11.0 for project-level clusters. > - Introduced in GitLab 12.3 for group and instance-level clusters. -[JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a -multi-user service for managing notebooks across a team. [Jupyter -Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a -web-based interactive programming environment used for data analysis, +[JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service +for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) +provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. -Authentication will be enabled only for [project -members](../project/members/index.md) for project-level clusters and group -members for group-level clusters with [Developer or -higher](../permissions.md) access to the associated project or group. - -We use a [custom Jupyter -image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) -that installs additional useful packages on top of the base Jupyter. You -will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/Nurtch/rubix). - -More information on -creating executable runbooks can be found in [our Runbooks -documentation](../project/clusters/runbooks/index.md#configure-an-executable-runbook-with-gitlab). Note that -Ingress must be installed and have an IP address assigned before -JupyterHub can be installed. - -NOTE: **Note:** The [`jupyter/jupyterhub`](https://jupyterhub.github.io/helm-chart/) chart is used to install this application with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/jupyter/values.yaml) file. +Authentication is enabled only for [project members](../project/members/index.md) +for project-level clusters and group members for group-level clusters with +[Developer or higher](../permissions.md) access to the associated project or group. + +GitLab uses a [custom Jupyter image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) +that installs additional useful packages on top of the base Jupyter. Ready-to-use +DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/Nurtch/rubix) +are also available. + +More information on creating executable runbooks can be found in +[our Runbooks documentation](../project/clusters/runbooks/index.md#configure-an-executable-runbook-with-gitlab). +Ingress must be installed and have an IP address assigned before +JupyterHub can be installed. + #### Jupyter Git Integration > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/28783) in GitLab 12.0 for project-level clusters. > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/32512) in GitLab 12.3 for group and instance-level clusters. -When installing JupyterHub onto your Kubernetes cluster, [JupyterLab's Git extension](https://github.com/jupyterlab/jupyterlab-git) -is automatically provisioned and configured using the authenticated user's: +When installing JupyterHub onto your Kubernetes cluster, +[JupyterLab's Git extension](https://github.com/jupyterlab/jupyterlab-git) +is provisioned and configured using the authenticated user's: - Name. - Email. - Newly created access token. -JupyterLab's Git extension enables full version control of your notebooks as well as issuance of Git commands within Jupyter. -Git commands can be issued via the **Git** tab on the left panel or via Jupyter's command line prompt. +JupyterLab's Git extension enables full version control of your notebooks, and +issuance of Git commands within Jupyter. You can issue Git commands through the +**Git** tab on the left panel, or through Jupyter's command-line prompt. -NOTE: **Note:** -JupyterLab's Git extension stores the user token in the JupyterHub DB in encrypted format -and in the single user Jupyter instance as plain text. This is because [Git requires storing -credentials as plain text](https://git-scm.com/docs/git-credential-store). Potentially, if -a nefarious user finds a way to read from the file system in the single user Jupyter instance -they could retrieve the token. +JupyterLab's Git extension stores the user token in the JupyterHub DB in encrypted +format, and in the single user Jupyter instance as plain text, because +[Git requires storing credentials as plain text](https://git-scm.com/docs/git-credential-store) +Potentially, if a nefarious user finds a way to read from the file system in the +single-user Jupyter instance, they could retrieve the token.  @@ -421,22 +416,20 @@ You can clone repositories from the files tab in Jupyter: [Knative](https://cloud.google.com/knative/) provides a platform to create, deploy, and manage serverless workloads from a Kubernetes -cluster. It is used in conjunction with, and includes +cluster. It's used in conjunction with, and includes [Istio](https://istio.io) to provide an external IP address for all programs hosted by Knative. -You will be prompted to enter a wildcard -domain where your applications will be exposed. Configure your DNS -server to use the external IP address for that domain. For any -application created and installed, they will be accessible as -`<program_name>.<kubernetes_namespace>.<domain_name>`. This will require -your Kubernetes cluster to have [RBAC -enabled](../project/clusters/add_remove_clusters.md#rbac-cluster-resources). - -NOTE: **Note:** The [`knative/knative`](https://storage.googleapis.com/triggermesh-charts) chart is used to install this application. +During installation, you must enter a wildcard domain where your applications +will be exposed. Configure your DNS server to use the external IP address for that +domain. Applications created and installed are accessible as +`<program_name>.<kubernetes_namespace>.<domain_name>`, which requires +your Kubernetes cluster to have +[RBAC enabled](../project/clusters/add_remove_clusters.md#rbac-cluster-resources). + ### Prometheus > - Introduced in GitLab 10.4 for project-level clusters. @@ -446,20 +439,19 @@ chart is used to install this application. open-source monitoring and alerting system useful to supervise your deployed applications. -GitLab is able to monitor applications automatically, using the +GitLab is able to monitor applications by using the [Prometheus integration](../project/integrations/prometheus.md). Kubernetes container CPU and -memory metrics are automatically collected, and response metrics are retrieved -from NGINX Ingress as well. +memory metrics are collected, and response metrics are also retrieved +from NGINX Ingress. -To enable monitoring, simply install Prometheus into the cluster with the -**Install** button. - -NOTE: **Note:** The [`stable/prometheus`](https://github.com/helm/charts/tree/master/stable/prometheus) chart is used to install this application with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/prometheus/values.yaml) file. +To enable monitoring, install Prometheus into the cluster with the **Install** +button. + ### Crossplane > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34702) in GitLab 12.5 for project-level clusters. @@ -481,17 +473,16 @@ The Crossplane GitLab-managed application: project repository. - Can then be used to provision infrastructure or managed applications such as PostgreSQL (for example, CloudSQL from GCP or RDS from AWS) and other services - required by the application via the Auto DevOps pipeline. + required by the application with the Auto DevOps pipeline. -For information on configuring Crossplane installed on the cluster, see -[Crossplane configuration](crossplane.md). - -NOTE: **Note:** [`alpha/crossplane`](https://github.com/crossplane/crossplane/tree/v0.4.1/cluster/charts/crossplane) chart v0.4.1 is used to install Crossplane using the [`values.yaml`](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) file. +For information about configuring Crossplane installed on the cluster, see +[Crossplane configuration](crossplane.md). + ### Elastic Stack > Introduced in GitLab 12.7 for project- and group-level clusters. @@ -500,37 +491,32 @@ file. log analysis solution which helps in deep searching, analyzing and visualizing the logs generated from different machines. -GitLab is able to gather logs from pods in your cluster automatically. -Filebeat will run as a DaemonSet on each node in your cluster, and it will ship container logs to Elasticsearch for querying. -GitLab will then connect to Elasticsearch for logs instead of the Kubernetes API, -and you will have access to more advanced querying capabilities. +GitLab can gather logs from pods in your cluster. Filebeat runs as a DaemonSet +on each node in your cluster, and ships container logs to Elasticsearch for +querying. GitLab then connects to Elasticsearch for logs, instead of the +Kubernetes API, giving you access to more advanced querying capabilities. Log +data is deleted after 30 days, using [Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/about.html). -Log data is automatically deleted after 30 days using [Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/about.html). +The Elastic Stack cluster application is intended as a log aggregation solution +and is not related to our [Advanced Search](../search/advanced_global_search.md) +functionality, which uses a separate Elasticsearch cluster. To enable log shipping: -1. Ensure your cluster contains at least 3 nodes of instance types larger than - `f1-micro`, `g1-small`, or `n1-standard-1`. +1. Ensure your cluster contains at least three nodes of instance types larger + than `f1-micro`, `g1-small`, or `n1-standard-1`. 1. Navigate to **Operations > Kubernetes**. 1. In **Kubernetes Cluster**, select a cluster. -1. In the **Applications** section, find **Elastic Stack** and click **Install**. +1. In the **Applications** section, find **Elastic Stack**, and then select + **Install**. -NOTE: **Note:** The [`gitlab/elastic-stack`](https://gitlab.com/gitlab-org/charts/elastic-stack) chart is used to install this application with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/elastic_stack/values.yaml) -file. - -NOTE: **Note:** -The chart deploys 3 identical Elasticsearch pods which can't be colocated, and each -requires 1 CPU and 2 GB of RAM, making them incompatible with clusters containing -fewer than 3 nodes or consisting of `f1-micro`, `g1-small`, `n1-standard-1`, or -`*-highcpu-2` instance types. - -NOTE: **Note:** -The Elastic Stack cluster application is intended as a log aggregation solution and is not related to our -[Advanced Search](../search/advanced_global_search.md) functionality, which uses a separate -Elasticsearch cluster. +file. The chart deploys three identical Elasticsearch pods which can't be +colocated, and each requires one CPU and 2 GB of RAM, making them +incompatible with clusters containing fewer than three nodes, or consisting of +`f1-micro`, `g1-small`, `n1-standard-1`, or `*-highcpu-2` instance types. #### Optional: deploy Kibana to perform advanced queries @@ -578,8 +564,8 @@ your data. Fluentd sends logs in syslog format. To enable Fluentd: 1. Navigate to **Operations > Kubernetes** and click - **Applications**. You will be prompted to enter a host, port and protocol - where the WAF logs will be sent to via syslog. + **Applications**. Enter a host, port, and protocol + for sending the WAF logs with syslog. 1. Provide the host domain name or URL in **SIEM Hostname**. 1. Provide the host port number in **SIEM Port**. 1. Select a **SIEM Protocol**. @@ -599,7 +585,7 @@ to get started. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20822) in GitLab 12.6. CAUTION: **Warning:** -This is an _alpha_ feature, and it is subject to change at any time without +This is an _alpha_ feature, and is subject to change at any time without prior notice. This alternative method allows users to install GitLab-managed @@ -639,7 +625,6 @@ To install applications using GitLab CI/CD: - template: Managed-Cluster-Applications.gitlab-ci.yml ``` - NOTE: **Note:** The job provided by this template connects to the cluster using tools provided in a custom Docker image. It requires that you have a runner registered with the Docker, Kubernetes, or Docker Machine executor. @@ -657,11 +642,10 @@ To install applications using GitLab CI/CD: 1. Optionally, define `.gitlab/managed-apps/<application>/values.yaml` file to customize values for the installed application. -A GitLab CI/CD pipeline will then run on the `master` branch to install the +A GitLab CI/CD pipeline runs on the `master` branch to install the applications you have configured. In case of pipeline failure, the -output of the [Helm -Tiller](https://v2.helm.sh/docs/install/#running-tiller-locally) binary -will be saved as a [CI job artifact](../../ci/pipelines/job_artifacts.md). +output of the [Helm Tiller](https://v2.helm.sh/docs/install/#running-tiller-locally) binary +is saved as a [CI job artifact](../../ci/pipelines/job_artifacts.md). ### Important notes @@ -669,10 +653,10 @@ Note the following: - We recommend using the cluster management project exclusively for managing deployments to a cluster. Do not add your application's source code to such projects. -- When you set the value for `installed` key back to `false`, the application will be +- When you set the value for `installed` key back to `false`, the application is unprovisioned from the cluster. - If you update `.gitlab/managed-apps/<application>/values.yaml` with new values, the - application will be redeployed. + application is redeployed. ### Install Ingress using GitLab CI/CD @@ -684,7 +668,7 @@ ingress: installed: true ``` -Ingress will then be installed into the `gitlab-managed-apps` namespace +Ingress is installed into the `gitlab-managed-apps` namespace of your cluster. You can customize the installation of Ingress by defining a @@ -693,9 +677,10 @@ management project. Refer to the [chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) for the available configuration options. -NOTE: **Note:** Support for installing the Ingress managed application is provided by the GitLab Configure group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). +If you run into unknown issues, [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), +and ping at least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). ### Install cert-manager using GitLab CI/CD @@ -705,7 +690,8 @@ cert-manager is installed using GitLab CI/CD by defining configuration in cert-manager: - Is installed into the `gitlab-managed-apps` namespace of your cluster. -- Can be installed with or without a default [Let's Encrypt `ClusterIssuer`](https://cert-manager.io/docs/configuration/acme/), which requires an +- Can be installed with or without a default + [Let's Encrypt `ClusterIssuer`](https://cert-manager.io/docs/configuration/acme/), which requires an email address to be specified. The email address is used by Let's Encrypt to contact you about expiring certificates and issues related to your account. @@ -734,14 +720,16 @@ management project. Refer to the [chart](https://hub.helm.sh/charts/jetstack/cert-manager) for the available configuration options. -NOTE: **Note:** -Support for installing the Cert Manager managed application is provided by the GitLab Configure group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). +Support for installing the Cert Manager managed application is provided by the +GitLab Configure group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). ### Install Sentry using GitLab CI/CD -NOTE: **Note:** -The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) at least 3GB of available RAM for database migrations. +The Sentry Helm chart [recommends](https://github.com/helm/charts/blob/f6e5784f265dd459c5a77430185d0302ed372665/stable/sentry/values.yaml#L284-L285) +at least 3 GB of available RAM for database migrations. To install Sentry, define the `.gitlab/managed-apps/config.yaml` file with: @@ -751,7 +739,7 @@ sentry: installed: true ``` -Sentry will then be installed into the `gitlab-managed-apps` namespace +Sentry is installed into the `gitlab-managed-apps` namespace of your cluster. You can customize the installation of Sentry by defining @@ -766,8 +754,10 @@ We recommend you pay close attention to the following configuration options: - `user`. Where you can set the login credentials for the default admin user. - `postgresql`. For a PostgreSQL password that can be used when running future updates. -NOTE: **Note:** -When upgrading it is important to provide the existing PostgreSQL password (given using the `postgresql.postgresqlPassword` key) or you will receive authentication errors. See the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) for more information. +When upgrading, it's important to provide the existing PostgreSQL password (given +using the `postgresql.postgresqlPassword` key) to avoid authentication errors. +Read the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) +for more information. Here is an example configuration for Sentry: @@ -799,9 +789,11 @@ postgresql: postgresqlPassword: example-postgresql-password ``` -NOTE: **Note:** -Support for installing the Sentry managed application is provided by the GitLab Health group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Health group](https://about.gitlab.com/handbook/product/product-categories/#health-group). +Support for installing the Sentry managed application is provided by the +GitLab Health group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Health group](https://about.gitlab.com/handbook/product/product-categories/#health-group). ### Install PostHog using GitLab CI/CD @@ -816,13 +808,14 @@ posthog: ``` You can customize the installation of PostHog by defining `.gitlab/managed-apps/posthog/values.yaml` -in your cluster management project. Refer to the [Configuration section of the PostHog chart's README](https://github.com/PostHog/charts/tree/master/charts/posthog) +in your cluster management project. Refer to the +[Configuration section of the PostHog chart's README](https://github.com/PostHog/charts/tree/master/charts/posthog) for the available configuration options. -NOTE: **Note:** You must provide a PostgreSQL password in `postgresql.postgresqlPassword` -or you will receive authentication errors. -See the [PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) for more information. +to avoid authentication errors. Read the +[PostgreSQL chart documentation](https://github.com/helm/charts/tree/master/stable/postgresql#upgrade) +for more information. Redis pods are restarted between upgrades. To prevent downtime, provide a Redis password using the `redis.password` key. This prevents a new password from @@ -848,9 +841,9 @@ redis: password: example-redis-password ``` -NOTE: **Note:** Support for the PostHog managed application is provided by the PostHog team. -If you run into issues, please [open a support ticket](https://github.com/PostHog/posthog/issues/new/choose) directly. +If you run into issues, +[open a support ticket](https://github.com/PostHog/posthog/issues/new/choose) directly. ### Install Prometheus using GitLab CI/CD @@ -874,9 +867,10 @@ project. Refer to the [Configuration section of the Prometheus chart's README](https://github.com/helm/charts/tree/master/stable/prometheus#configuration) for the available configuration options. -NOTE: **Note:** -Support for installing the Prometheus managed application is provided by the GitLab APM group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). +Support for installing the Prometheus managed application is provided by the +GitLab APM group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). ### Install GitLab Runner using GitLab CI/CD @@ -892,16 +886,17 @@ gitlabRunner: GitLab Runner is installed into the `gitlab-managed-apps` namespace of your cluster. -In order for GitLab Runner to function, you **must** specify the following: +For GitLab Runner to function, you _must_ specify the following: -- `gitlabUrl` - the GitLab server full URL (for example, `https://gitlab.example.com`) to register the Runner against. -- `runnerRegistrationToken` - The registration token for adding new runners to GitLab. This must be - [retrieved from your GitLab instance](../../ci/runners/README.md). +- `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) + to register the Runner against. +- `runnerRegistrationToken`: The registration token for adding new runners to GitLab. + This must be [retrieved from your GitLab instance](../../ci/runners/README.md). These values can be specified using [CI variables](../../ci/variables/README.md): -- `GITLAB_RUNNER_GITLAB_URL` will be used for `gitlabUrl`. -- `GITLAB_RUNNER_REGISTRATION_TOKEN` will be used for `runnerRegistrationToken` +- `GITLAB_RUNNER_GITLAB_URL` is used for `gitlabUrl`. +- `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` You can customize the installation of GitLab Runner by defining `.gitlab/managed-apps/gitlab-runner/values.yaml` file in your cluster @@ -909,9 +904,11 @@ management project. Refer to the [chart](https://gitlab.com/gitlab-org/charts/gitlab-runner) for the available configuration options. -NOTE: **Note:** -Support for installing the GitLab Runner managed application is provided by the GitLab Runner group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Runner group](https://about.gitlab.com/handbook/product/product-categories/#runner-group). +Support for installing the GitLab Runner managed application is provided by the +GitLab Runner group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Runner group](https://about.gitlab.com/handbook/product/product-categories/#runner-group). ### Install Cilium using GitLab CI/CD @@ -948,8 +945,10 @@ for the available configuration options. You can check Cilium's installation status on the cluster management page: -- [Project-level cluster](../project/clusters/index.md): Navigate to your project's **Operations > Kubernetes** page. -- [Group-level cluster](../group/clusters/index.md): Navigate to your group's **Kubernetes** page. +- [Project-level cluster](../project/clusters/index.md): Navigate to your project's + **Operations > Kubernetes** page. +- [Group-level cluster](../group/clusters/index.md): Navigate to your group's + **Kubernetes** page. CAUTION: **Caution:** Installation and removal of the Cilium requires a **manual** @@ -959,12 +958,11 @@ of all affected pods in all namespaces to ensure that they are by the correct networking plugin. NOTE: **Note:** -Major upgrades might require additional setup steps, please consult -the official [upgrade guide](https://docs.cilium.io/en/stable/install/upgrade/) for more -information. +Major upgrades might require additional setup steps. For more information, see +the official [upgrade guide](https://docs.cilium.io/en/stable/install/upgrade/). -By default, Cilium's [audit -mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/?highlight=policy-audit#enable-policy-audit-mode) +By default, Cilium's +[audit mode](https://docs.cilium.io/en/v1.8/gettingstarted/policy-creation/?highlight=policy-audit#enable-policy-audit-mode) is enabled. In audit mode, Cilium doesn't drop disallowed packets. You can use `policy-verdict` log to observe policy-related decisions. You can disable audit mode by adding the following to @@ -1006,7 +1004,7 @@ global: enabled: false ``` -You can also adjust Helm values for Hubble via +You can also adjust Helm values for Hubble by using `.gitlab/managed-apps/cilium/values.yaml`: ```yaml @@ -1018,9 +1016,11 @@ global: - 'flow:sourceContext=namespace;destinationContext=namespace' ``` -NOTE: **Note:** -Support for installing the Cilium managed application is provided by the GitLab Container Security group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). +Support for installing the Cilium managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). ### Install Falco using GitLab CI/CD @@ -1046,17 +1046,20 @@ management project. Refer to the for the available configuration options. CAUTION: **Caution:** -By default eBPF support is enabled and Falco will use an [eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) to pass system calls to userspace. -If your cluster doesn't support this, you can configure it to use Falco kernel module instead by adding the following to `.gitlab/managed-apps/falco/values.yaml`: +By default eBPF support is enabled and Falco uses an +[eBPF probe](https://falco.org/docs/event-sources/drivers/#using-the-ebpf-probe) +to pass system calls to user space. If your cluster doesn't support this, you can +configure it to use Falco kernel module instead by adding the following to +`.gitlab/managed-apps/falco/values.yaml`: ```yaml ebpf: enabled: false ``` -In rare cases where automatic probe installation on your cluster isn't possible and the kernel/probe -isn't precompiled, you may need to manually prepare the kernel module or eBPF probe with -[driverkit](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) +In rare cases where probe installation on your cluster isn't possible and the kernel/probe +isn't pre-compiled, you may need to manually prepare the kernel module or eBPF probe with +[`driverkit`](https://github.com/falcosecurity/driverkit#against-a-kubernetes-cluster) and install it on each cluster node. By default, Falco is deployed with a limited set of rules. To add more rules, add the following to @@ -1109,22 +1112,27 @@ You can check these logs with the following command: kubectl -n gitlab-managed-apps logs -l app=falco ``` -NOTE: **Note:** -Support for installing the Falco managed application is provided by the GitLab Container Security group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). +Support for installing the Falco managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). ### Install Vault using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9982) in GitLab 12.9. -[Hashicorp Vault](https://www.vaultproject.io/) is a secrets management solution which -can be used to safely manage and store passwords, credentials, certificates and more. A Vault +[HashiCorp Vault](https://www.vaultproject.io/) is a secrets management solution which +can be used to safely manage and store passwords, credentials, certificates, and more. A Vault installation could be leveraged to provide a single secure data store for credentials used in your applications, GitLab CI/CD jobs, and more. It could also serve as a way of providing SSL/TLS certificates to systems and deployments in your infrastructure. Leveraging Vault as a single source for all these credentials allows greater security by having a single source of access, control, and auditability around all your sensitive -credentials and certificates. +credentials and certificates. This feature requires giving GitLab the highest level of access and +control. Therefore, if GitLab is compromised, the security of this Vault instance is as well. To +avoid this security risk, GitLab recommends using your own HashiCorp Vault to leverage +[external secrets with CI](../../ci/secrets/index.md). To install Vault, enable it in the `.gitlab/managed-apps/config.yaml` file: @@ -1133,24 +1141,25 @@ vault: installed: true ``` -By default you will get a basic Vault setup with no scalable -storage backend. This is enough for simple testing and small-scale deployments, though has limits -to how much it can scale, and as it is a single instance deployment, you will experience downtime -when upgrading the Vault application. +By default you receive a basic Vault setup with no scalable storage backend. This +is enough for simple testing and small-scale deployments, though has limits +to how much it can scale, and as it's a single instance deployment, upgrading the +Vault application causes downtime. To optimally use Vault in a production environment, it's ideal to have a good understanding -of the internals of Vault and how to configure it. This can be done by reading the -[the Vault documentation](https://www.vaultproject.io/docs/internals) as well as +of the internals of Vault and how to configure it. This can be done by reading +the [Vault Configuration guide](../../ci/secrets/#configure-your-vault-server), +the [Vault documentation](https://www.vaultproject.io/docs/internals) and the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). -At a minimum you will likely set up: +At a minimum, most users set up: - A [seal](https://www.vaultproject.io/docs/configuration/seal) for extra encryption - of the master key. -- A [storage backend](https://www.vaultproject.io/docs/configuration/storage) that is + of the main key. +- A [storage backend](https://www.vaultproject.io/docs/configuration/storage) that's suitable for environment and storage security requirements. - [HA Mode](https://www.vaultproject.io/docs/concepts/ha). -- [The Vault UI](https://www.vaultproject.io/docs/configuration/ui). +- The [Vault UI](https://www.vaultproject.io/docs/configuration/ui). The following is an example values file (`.gitlab/managed-apps/vault/values.yaml`) that configures Google Key Management Service for auto-unseal, using a Google Cloud Storage backend, enabling @@ -1180,7 +1189,7 @@ server: path = "gcs://my-vault-storage/vault-bucket" ha_enabled = "true" } - # Configure Vault to automatically unseal storage using a GKMS key + # Configure Vault to unseal storage using a GKMS key seal "gcpckms" { project = "vault-helm-dev-246514" region = "global" @@ -1189,10 +1198,13 @@ server: } ``` -Once you have successfully installed Vault, you will need to [initialize the Vault](https://learn.hashicorp.com/tutorials/vault/getting-started-deploy#initializing-the-vault) -and obtain the initial root token. You will need access to your Kubernetes cluster that Vault has been deployed into in order to do this. -To initialize the Vault, get a shell to one of the Vault pods running inside Kubernetes (typically this is done by using the `kubectl` command line tool). -Once you have a shell into the pod, run the `vault operator init` command: +Once you have successfully installed Vault, you must +[initialize the Vault](https://learn.hashicorp.com/tutorials/vault/getting-started-deploy#initializing-the-vault) +and obtain the initial root token. You need access to your Kubernetes cluster that +Vault has been deployed into in order to do this. To initialize the Vault, get a +shell to one of the Vault pods running inside Kubernetes (typically this is done +by using the `kubectl` command line tool). Once you have a shell into the pod, +run the `vault operator init` command: ```shell kubectl -n gitlab-managed-apps exec -it vault-0 sh @@ -1200,11 +1212,13 @@ kubectl -n gitlab-managed-apps exec -it vault-0 sh ``` This should give you your unseal keys and initial root token. Make sure to note these down -and keep these safe as you will need them to unseal the Vault throughout its lifecycle. +and keep these safe, as they're required to unseal the Vault throughout its lifecycle. -NOTE: **Note:** -Support for installing the Vault managed application is provided by the GitLab Release Management group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Release Management group](https://about.gitlab.com/handbook/product/product-categories/#release-management-group). +Support for installing the Vault managed application is provided by the +GitLab Release Management group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Release Management group](https://about.gitlab.com/handbook/product/product-categories/#release-management-group). ### Install JupyterHub using GitLab CI/CD @@ -1224,7 +1238,7 @@ In the configuration: - `gitlabProjectIdWhitelist` restricts GitLab authentication to only members of the specified projects. - `gitlabGroupWhitelist` restricts GitLab authentication to only members of the specified groups. -- Specifying an empty array for both will allow any user on the GitLab instance to sign in. +- Specifying an empty array for both allows any user on the GitLab instance to sign in. JupyterHub is installed into the `gitlab-managed-apps` namespace of your cluster. @@ -1236,17 +1250,19 @@ Set: In addition, the following variables must be specified using [CI variables](../../ci/variables/README.md): -| CI Variable | Description | -|:---------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `JUPYTERHUB_PROXY_SECRET_TOKEN` | Secure string used for signing communications from the hub. See[`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#proxy-secrettoken). | -| `JUPYTERHUB_COOKIE_SECRET` | Secure string used for signing secure cookies. See [`hub.cookieSecret`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#hub-cookiesecret). | -| `JUPYTERHUB_HOST` | Hostname used for the installation. For example, `jupyter.gitlab.example.com`. | -| `JUPYTERHUB_GITLAB_HOST` | Hostname of the GitLab instance used for authentication. For example, `gitlab.example.com`. | -| `JUPYTERHUB_AUTH_CRYPTO_KEY` | A 32-byte encryption key used to set [`auth.state.cryptoKey`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#auth-state-cryptokey). | -| `JUPYTERHUB_AUTH_GITLAB_CLIENT_ID` | "Application ID" for the OAuth Application. | -| `JUPYTERHUB_AUTH_GITLAB_CLIENT_SECRET` | "Secret" for the OAuth Application. | - -By default, JupyterHub will be installed using a +- `JUPYTERHUB_PROXY_SECRET_TOKEN` - Secure string used for signing communications + from the hub. Read [`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#proxy-secrettoken). +- `JUPYTERHUB_COOKIE_SECRET` - Secure string used for signing secure cookies. Read + [`hub.cookieSecret`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#hub-cookiesecret). +- `JUPYTERHUB_HOST` - Hostname used for the installation. For example, `jupyter.gitlab.example.com`. +- `JUPYTERHUB_GITLAB_HOST` - Hostname of the GitLab instance used for authentication. + For example, `gitlab.example.com`. +- `JUPYTERHUB_AUTH_CRYPTO_KEY` - A 32-byte encryption key used to set + [`auth.state.cryptoKey`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#auth-state-cryptokey). +- `JUPYTERHUB_AUTH_GITLAB_CLIENT_ID` - "Application ID" for the OAuth Application. +- `JUPYTERHUB_AUTH_GITLAB_CLIENT_SECRET` - "Secret" for the OAuth Application. + +By default, JupyterHub is installed using a [default values file](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/blob/master/src/default-data/jupyterhub/values.yaml.gotmpl). You can customize the installation of JupyterHub by defining a `.gitlab/managed-apps/jupyterhub/values.yaml` file in your cluster management project. @@ -1255,9 +1271,11 @@ Refer to the [chart reference](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html) for the available configuration options. -NOTE: **Note:** Support for installing the JupyterHub managed application is provided by the GitLab Configure group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). +If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). ### Install Elastic Stack using GitLab CI/CD @@ -1275,20 +1293,25 @@ elasticStack: Elastic Stack is installed into the `gitlab-managed-apps` namespace of your cluster. -You can check the default [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) we set for this chart. +You can check the default +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) +we set for this chart. You can customize the installation of Elastic Stack by defining `.gitlab/managed-apps/elastic-stack/values.yaml` file in your cluster management project. Refer to the -[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for the +[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for all available configuration options. NOTE: **Note:** -In this alpha implementation of installing Elastic Stack through CI, reading the environment logs through Elasticsearch is unsupported. This is supported if [installed via the UI](#elastic-stack). +In this alpha implementation of installing Elastic Stack through CI, reading the +environment logs through Elasticsearch is unsupported. This is supported if +[installed with the UI](#elastic-stack). -NOTE: **Note:** -Support for installing the Elastic Stack managed application is provided by the GitLab APM group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). +Support for installing the Elastic Stack managed application is provided by the +GitLab APM group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the [APM group](https://about.gitlab.com/handbook/product/product-categories/#apm-group). ### Install Crossplane using GitLab CI/CD @@ -1316,37 +1339,40 @@ management project. Refer to the [chart](https://github.com/crossplane/crossplane/tree/master/cluster/charts/crossplane#configuration) for the available configuration options. Note that this link points to the documentation for the current development release, which may differ from the version you have installed. -NOTE: **Note:** Support for the Crossplane managed application is provided by the Crossplane team. -If you run into issues, please [open a support ticket](https://github.com/crossplane/crossplane/issues/new/choose) directly. +If you run into issues, +[open a support ticket](https://github.com/crossplane/crossplane/issues/new/choose) directly. ### Install Fluentd using GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/76) in GitLab 12.10. -To install Fluentd into the `gitlab-managed-apps` namespace of your cluster using GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: +To install Fluentd into the `gitlab-managed-apps` namespace of your cluster using +GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: ```yaml Fluentd: installed: true ``` -You can also review the default values set for this chart in the [`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. +You can also review the default values set for this chart in the +[`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. You can customize the installation of Fluentd by defining `.gitlab/managed-apps/fluentd/values.yaml` file in your cluster management project. Refer to the [configuration chart for the current development release of Fluentd](https://github.com/helm/charts/tree/master/stable/fluentd#configuration) -for the available configuration options. +for all available configuration options. -NOTE: **Note:** The configuration chart link points to the current development release, which may differ from the version you have installed. To ensure compatibility, switch to the specific branch or tag you are using. -NOTE: **Note:** -Support for installing the Fluentd managed application is provided by the GitLab Container Security group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). +Support for installing the Fluentd managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). ### Install Knative using GitLab CI/CD @@ -1360,7 +1386,7 @@ knative: You can customize the installation of Knative by defining `.gitlab/managed-apps/knative/values.yaml` file in your cluster management project. Refer to the [chart](https://gitlab.com/gitlab-org/charts/knative) -for the available configuration options. +for all available configuration options. Here is an example configuration for Knative: @@ -1368,11 +1394,14 @@ Here is an example configuration for Knative: domain: 'my.wildcard.A.record.dns' ``` -If you plan to use GitLab Serverless capabilities, be sure to set an A record wildcard domain on your custom configuration. +If you plan to use GitLab Serverless capabilities, be sure to set an `A record` +wildcard domain on your custom configuration. -NOTE: **Note:** -Support for installing the Knative managed application is provided by the GitLab Configure group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). +Support for installing the Knative managed application is provided by the +GitLab Configure group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping at +least 2 people from the +[Configure group](https://about.gitlab.com/handbook/product/product-categories/#configure-group). #### Knative Metrics @@ -1398,14 +1427,16 @@ kubectl delete -f https://gitlab.com/gitlab-org/cluster-integration/cluster-appl > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/100) in GitLab 13.1. -To install AppArmor into the `gitlab-managed-apps` namespace of your cluster using GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: +To install AppArmor into the `gitlab-managed-apps` namespace of your cluster using +GitLab CI/CD, define the following configuration in `.gitlab/managed-apps/config.yaml`: ```yaml apparmor: installed: true ``` -You can define one or more AppArmor profiles by adding them into `.gitlab/managed-apps/apparmor/values.yaml` as the following: +You can define one or more AppArmor profiles by adding them into +`.gitlab/managed-apps/apparmor/values.yaml` as the following: ```yaml profiles: @@ -1419,25 +1450,27 @@ Refer to the [AppArmor chart](https://gitlab.com/gitlab-org/charts/apparmor) for #### Using AppArmor profiles in your deployments -After installing AppAmor, you can use profiles by adding Pod Annotations. If you're using Auto -DevOps, you can [customize `auto-deploy-values.yaml`](../../topics/autodevops/customize.md#customize-values-for-helm-chart) -to annotate your pods. Although it's helpful to be aware of the [list of custom attributes](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app#gitlabs-auto-deploy-helm-chart), you're only required to set -`podAnnotations` as follows: +After installing AppAmor, you can use profiles by adding Pod Annotations. If you're using +Auto DevOps, you can [customize `auto-deploy-values.yaml`](../../topics/autodevops/customize.md#customize-values-for-helm-chart) +to annotate your pods. Although it's helpful to be aware of the +[list of custom attributes](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app#gitlabs-auto-deploy-helm-chart), +you're only required to set `podAnnotations` as follows: ```yaml podAnnotations: container.apparmor.security.beta.kubernetes.io/auto-deploy-app: localhost/profile-one ``` -The only information to be changed here is the profile name which is `profile-one` in this example. Refer to the [AppArmor tutorial](https://kubernetes.io/docs/tutorials/clusters/apparmor/#securing-a-pod) for more information on how AppArmor is integrated in Kubernetes. +The only information to be changed here is the profile name which is `profile-one` +in this example. Refer to the [AppArmor tutorial](https://kubernetes.io/docs/tutorials/clusters/apparmor/#securing-a-pod) +for more information on how AppArmor is integrated in Kubernetes. #### Using PodSecurityPolicy in your deployments -NOTE: **Note:** To enable AppArmor annotations on a Pod Security Policy you must first -load the correspondingAppArmor profile. +load the corresponding AppArmor profile. -[Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/)are +[Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) are resources at the cluster level that control security-related properties of deployed pods. You can use such a policy to enable loaded AppArmor profiles and apply necessary pod restrictions across a @@ -1465,14 +1498,14 @@ securityPolicies: - '*' ``` -This example creates a single policy named `example` with the provided -specification, and enables [AppArmor -annotations](https://kubernetes.io/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations)on -it. +This example creates a single policy named `example` with the provided specification, +and enables [AppArmor annotations](https://kubernetes.io/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations) on it. -NOTE: **Note:** -Support for installing the AppArmor managed application is provided by the GitLab Container Security group. -If you run into unknown issues, please [open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new) and ping at least 2 people from the [Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). +Support for installing the AppArmor managed application is provided by the +GitLab Container Security group. If you run into unknown issues, +[open a new issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new), and ping +at least 2 people from the +[Container Security group](https://about.gitlab.com/handbook/product/product-categories/#container-security-group). ## Browse applications logs @@ -1500,9 +1533,7 @@ To upgrade an application: 1. Select your cluster. 1. If an upgrade is available, the **Upgrade** button is displayed. Click the button to upgrade. -NOTE: **Note:** -Upgrades will reset values back to the values built into the `runner` -chart plus the values set by +Upgrades reset values back to the values built into the `runner` chart, plus the values set by [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/runner/values.yaml) ## Uninstalling applications @@ -1513,16 +1544,16 @@ The applications below can be uninstalled. | Application | GitLab version | Notes | | ----------- | -------------- | ----- | -| cert-manager | 12.2+ | The associated private key will be deleted and cannot be restored. Deployed applications will continue to use HTTPS, but certificates will not be renewed. Before uninstalling, you may wish to [back up your configuration](https://cert-manager.io/docs/tutorials/backup/) or [revoke your certificates](https://letsencrypt.org/docs/revoking/). | -| GitLab Runner | 12.2+ | Any running pipelines will be canceled. | -| Helm | 12.2+ | The associated Tiller pod, the `gitlab-managed-apps` namespace, and all of its resources will be deleted and cannot be restored. | -| Ingress | 12.1+ | The associated load balancer and IP will be deleted and cannot be restored. Furthermore, it can only be uninstalled if JupyterHub is not installed. | -| JupyterHub | 12.1+ | All data not committed to GitLab will be deleted and cannot be restored. | -| Knative | 12.1+ | The associated IP will be deleted and cannot be restored. | -| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | -| Crossplane | 12.5+ | All data will be deleted and cannot be restored. | -| Elastic Stack | 12.7+ | All data will be deleted and cannot be restored. | -| Sentry | 12.6+ | The PostgreSQL persistent volume will remain and should be manually removed for complete uninstall. | +| cert-manager | 12.2+ | The associated private key is deleted and cannot be restored. Deployed applications continue to use HTTPS, but certificates aren't renewed. Before uninstalling, you may want to [back up your configuration](https://cert-manager.io/docs/tutorials/backup/) or [revoke your certificates](https://letsencrypt.org/docs/revoking/). | +| GitLab Runner | 12.2+ | Any running pipelines are canceled. | +| Helm | 12.2+ | The associated Tiller pod, the `gitlab-managed-apps` namespace, and all of its resources are deleted and cannot be restored. | +| Ingress | 12.1+ | The associated load balancer and IP are deleted and cannot be restored. Furthermore, it can only be uninstalled if JupyterHub is not installed. | +| JupyterHub | 12.1+ | All data not committed to GitLab are deleted and cannot be restored. | +| Knative | 12.1+ | The associated IP are deleted and cannot be restored. | +| Prometheus | 11.11+ | All data are deleted and cannot be restored. | +| Crossplane | 12.5+ | All data are deleted and cannot be restored. | +| Elastic Stack | 12.7+ | All data are deleted and cannot be restored. | +| Sentry | 12.6+ | The PostgreSQL persistent volume remains and should be manually removed for complete uninstall. | To uninstall an application: @@ -1535,8 +1566,7 @@ To uninstall an application: 1. Click the **Uninstall** button for the application. Support for uninstalling all applications is planned for progressive rollout. -To follow progress, see [the relevant -epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). +To follow progress, see the [relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). ## Troubleshooting applications @@ -1550,9 +1580,10 @@ To avoid installation errors: - Before starting the installation of applications, make sure that time is synchronized between your GitLab server and your Kubernetes cluster. -- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. +- Ensure certificates are not out of sync. When installing applications, GitLab + expects a new cluster with no previous installation of Helm. - You can confirm that the certificates match via `kubectl`: + You can confirm that the certificates match by using `kubectl`: ```shell kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ @@ -1590,5 +1621,5 @@ Installing Prometheus is failing with the following error: Error: Could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: admission.certmanager.k8s.io/v1beta1: the server is currently unable to handle the request ``` -This is a bug that was introduced in Helm `2.15` and fixed in `3.0.2`. As a workaround, you'll need -to make sure that [`cert-manager`](#cert-manager) is installed successfully prior to installing Prometheus. +This is a bug that was introduced in Helm `2.15` and fixed in `3.0.2`. As a workaround, +ensure [`cert-manager`](#cert-manager) is installed successfully prior to installing Prometheus. diff --git a/doc/user/clusters/cost_management.md b/doc/user/clusters/cost_management.md new file mode 100644 index 00000000000..f13be15c6bc --- /dev/null +++ b/doc/user/clusters/cost_management.md @@ -0,0 +1,74 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Cluster cost management **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216737) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. + +Cluster cost management provides insights into cluster resource usage. GitLab provides an example +[`kubecost-cost-model`](https://gitlab.com/gitlab-examples/kubecost-cost-model/) +project that uses GitLab's Prometheus integration and +[Kubecost's `cost-model`](https://github.com/kubecost/cost-model) to provide cluster cost +insights within GitLab: + + + +## Configure cluster cost management + +To get started with cluster cost management, you need [Maintainer](../permissions.md) +permissions in a project or group. + +1. Clone the [`kubecost-cost-model`](https://gitlab.com/gitlab-examples/kubecost-cost-model/) + example repository, which contains minor modifications to the upstream Kubecost + `cost-model` project: + - Configures your Prometheus endpoint to the GitLab-managed Prometheus. You may + need to change this value if you use a non-managed Prometheus. + - Adds the necessary annotations to the deployment manifest to be scraped by + GitLab-managed Prometheus. + - Changes the Google Pricing API access key to GitLab's access key. + - Contains definitions for a custom GitLab Metrics dashboard to show the cost insights. +1. Connect GitLab with Prometheus, depending on your configuration: + - *If Prometheus is already configured,* navigate to **Settings > Integrations > Prometheus** + to provide the API endpoint of your Prometheus server. + - *For GitLab-managed Prometheus,* navigate to your cluster's **Details** page, + select the **Applications** tab, and install Prometheus. The integration is + auto-configured for you. +1. Set up the Prometheus integration on the cloned example project. +1. Add the Kubecost `cost-model` to your cluster: + - *For non-managed clusters*, deploy it with GitLab CI/CD. + - *To deploy it manually*, use the following commands: + + ```shell + kubectl create namespace cost-model + kubectl apply -f kubernetes/ --namespace cost-model + ``` + +To access the cost insights, navigate to **Operations > Metrics** and select +the `default_costs.yml` dashboard. You can [customize](#customize-the-cost-dashboard) +this dashboard. + +### Customize the cost dashboard + +You can customize the cost dashboard by editing the `.gitlab/dashboards/default_costs.yml` +file or creating similar dashboard configuration files. To learn more, read about +[customizing dashboards in our documentation](/ee/operations/metrics/dashboards/). + +#### Available metrics + +Metrics contain both instance and node labels. The instance label will be deprecated in a future version. + +- `node_cpu_hourly_cost` - Hourly cost per vCPU on this node. +- `node_gpu_hourly_cost` - Hourly cost per GPU on this node. +- `node_ram_hourly_cost` - Hourly cost per gigabyte of memory on this node. +- `node_total_hourly_cost` - Total node cost per hour. +- `container_cpu_allocation` - Average number of CPUs requested/used over the previous minute. +- `container_gpu_allocation` - Average number of GPUs requested over the previous minute. +- `container_memory_allocation_bytes` - Average bytes of RAM requested/used over the previous minute. +- `pod_pvc_allocation` - Bytes provisioned for a PVC attached to a pod. +- `pv_hourly_cost` - Hourly cost per GB on a persistent volume. + +Some examples are provided in the +[`kubecost-cost-model` repository](https://gitlab.com/gitlab-examples/kubecost-cost-model/-/blob/master/PROMETHEUS.md#example-queries). diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index b30ebc57338..8463917f2f3 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -78,7 +78,6 @@ provided can manage resources in the `database.crossplane.io` API group: See [Configure Your Cloud Provider Account](https://crossplane.github.io/docs/v0.4/cloud-providers.html) to configure the installed cloud provider stack with a user account. -NOTE: **Note:** The Secret, and the Provider resource referencing the Secret, must be applied to the `gitlab-managed-apps` namespace in the guide. Make sure you change that while following the process. diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index 2b342ceb06d..3ab20c5466e 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -43,6 +43,5 @@ Once you have successful deployments to your group-level or instance-level clust 1. Navigate to your group's **Kubernetes** page. 1. Click on the **Environments** tab. -NOTE: **Note:** -Only successful deployments to the cluster is included in this page. -Non-cluster environments will not be included. +Only successful deployments to the cluster are included in this page. +Non-cluster environments aren't included. diff --git a/doc/user/clusters/img/kubecost_v13_5.png b/doc/user/clusters/img/kubecost_v13_5.png Binary files differnew file mode 100644 index 00000000000..a93e2531b16 --- /dev/null +++ b/doc/user/clusters/img/kubecost_v13_5.png diff --git a/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_3_1.png b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_3_1.png Binary files differindex a06f8812b41..89f4e917567 100644 --- a/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_3_1.png +++ b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v13_3_1.png diff --git a/doc/user/compliance/license_compliance/img/license-check_v13_4.png b/doc/user/compliance/license_compliance/img/license-check_v13_4.png Binary files differindex d3658cbaa18..bc80f938395 100644 --- a/doc/user/compliance/license_compliance/img/license-check_v13_4.png +++ b/doc/user/compliance/license_compliance/img/license-check_v13_4.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 79c2d97b972..894c0e14862 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -121,12 +121,17 @@ The results will be saved as a [License Compliance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportslicense_scanning) that you can later download and analyze. Due to implementation limitations, we always take the latest License Compliance artifact available. Behind the scenes, the -[GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) +[GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/license-finder) is used to detect the languages/frameworks and in turn analyzes the licenses. The License Compliance settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. +### When License Compliance runs + +When using the GitLab `License-Scanning.gitlab-ci.yml` template, the License Compliance job doesn't +wait for other stages to complete. + ### Available variables License Compliance can be configured using environment variables. @@ -446,7 +451,7 @@ package manager. For a comprehensive list, consult [the Conan documentation](htt The default [Conan](https://conan.io/) configuration sets [`CONAN_LOGIN_USERNAME`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name) to `ci_user`, and binds [`CONAN_PASSWORD`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-password-conan-password-remote-name) to the [`CI_JOB_TOKEN`](../../../ci/variables/predefined_variables.md) -for the running job. This allows Conan projects to fetch packages from a [GitLab Conan Repository](../../packages/conan_repository/#fetching-conan-package-information-from-the-gitlab-package-registry) +for the running job. This allows Conan projects to fetch packages from a [GitLab Conan Repository](../../packages/conan_repository/#fetch-conan-package-information-from-the-package-registry) if a GitLab remote is specified in the `.conan/remotes.json` file. To override the default credentials specify a [`CONAN_LOGIN_USERNAME_{REMOTE_NAME}`](https://docs.conan.io/en/latest/reference/env_vars.html#conan-login-username-conan-login-username-remote-name) @@ -629,7 +634,7 @@ import the following default License Compliance analyzer images from `registry.g offline [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/gitlab-org/security-products/license-management:latest +registry.gitlab.com/gitlab-org/security-products/analyzers/license-finder:latest ``` The process for importing Docker images into a local offline Docker registry depends on diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 67dd31efe2c..ec0c207e190 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -38,6 +38,14 @@ The IP address for `mg.gitlab.com` is subject to change at any time. [See our backup strategy](https://about.gitlab.com/handbook/engineering/infrastructure/production/#backups). +There are several ways to perform backups of your content on GitLab.com. + +Projects can be backed up in their entirety by exporting them either [through the UI](../project/settings/import_export.md) or [API](../../api/project_import_export.md#schedule-an-export), the latter of which can be used to programmatically upload exports to a storage platform such as AWS S3. + +With exports, be sure to take note of [what is and is not](../project/settings/import_export.md#exported-contents), included in a project export. + +Since GitLab is built on Git, you can back up **just** the repository of a project by [cloning](../../gitlab-basics/start-using-git.md#clone-a-repository) it to another machine. Similarly, if you need to back up just the wiki of a repository it can also be cloned and all files uploaded to that wiki will come with it [if they were uploaded after 2020-08-22](../project/wiki/index.md#creating-a-new-wiki-page). + ## Alternative SSH port GitLab.com can be reached via a [different SSH port](https://about.gitlab.com/blog/2016/02/18/gitlab-dot-com-now-supports-an-alternate-git-plus-ssh-port/) for `git+ssh`. @@ -88,6 +96,7 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/README.md). | [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited | | [Max number of instance level variables](../../administration/instance_limits.md#number-of-instance-level-variables) | `25` | `25` | | [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs) | 3 months | Never | +| Max test cases per [unit test report](../../ci/unit_test_reports.md) | `500_000` | Unlimited | ## Account and limit settings @@ -474,6 +483,10 @@ More information on this particular change can be found at of proposed changes can be found at <https://gitlab.com/gitlab-com/infrastructure/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=database&label_name[]=change>. +## Puma + +GitLab.com uses the default of 60 seconds for [Puma request timeouts](https://docs.gitlab.com/omnibus/settings/puma.html#worker-timeout). + ## Unicorn GitLab.com adjusts the memory limits for the [unicorn-worker-killer](https://rubygems.org/gems/unicorn-worker-killer) gem. @@ -529,9 +542,9 @@ Source: For performance reasons, if a query returns more than 10,000 records, GitLab doesn't return the following headers: -- `X-Total`. -- `X-Total-Pages`. -- `rel="last"` `Link`. +- `x-total`. +- `x-total-pages`. +- `rel="last"` `link`. ### Rack Attack initializer @@ -596,6 +609,11 @@ dropped and users get To help avoid abuse, project and group imports, exports, and export downloads are rate limited. See [Project import/export rate limits](../../user/project/settings/import_export.md#rate-limits) and [Group import/export rate limits](../../user/group/settings/import_export.md#rate-limits) for details. +### Non-configurable limits + +See [non-configurable limits](../../security/rate_limits.md#non-configurable-limits) for information on +rate limits that are not configurable, and therefore also used on GitLab.com. + ## GitLab.com Logging We use [Fluentd](https://gitlab.com/gitlab-com/runbooks/tree/master/logging/doc#fluentd) to parse our logs. Fluentd sends our logs to diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index ebf38aef4a6..1a62d67e468 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -14,6 +14,9 @@ Similar to [project-level](../../project/clusters/index.md) and group-level Kubernetes clusters allow you to connect a Kubernetes cluster to your group, enabling you to use the same cluster across multiple projects. +To view your group level Kubernetes clusters, navigate to your project and select +**Kubernetes** from the left-hand menu. + ## Installing applications GitLab can install and manage some applications in your group-level @@ -69,9 +72,8 @@ for deployments with a cluster not managed by GitLab, you must ensure: (this is [not automatic](https://gitlab.com/gitlab-org/gitlab/-/issues/31519)). Editing `KUBE_NAMESPACE` directly is discouraged. -NOTE: **Note:** If you [install applications](#installing-applications) on your cluster, GitLab creates -the resources required to run them even if you choose to manage your own cluster. +the resources required to run them, even if you choose to manage your own cluster. ### Clearing the cluster cache diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index bcc6d958427..a689b7c380b 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -9,8 +9,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3090) for subgroups in GitLab 12.2. -## Overview - With Contribution Analytics you can get an overview of the following activity in your group: diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index e8bcb7219fc..f380b36cc00 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -10,9 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.2. > - Single-level Epics [were moved](https://gitlab.com/gitlab-org/gitlab/-/issues/37081) to [GitLab Premium](https://about.gitlab.com/pricing/) in 12.8. -Epics let you manage your portfolio of projects more efficiently and with less -effort by tracking groups of issues that share a theme, across projects and -milestones. +Epics let you manage your portfolio of projects more efficiently by tracking groups of issues that +share a theme across projects and milestones. An epic's page contains the following tabs: @@ -22,7 +21,7 @@ An epic's page contains the following tabs: - Hover over the total counts to see a breakdown of open and closed items. NOTE: **Note:** - The number provided here includes all epics associated with this project. The number includes epics for which users may not currently have permission. + The number provided here includes all epics associated with this project. The number includes epics for which users may not yet have permission. - **Roadmap**: a roadmap view of child epics which have start and due dates. @@ -65,16 +64,19 @@ graph TD ``` See [Manage issues assigned to an epic](manage_epics.md#manage-issues-assigned-to-an-epic) for steps -to add an issue to an epic, reorder issues, move issues between epics, or promote an issue to an epic. +to: + +- Add an issue to an epic +- Reorder issues +- Move an issue between epics +- Promote an issue to an epic ## Issue health status in Epic tree **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/199184) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. -> - The health status of a closed issue [will be hidden](https://gitlab.com/gitlab-org/gitlab/-/issues/220867) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.3 or later. +> - The health status of a closed issue [is hidden](https://gitlab.com/gitlab-org/gitlab/-/issues/220867) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.3 or later. -You can report on and quickly respond to the health of individual issues and epics by setting a -red, amber, or green [health status on an issue](../../project/issues/index.md#health-status), -which will appear on your Epic tree. +Report or respond to the health of issues and epics by setting a red, amber, or green [health status](../../project/issues/index.md#health-status), which then appears on your Epic tree. ### Disable Issue health status in Epic tree @@ -100,7 +102,7 @@ steps to create, move, reorder, or delete child epics. To set a **Start date** and **Due date** for an epic, select one of the following: - **Fixed**: Enter a fixed value. -- **From milestones**: Inherit a dynamic value from the milestones currently assigned to the epic's issues. +- **From milestones**: Inherit a dynamic value from the milestones that are assigned to the epic's issues. Note that GitLab 12.5 replaced this option with **Inherited**. - **Inherited**: Inherit a dynamic value from the epic's issues, child epics, and milestones ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7332) in GitLab 12.5 to replace **From milestones**). @@ -108,10 +110,10 @@ To set a **Start date** and **Due date** for an epic, select one of the followin > [Replaced](https://gitlab.com/gitlab-org/gitlab/-/issues/7332) in GitLab 12.5 by **Inherited**. -If you select **From milestones** for the start date, GitLab will automatically set the date to be earliest -start date across all milestones that are currently assigned to the issues that are added to the epic. -Similarly, if you select **From milestones** for the due date, GitLab will set it to be the latest due date across -all milestones that are currently assigned to those issues. +If you select **From milestones** for the start date, GitLab automatically sets the date to be earliest +start date across all milestones that are assigned to the issues that belong to the epic. +If you select **From milestones** for the due date, GitLab sets the date to be the latest due date across +all milestones that are assigned to those issues. These are dynamic dates which are recalculated if any of the following occur: @@ -138,8 +140,8 @@ These are dynamic dates and recalculated if any of the following occur: - Issues are added to, or removed from, the epic. Because the epic's dates can inherit dates from its children, the start date and due date propagate from the bottom to the top. -If the start date of a child epic on the lowest level changes, that becomes the earliest possible start date for its parent epic, -then the parent epic's start date will reflect the change and this will propagate upwards to the top epic. +If the start date of a child epic on the lowest level changes, that becomes the earliest possible start date for its parent epic. +The parent epic's start date then reflects this change and propagates upwards to the top epic. ## Roadmap in epics @@ -153,11 +155,11 @@ have a [start or due date](#start-date-and-due-date), a ## Permissions -If you have access to view an epic and have access to view an issue already -added to that epic, then you can view the issue in the epic issue list. +If you have access to view an epic and an issue added to that epic, you can view the issue in the +epic's issue list. -If you have access to edit an epic and have access to edit an issue, then you -can add the issue to or remove it from the epic. +If you have access to edit an epic and an issue added to that epic, you can add the issue to or +remove it from the epic. Note that for a given group, the visibility of all projects must be the same as the group, or less restrictive. That means if you have access to a group's epic, @@ -175,8 +177,8 @@ You can also consult the [group permissions table](../../permissions.md#group-me Once you write your comment, you can either: -- Click **Comment**, and your comment will be published. -- Click **Start thread**, and you will start a thread within that epic's discussion. +- Click **Comment** to publish your comment. +- Click **Start thread** to start a thread within that epic's discussion. ### Activity sort order diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 32b76cf9280..2c838724cb3 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -371,7 +371,7 @@ To create group links via filter: ### Overriding user permissions **(STARTER ONLY)** -Since GitLab [v8.15](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/822) LDAP user permissions can now be manually overridden by an admin user. To override a user's permissions: +In GitLab [8.15](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/822) and later, LDAP user permissions can now be manually overridden by an admin user. To override a user's permissions: 1. Go to your group's **Members** page. 1. Select the pencil icon in the row for the user you are editing. @@ -391,11 +391,56 @@ milestones. [Learn more about Epics.](epics/index.md) +## Group wikis **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13195) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5. +> - It's [deployed behind a feature flag](../feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-group-wikis). + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +Group wikis work the same way as [project wikis](../project/wiki/index.md), please refer to those docs for details on usage. + +Group wikis can be edited by members with [Developer permissions](../../user/permissions.md#group-members-permissions) +and above. + +### Group wikis limitations + +There are a few limitations compared to project wikis: + +- Local Git access is not supported yet. +- Group wikis are not included in global search, group exports, backups, and Geo replication. +- Changes to group wikis don't show up in the group's activity feed. + +You can follow [this epic](https://gitlab.com/groups/gitlab-org/-/epics/2782) for updates. + +### Enable or disable group wikis **(CORE ONLY)** + +Group wikis are under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) +can opt to disable it for your instance. + +To enable it: + +```ruby +Feature.enable(:group_wikis) +``` + +To disable it: + +```ruby +Feature.disable(:group_wikis) +``` + ## Group Security Dashboard **(ULTIMATE)** Get an overview of the vulnerabilities of all the projects in a group and its subgroups. -[Learn more about the Group Security Dashboard.](security_dashboard/index.md) +[Learn more about the Group Security Dashboard.](../application_security/security_dashboard/index.md) ## Insights **(ULTIMATE)** diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index 20cbc043d83..2eb50f07de3 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -64,6 +64,15 @@ To edit an iteration, click the three-dot menu (**{ellipsis_v}**) > **Edit itera To learn how to add an issue to an iteration, see the steps in [Managing issues](../../project/issues/managing_issues.md#add-an-issue-to-an-iteration). +## View an iteration report + +> Viewing iteration reports in projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222763) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.5. + +You can track the progress of an iteration by reviewing iteration reports. +An iteration report displays a list of all the issues assigned to an iteration and their status. + +To view an iteration report, go to the iterations list page and click an iteration's title. + ## Disable Iterations **(CORE ONLY)** GitLab Iterations feature is deployed with a feature flag that is **enabled by default**. diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index b013e371ed2..c9a10146440 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -1,17 +1,13 @@ --- type: reference stage: Verify -group: Analytics +group: Testing info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Repositories Analytics **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215104) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. -> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-repositories-analytics). **(CORE ONLY)** +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215104) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. CAUTION: **Warning:** This feature might not be available to you. Check the **version history** note above for details. @@ -35,25 +31,6 @@ For each day that a coverage report was generated by a job in a project's pipeli If the project's code coverage was calculated more than once in a day, we will take the last value from that day. -## Enable or disable Repositories Analytics **(CORE ONLY)** - -Repositories Analytics is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -Feature.enable(:group_coverage_reports) -``` - -To disable it: - -```ruby -Feature.disable(:group_coverage_reports) -``` - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 57b9cc92c51..3cb566c7f77 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -5,7 +5,7 @@ group: Access info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# SAML SSO for GitLab.com groups **(PREMIUM)** +# SAML SSO for GitLab.com groups **(SILVER ONLY)** > Introduced in GitLab 11.0. @@ -256,53 +256,6 @@ For example, to unlink the `MyOrg` account, the following **Disconnect** button | Issuer | How GitLab identifies itself to the identity provider. Also known as a "Relying party trust identifier". | | Certificate fingerprint | Used to confirm that communications over SAML are secure by checking that the server is signing communications with the correct certificate. Also known as a certificate thumbprint. | -## Configuring on a self-managed GitLab instance **(PREMIUM ONLY)** - -For self-managed GitLab instances we strongly recommend using the -[instance-wide SAML OmniAuth Provider](../../../integration/saml.md) instead. - -Group SAML SSO helps if you need to allow access via multiple SAML identity providers, but as a multi-tenant solution is less suited to cases where you administer your own GitLab instance. - -To proceed with configuring Group SAML SSO instead, you'll need to enable the `group_saml` OmniAuth provider. This can be done from: - -- `gitlab.rb` for [Omnibus GitLab installations](#omnibus-installations). -- `gitlab/config/gitlab.yml` for [source installations](#source-installations). - -### Limitations - -Group SAML on a self-managed instance is limited when compared to the recommended -[instance-wide SAML](../../../integration/saml.md). The recommended solution allows you to take advantage of: - -- [LDAP compatibility](../../../administration/auth/ldap/index.md). -- [LDAP Group Sync](../index.md#manage-group-memberships-via-ldap). -- [Required groups](../../../integration/saml.md#required-groups). -- [Admin groups](../../../integration/saml.md#admin-groups). -- [Auditor groups](../../../integration/saml.md#auditor-groups). - -### Omnibus installations - -1. Make sure GitLab is - [configured with HTTPS](../../../install/installation.md#using-https). -1. Enable OmniAuth and the `group_saml` provider in `gitlab.rb`: - - ```ruby - gitlab_rails['omniauth_enabled'] = true - gitlab_rails['omniauth_providers'] = [{ name: 'group_saml' }] - ``` - -### Source installations - -1. Make sure GitLab is - [configured with HTTPS](../../../install/installation.md#using-https). -1. Enable OmniAuth and the `group_saml` provider in `gitlab/config/gitlab.yml`: - - ```yaml - omniauth: - enabled: true - providers: - - { name: 'group_saml' } - ``` - ## Passwords for users created via SAML SSO for Groups The [Generated passwords for users created through integrated authentication](../../../security/passwords_for_integrated_authentication_methods.md) guide provides an overview of how GitLab generates and sets passwords for users created via SAML SSO for Groups. @@ -336,6 +289,11 @@ Similarly, group members of a role with the appropriate permissions can make use This can then be compared to the [NameID](#nameid) being sent by the Identity Provider by decoding the message with a [SAML debugging tool](#saml-debugging-tools). We require that these match in order to identify users. +### Users receive a 404 + +If a user is trying to sign in for the first time and the GitLab single sign-on URL has not [been configured](#configuring-your-identity-provider), they may see a 404. +As outlined in the [user access section](#linking-saml-to-your-existing-gitlabcom-account), a group Owner will need to provide the URL to users. + ### Message: "SAML authentication failed: Extern uid has already been taken" This error suggests you are signed in as a GitLab user but have already linked your SAML identity to a different GitLab user. Sign out and then try to sign in again using the SSO SAML link, which should log you into GitLab with the linked user account. diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 4f74e672392..c6444ada165 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -17,7 +17,7 @@ GitLab's [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protoc ## Features -Currently, the following actions are available: +The following actions are available: - Create users - Update users (Azure only) @@ -51,7 +51,7 @@ Once [Group Single Sign-On](index.md) has been configured, we can: The SAML application that was created during [Single sign-on](index.md) setup for [Azure](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/view-applications-portal) now needs to be set up for SCIM. -1. Check the configuration for your GitLab SAML app and ensure that **Name identifier value** (NameID) points to `user.objectid` or another unique identifier. This will match the `extern_uid` used on GitLab. +1. Check the configuration for your GitLab SAML app and ensure that **Name identifier value** (NameID) points to `user.objectid` or another unique identifier. This matches the `extern_uid` used on GitLab.  @@ -63,7 +63,7 @@ During this configuration, note the following: - The `Tenant URL` and `secret token` are the ones retrieved in the [previous step](#gitlab-configuration). - Should there be any problems with the availability of GitLab or similar - errors, the notification email set will get those. + errors, the notification email set gets those. - It is recommended to set a notification email and check the **Send an email notification when a failure occurs** checkbox. - For mappings, we will only leave `Synchronize Azure Active Directory Users to AppName` enabled. @@ -75,10 +75,10 @@ You can then test the connection by clicking on **Test Connection**. If the conn 1. Click **Delete** next to the `mail` mapping. 1. Map `userPrincipalName` to `emails[type eq "work"].value` and change its **Matching precedence** to `2`. 1. Map `mailNickname` to `userName`. -1. Determine how GitLab will uniquely identify users. +1. Determine how GitLab uniquely identifies users. - Use `objectId` unless users already have SAML linked for your group. - - If you already have users with SAML linked then use the `Name ID` value from the [SAML configuration](#azure). Using a different value will likely cause duplicate users and prevent users from accessing the GitLab group. + - If you already have users with SAML linked then use the `Name ID` value from the [SAML configuration](#azure). Using a different value may cause duplicate users and prevent users from accessing the GitLab group. 1. Create a new mapping: 1. Click **Add New Mapping**. @@ -110,14 +110,14 @@ You can then test the connection by clicking on **Test Connection**. If the conn NOTE: **Note:** You can control what is actually synced by selecting the `Scope`. For example, - `Sync only assigned users and groups` will only sync the users assigned to - the application (`Users and groups`), otherwise, it will sync the whole Active Directory. + `Sync only assigned users and groups` only syncs the users assigned to + the application (`Users and groups`), otherwise, it syncs the whole Active Directory. -Once enabled, the synchronization details and any errors will appear on the +Once enabled, the synchronization details and any errors appears on the bottom of the **Provisioning** screen, together with a link to the audit logs. CAUTION: **Warning:** -Once synchronized, changing the field mapped to `id` and `externalId` will likely cause provisioning errors, duplicate users, and prevent existing users from accessing the GitLab group. +Once synchronized, changing the field mapped to `id` and `externalId` may cause a number of errors. These include provisioning errors, duplicate users, and may prevent existing users from accessing the GitLab group. ### Okta configuration steps @@ -260,15 +260,9 @@ It is important that this SCIM `id` and SCIM `externalId` are configured to the Group owners can see the list of users and the `externalId` stored for each user in the group SAML SSO Settings page. -Alternatively, the [SCIM API](../../../api/scim.md#get-a-list-of-saml-users) can be used to manually retrieve the `externalId` we have stored for users, also called the `external_uid` or `NameId`. +A possible alternative is to use the [SCIM API](../../../api/scim.md#get-a-list-of-saml-users) to manually retrieve the `externalId` we have stored for users, also called the `external_uid` or `NameId`. -For example: - -```shell -curl 'https://gitlab.example.com/api/scim/v2/groups/GROUP_NAME/Users?startIndex=1"' --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json" -``` - -To see how this compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](index.md#saml-debugging-tools). +To see how the `external_uid` compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](index.md#saml-debugging-tools). #### Update or fix mismatched SCIM externalId and SAML NameId @@ -285,15 +279,9 @@ you can address the problem in the following ways: - You can have users unlink and relink themselves, based on the ["SAML authentication failed: User has already been taken"](./index.md#message-saml-authentication-failed-user-has-already-been-taken) section. - You can unlink all users simultaneously, by removing all users from the SAML app while provisioning is turned on. -- You can use the [SCIM API](../../../api/scim.md#update-a-single-saml-user) to manually correct the `externalId` stored for users to match the SAML `NameId`. +- It may be possible to use the [SCIM API](../../../api/scim.md#update-a-single-saml-user) to manually correct the `externalId` stored for users to match the SAML `NameId`. To look up a user, you'll need to know the desired value that matches the `NameId` as well as the current `externalId`. -It is then possible to issue a manual SCIM#update request, for example: - -```shell -curl --verbose --request PATCH 'https://gitlab.com/api/scim/v2/groups/YOUR_GROUP/Users/OLD_EXTERNAL_UID' --data '{ "Operations": [{"op":"Replace","path":"externalId","value":"NEW_EXTERNAL_UID"}] }' --header "Authorization: Bearer <your_scim_token>" --header "Content-Type: application/scim+json" -``` - It is important not to update these to incorrect values, since this will cause users to be unable to sign in. It is also important not to assign a value to the wrong user, as this would cause users to get signed into the wrong account. #### I need to change my SCIM app diff --git a/doc/user/img/feature_flags_history_note_info_v13_2.png b/doc/user/img/feature_flags_history_note_info_v13_2.png Binary files differindex 403a6002603..07d096b6dde 100644 --- a/doc/user/img/feature_flags_history_note_info_v13_2.png +++ b/doc/user/img/feature_flags_history_note_info_v13_2.png diff --git a/doc/user/img/gitlab_snippet_v13_5.png b/doc/user/img/gitlab_snippet_v13_5.png Binary files differnew file mode 100644 index 00000000000..3fce1d25c3d --- /dev/null +++ b/doc/user/img/gitlab_snippet_v13_5.png diff --git a/doc/user/img/version_history_notes_collapsed_v13_2.png b/doc/user/img/version_history_notes_collapsed_v13_2.png Binary files differindex 42ea11ae8ff..b85c9cb36dd 100644 --- a/doc/user/img/version_history_notes_collapsed_v13_2.png +++ b/doc/user/img/version_history_notes_collapsed_v13_2.png diff --git a/doc/user/index.md b/doc/user/index.md index ce8713591ab..32a1c235882 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -46,8 +46,9 @@ GitLab is a Git-based platform that integrates a great number of essential tools - Building, testing, and deploying with built-in [Continuous Integration](../ci/README.md). - Deploying personal and professional static websites with [GitLab Pages](project/pages/index.md). - Integrating with Docker by using [GitLab Container Registry](packages/container_registry/index.md). -- Tracking the development lifecycle by using [GitLab Value Stream Analytics](project/cycle_analytics.md). +- Tracking the development lifecycle by using [GitLab Value Stream Analytics](analytics/value_stream_analytics.md). - Provide support with [Service Desk](project/service_desk.md). +- [Export issues as CSV](project/issues/csv_export.md). With GitLab Enterprise Edition, you can also: @@ -60,8 +61,7 @@ With GitLab Enterprise Edition, you can also: - Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Search](search/advanced_global_search.md) and [Advanced Search Syntax](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. - [Authenticate users with Kerberos](../integration/kerberos.md). - [Mirror a repository](project/repository/repository_mirroring.md) from elsewhere on your local server. -- [Export issues as CSV](project/issues/csv_export.md). -- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipeline_graphs.md). +- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipelines.md). - [Lock files](project/file_lock.md) to prevent conflicts. - View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](project/deploy_boards.md). - Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index 227a67f8c8a..cee6e21a5c5 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -13,6 +13,32 @@ workflows to tie into GitLab's authentication and authorization. These features lowering the barrier to entry for teams to adopt Terraform, collaborate effectively within GitLab, and support Terraform best practices. +## Quick Start + +Use the following `.gitlab-ci.yml` to set up a simple Terraform project integration +for GitLab versions 13.5 and greater: + +```yaml +include: + - template: Terraform.latest.gitlab-ci.yml + +variables: + # If not using GitLab's HTTP backend, remove this line and specify TF_HTTP_* variables + TF_STATE_NAME: default + TF_CACHE_KEY: default +``` + +This template uses `.latest.`, instead of stable, and may include breaking changes. +This template also includes some opinionated decisions, which you can override: + +- Including the latest [GitLab Terraform Image](https://gitlab.com/gitlab-org/terraform-images). +- Using the [GitLab managed Terraform State](#gitlab-managed-terraform-state) as + the Terraform state storage backend. +- Creating [four pipeline stages](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml): + `init`, `validate`, `build`, and `deploy`. These stages + [run the Terraform commands](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.latest.gitlab-ci.yml) + `init`, `validate`, `plan`, `plan-json`, and `apply`. The `apply` command only runs on `master`. + ## GitLab managed Terraform State > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. @@ -67,8 +93,9 @@ local machine, this is a simple way to get started: 1. On your local machine, run `terraform init`, passing in the following options, replacing `<YOUR-STATE-NAME>`, `<YOUR-PROJECT-ID>`, `<YOUR-USERNAME>` and `<YOUR-ACCESS-TOKEN>` with the relevant values. This command initializes your - Terraform state, and stores that state within your GitLab project. This example - uses `gitlab.com`: + Terraform state, and stores that state within your GitLab project. The name of + your state can contain only uppercase and lowercase letters, decimal digits, + hyphens, and underscores. This example uses `gitlab.com`: ```shell terraform init \ @@ -198,7 +225,7 @@ recommends encrypting plan output or modifying the project visibility settings. ## Example project -See [this reference project](https://gitlab.com/nicholasklick/gitlab-terraform-aws) using GitLab and Terraform to deploy a basic AWS EC2 within a custom VPC. +See [this reference project](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-aws) using GitLab and Terraform to deploy a basic AWS EC2 within a custom VPC. ## Copy Terraform state between backends @@ -294,15 +321,15 @@ rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. ``` -If you type `yes`, it will copy your state from the old location to the new +If you type `yes`, it copies your state from the old location to the new location. You can then go back to running it from within GitLab CI. ## Output Terraform Plan information into a merge request Using the [GitLab Terraform Report artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform), you can expose details from `terraform plan` runs directly into a merge request widget, -enabling you to see statistics about the resources that Terraform will create, -modify, or destroy. +enabling you to see statistics about the resources that Terraform creates, +modifies, or destroys. Let's explore how to configure a GitLab Terraform Report artifact. You can either use a pre-built image which includes a `gitlab-terraform` helper as @@ -434,7 +461,8 @@ apply: ### Multiple Terraform Plan reports -Starting with 13.2, you can display multiple reports on the Merge Request page. The reports will also display the `artifacts: name:`. See example below for a suggested setup. +Starting with GitLab version 13.2, you can display multiple reports on the Merge Request +page. The reports also display the `artifacts: name:`. See example below for a suggested setup. ```yaml default: diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md index 8059b8ca642..c370f9d8725 100644 --- a/doc/user/instance/clusters/index.md +++ b/doc/user/instance/clusters/index.md @@ -2,14 +2,14 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in GitLab 11.11. -## Overview - Similar to [project-level](../../project/clusters/index.md) and [group-level](../../group/clusters/index.md) Kubernetes clusters, instance-level Kubernetes clusters allow you to connect a Kubernetes cluster to the GitLab instance, which enables you to use the same cluster across multiple projects. +The instance level Kubernetes clusters can be found in the top menu by navigating to your instance's **{admin}** **Admin Area > Kubernetes**. + ## Cluster precedence GitLab will try to match clusters in the following order: diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 03a4e4cb244..8b65da4ab94 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -78,13 +78,11 @@ character of the top list item (`C` in this case): - dark - milk -NOTE: **Note:** -We flag any significant differences between Redcarpet and CommonMark - Markdown in this document. +We flag any significant differences between Redcarpet and CommonMark Markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine if they display correctly or not. You can use the -[diff_redcarpet_cmark](https://gitlab.com/digitalmoksha/diff_redcarpet_cmark) +[`diff_redcarpet_cmark`](https://gitlab.com/digitalmoksha/diff_redcarpet_cmark) tool (not an officially supported product) to generate a list of files and the differences between how RedCarpet and CommonMark render the files. It gives an indication if anything needs to be changed - often nothing needs @@ -126,7 +124,7 @@ changing how standard Markdown is used: ### Colors -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#colors). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#colors). It's possible to have color written in HEX, RGB, or HSL format rendered with a color indicator. @@ -235,7 +233,7 @@ To make PlantUML available in GitLab, a GitLab administrator needs to enable it ### Emoji -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#emoji). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#emoji). ```markdown Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: @@ -259,13 +257,14 @@ If you're new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-f Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> -NOTE: **Note:** +#### Emoji and your OS + The emoji example above uses hard-coded images for this documentation. The emoji, when rendered within GitLab, may appear different depending on the OS and browser used. -Most emoji are natively supported on macOS, Windows, iOS, Android, and fall back on image-based emoji where there is no support. +Most emoji are natively supported on macOS, Windows, iOS, Android, and fall back on image-based +emoji where there is no support. -NOTE: **Note:** On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. Ubuntu 18.04 (like many modern Linux distributions) has this font installed by default. @@ -335,7 +334,7 @@ $example = array( ### Inline diff -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-diff). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-diff). With inline diff tags you can display `{+ additions +}` or `[- deletions -]`. @@ -374,7 +373,7 @@ backslash `\`, otherwise the diff highlight don't render correctly: ### Math -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#math). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#math). It's possible to have math written with LaTeX syntax rendered using [KaTeX](https://github.com/KaTeX/KaTeX). @@ -402,8 +401,7 @@ a^2+b^2=c^2 _Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -NOTE: **Note:** -This also works for the Asciidoctor `:stem: latexmath`. For details see +This also works for the Asciidoctor `:stem: latexmath`. For details, see the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). ### Special GitLab references @@ -446,7 +444,7 @@ recognized and formatted with text `#123`. In addition to this, links to some objects are also recognized and formatted. Some examples of these are: -- Comments on issues: `"https://gitlab.com/gitlab-org/gitlab/-/issues/1234#note_101075757"`, which are rendered as `#1234 (note1)` +- Comments on issues: `"https://gitlab.com/gitlab-org/gitlab/-/issues/1234#note_101075757"`, which are rendered as `#1234 (comment 101075757)` - The issues designs tab: `"https://gitlab.com/gitlab-org/gitlab/-/issues/1234/designs"`, which are rendered as `#1234 (designs)`. - Links to individual designs: `"https://gitlab.com/gitlab-org/gitlab/-/issues/1234/designs/layout.png"`, which are rendered as `#1234[layout.png]`. @@ -607,9 +605,9 @@ Quote break. #### Multiline blockquote -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiline-blockquote). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiline-blockquote). -GFM extends the standard Markdown standard by also supporting multi-line blockquotes +GFM extends the standard Markdown by also supporting multi-line blockquotes fenced by `>>>`: ```markdown @@ -688,7 +686,7 @@ Tildes are OK too. #### Colored code and syntax highlighting -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). GitLab uses the [Rouge Ruby library](http://rouge.jneen.net/) for more colorful syntax highlighting in code blocks. For a list of supported languages visit the @@ -781,7 +779,7 @@ Strikethrough is not part of the core Markdown standard, but is part of GFM. #### Multiple underscores in words and mid-word emphasis -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiple-underscores-in-words). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiple-underscores-in-words). It's not usually useful to italicize just _part_ of a word, especially when you're dealing with code and names that often appear with multiple underscores. As a result, @@ -973,7 +971,7 @@ Do not change to a reference style link. #### Videos -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#videos). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#videos). Image tags that link to files with a video extension are automatically converted to a video player. The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`: @@ -990,7 +988,7 @@ Here's a sample video: #### Audio -> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#audio). +If this section is not rendered correctly, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#audio). Similar to videos, link tags for files with an audio extension are automatically converted to an audio player. The valid audio extensions are `.mp3`, `.oga`, `.ogg`, `.spx`, and `.wav`: @@ -1007,7 +1005,7 @@ Here's a sample audio clip: ### Inline HTML -> To see the Markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-html). +To see the Markdown rendered within HTML in the second example, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#inline-html). You can also use raw HTML in your Markdown, and it usually works pretty well. @@ -1071,7 +1069,7 @@ Markdown is fine in GitLab. #### Details and summary -> To see the Markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#details-and-summary). +To see the Markdown rendered within HTML in the second example, [view it in GitLab](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#details-and-summary). Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) @@ -1105,7 +1103,7 @@ These details <em>remain</em> <strong>hidden</strong> until expanded. Markdown inside these tags is supported as well. -NOTE: **Note:** +TIP: **Tip:** If your Markdown isn't rendering correctly, try adding `{::options parse_block_html="true" /}` to the top of the page, and add `markdown="span"` to the opening summary tag like this: `<summary markdown="span">`. @@ -1420,26 +1418,20 @@ Example: ```markdown | header 1 | header 2 | header 3 | -| --- | ------ |---------:| +| --- | ------ |----------| | cell 1 | cell 2 | cell 3 | | cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It eventually wraps the text when the cell is too large for the display size. | -| cell 7 | | cell <br> 9 | -| cell 10 | <ul><li> - [ ] Task One </li></ul> | <ul><li> - [ ] Task Two </li><li> - [ ] Task Three </li></ul> | +| cell 7 | | cell 9 | ``` | header 1 | header 2 | header 3 | -| --- | ------ |---------:| +| --- | ------ |----------| | cell 1 | cell 2 | cell 3 | | cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It eventually wraps the text when the cell is too large for the display size. | -| cell 7 | | cell <br> 9 | -| cell 10 | <ul><li> - [ ] Task One </li></ul> | <ul><li> - [ ] Task Two </li><li> - [ ] Task Three </li></ul> | +| cell 7 | | cell 9 | Additionally, you can choose the alignment of text within columns by adding colons (`:`) -to the sides of the "dash" lines in the second row. This affects every cell in the column. - -NOTE: **Note:** -[Within GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#tables), -the headers are always left-aligned in Chrome and Firefox, and centered in Safari. +to the sides of the "dash" lines in the second row. This affects every cell in the column: ```markdown | Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | @@ -1453,6 +1445,34 @@ the headers are always left-aligned in Chrome and Firefox, and centered in Safar | Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | | Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +[Within GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#tables), +the headers are always left-aligned in Chrome and Firefox, and centered in Safari. + +You can use HTML formatting to adjust the rendering of tables. For example, you can +use `<br>` tags to force a cell to have multiple lines: + +```markdown +| Name | Details | +|------|---------| +| Item1 | This is on one line | +| Item2 | This item has:<br>- Multiple items<br>- That we want listed separately | +``` + +| Name | Details | +|------|---------| +| Item1 | This is on one line | +| Item2 | This item has:<br>- Multiple items<br>- That we want listed separately | + +You can use HTML formatting within GitLab itself to add [task lists](#task-lists) with checkboxes, +but they do not render properly on `docs.gitlab.com`: + +```markdown +| header 1 | header 2 | +|----------|----------| +| cell 1 | cell 2 | +| cell 3 | <ul><li> - [ ] Task one </li><li> - [ ] Task two </li></ul> | +``` + #### Copy from spreadsheet and paste in Markdown [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27205) in GitLab 12.7. @@ -1474,5 +1494,5 @@ entry and paste the spreadsheet: - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). - The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard Markdown. -- The detailed specification for CommonMark can be found in the [CommonMark Spec](https://spec.commonmark.org/current/) -- The [CommonMark Dingus](http://try.commonmark.org) is a handy tool for testing CommonMark syntax. +- You can find the detailed specification for CommonMark in the [CommonMark Spec](https://spec.commonmark.org/current/). +- The [CommonMark Dingus](https://spec.commonmark.org/dingus/) is a handy tool for testing CommonMark syntax. diff --git a/doc/user/packages/composer_repository/img/project_id_v13_5.png b/doc/user/packages/composer_repository/img/project_id_v13_5.png Binary files differnew file mode 100644 index 00000000000..45861b6ff1b --- /dev/null +++ b/doc/user/packages/composer_repository/img/project_id_v13_5.png diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 89e02b4847c..d4fb662c3a6 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -4,172 +4,259 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab Composer Repository +# Composer packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15886) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab Composer Repository, every project can have its own space to store [Composer](https://getcomposer.org/) packages. +Publish [Composer](https://getcomposer.org/) packages in your project's Package Registry. +Then, install the packages whenever you need to use them as a dependency. -## Enabling the Composer Repository +Only Composer 1.x is supported. Consider contributing or even adding support for +[Composer 2.0 in the Package Registry](https://gitlab.com/gitlab-org/gitlab/-/issues/259840). -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Package Registry](../../../administration/packages/index.md). +## Create a Composer package -When the Composer Repository is enabled, it is available for all new projects -by default. To enable it for existing projects, or if you want to disable it: +If you do not have a Composer package, create one and check it in to +a repository. This example shows a GitLab repository, but the repository +can be any public or private repository. -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. +1. Create a directory called `my-composer-package` and change to that directory: -You should then be able to see the **Packages & Registries** section on the left sidebar. + ```shell + mkdir my-composer-package && cd my-composer-package + ``` -## Getting started +1. Run [`composer init`](https://getcomposer.org/doc/03-cli.md#init) and answer the prompts. -This section covers creating a new example Composer package to publish. This is a -quickstart to test out the **GitLab Composer Registry**. + For namespace, enter your unique [namespace](../../../user/group/index.md#namespaces), like your GitLab username or group name. -To complete this section, you need a recent version of [Composer](https://getcomposer.org/). + A file called `composer.json` is created: -### Creating a package project + ```json + { + "name": "<namespace>/composer-test", + "description": "Library XY", + "type": "library", + "license": "GPL-3.0-only", + "authors": [ + { + "name": "John Doe", + "email": "john@example.com" + } + ], + "require": {} + } + ``` -Understanding how to create a full Composer project is outside the scope of this -guide, but you can create a small package to test out the registry. Start by -creating a new directory called `my-composer-package`: +1. Run Git commands to tag the changes and push them to your repository: -```shell -mkdir my-composer-package && cd my-composer-package -``` + ```shell + git init + git add composer.json + git commit -m 'Composer package test' + git tag v1.0.0 + git remote add origin git@gitlab.example.com:<namespace>/<project-name>.git + git push --set-upstream origin master + git push origin v1.0.0 + ``` -Create a new `composer.json` file inside this directory to set up the basic project: +The package is now in your GitLab Package Registry. -```shell -touch composer.json -``` +## Publish a Composer package by using the API -Inside `composer.json`, add the following code: +Publish a Composer package to the Package Registry, +so that anyone who can access the project can use the package as a dependency. -```json -{ - "name": "<namespace>/composer-test", - "type": "library", - "license": "GPL-3.0-only", - "version": "1.0.0" -} -``` +Prerequisites: -Replace `<namespace>` with a unique namespace like your GitLab username or group name. +- A package in a GitLab repository. +- A valid `composer.json` file. +- The Packages feature is enabled in a GitLab repository. +- The project ID, which is on the project's home page. +- A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. -After this basic package structure is created, we need to tag it in Git and push it to the repository. + NOTE: **Note:** + [Deploy tokens](./../../project/deploy_tokens/index.md) are + [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. -```shell -git init -git add composer.json -git commit -m 'Composer package test' -git tag v1.0.0 -git remote add origin git@gitlab.com:<namespace>/<project-name>.git -git push --set-upstream origin master -git push origin v1.0.0 -``` +To publish the package: -### Publishing the package +- Send a `POST` request to the [Packages API](../../../api/packages.md). -Now that the basics of our project is completed, we can publish the package. -To publish the package, you need: + For example, you can use `curl`: -- A personal access token or `CI_JOB_TOKEN`. + ```shell + curl --data tag=<tag> "https://__token__:<personal-access-token>@gitlab.example.com/api/v4/projects/<project_id>/packages/composer" + ``` - ([Deploy tokens](./../../project/deploy_tokens/index.md) are not yet supported for use with Composer.) + - `<personal-access-token>` is your personal access token. + - `<project_id>` is your project ID. + - `<tag>` is the Git tag name of the version you want to publish. + To publish a branch, use `branch=<branch>` instead of `tag=<tag>`. -- Your project ID which can be found on the home page of your project. +You can view the published package by going to **Packages & Registries > Package Registry** and +selecting the **Composer** tab. -To publish the package hosted on GitLab, make a `POST` request to the GitLab package API. -A tool like `curl` can be used to make this request: +## Publish a Composer package by using CI/CD -You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. For example: +You can publish a Composer package to the Package Registry as part of your CI/CD process. -```shell -curl --data tag=<tag> 'https://__token__:<personal-access-token>@gitlab.com/api/v4/projects/<project_id>/packages/composer' -``` +1. Specify a `CI_JOB_TOKEN` in your `.gitlab-ci.yml` file: -Where: + ```yaml + stages: + - deploy -- `<personal-access-token>` is your personal access token. -- `<project_id>` is your project ID. -- `<tag>` is the Git tag name of the version you want to publish. In this example it should be `v1.0.0`. Notice that instead of `tag=<tag>` you can also use `branch=<branch>` to publish branches. + deploy: + stage: deploy + script: + - 'curl --header "Job-Token: $CI_JOB_TOKEN" --data tag=<tag> "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/packages/composer"' + ``` -If the above command succeeds, you now should be able to see the package under the **Packages & Registries** section of your project page. +1. Run the pipeline. -### Publishing the package with CI/CD +You can view the published package by going to **Packages & Registries > Package Registry** and selecting the **Composer** tab. -To work with Composer commands within [GitLab CI/CD](./../../../ci/README.md), you can -publish Composer packages by using `CI_JOB_TOKEN` in your `.gitlab-ci.yml` file: +### Use a CI/CD template -```yaml -stages: - - deploy +A more detailed Composer CI/CD file is also available as a `.gitlab-ci.yml` template: -deploy: - stage: deploy - script: - - 'curl --header "Job-Token: $CI_JOB_TOKEN" --data tag=<tag> "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/packages/composer"' -``` +1. On the left sidebar, click **Project overview**. +1. Above the file list, click **Set up CI/CD**. If this button is not available, select **CI/CD Configuration** and then **Edit**. +1. From the **Apply a template** list, select **Composer**. -### Installing a package +CAUTION: **Warning:** +Do not save unless you want to overwrite the existing CI/CD file. -To install your package, you need: +## Install a Composer package -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. -- Your group ID which can be found on the home page of your project's group. +Install a package from the Package Registry so you can use it as a dependency. -Add the GitLab Composer package repository to your existing project's `composer.json` file, along with the package name and version you want to install like so: +Prerequisites: -```json -{ - ... - "repositories": [ - { "type": "composer", "url": "https://gitlab.com/api/v4/group/<group_id>/-/packages/composer/packages.json" } - ], - "require": { - ... - "<package_name>": "<version>" - }, - ... -} -``` +- A package in the Package Registry. +- The group ID, which is on the group's home page. +- A [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to, at minimum, `read_api`. -Where: + NOTE: **Note:** + [Deploy tokens](./../../project/deploy_tokens/index.md) are + [not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/240897) for use with Composer. -- `<group_id>` is the group ID found under your project's group page. -- `<package_name>` is your package name as defined in your package's `composer.json` file. -- `<version>` is your package version (`1.0.0` in this example). +To install a package: -You also need to create a `auth.json` file with your GitLab credentials: +1. Add the Package Registry URL to your project's `composer.json` file, along with the package name and version you want to install: -```json -{ - "gitlab-token": { - "gitlab.com": "<personal_access_token>" - } -} -``` + - Connect to the Package Registry for your group: -Where: + ```shell + composer config repositories.<group_id> composer https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/ + ``` -- `<personal_access_token>` is your personal access token. + - Set the required package version: -With the `composer.json` and `auth.json` files configured, you can install the package by running `composer`: + ```shell + composer require <package_name>:<version> + ``` -```shell -composer update -``` + Result in the `composer.json` file: -If successful, you should be able to see the output indicating that the package has been successfully installed. + ```json + { + ... + "repositories": { + "<group_id>": { + "type": "composer", + "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/" + }, + ... + }, + "require": { + ... + "<package_name>": "<version>" + }, + ... + } + ``` + + You can unset this with the command: + + ```shell + composer config --unset repositories.<group_id> + ``` + + - `<group_id>` is the group ID. + - `<package_name>` is the package name defined in your package's `composer.json` file. + - `<version>` is the package version. + +1. Create an `auth.json` file with your GitLab credentials: + + ```shell + composer config gitlab-token.<DOMAIN-NAME> <personal_access_token> + ``` + + Result in the `auth.json` file: + + ```json + { + ... + "gitlab-token": { + "<DOMAIN-NAME>": "<personal_access_token>", + ... + } + } + ``` + + You can unset this with the command: + + ```shell + composer config --unset --auth gitlab-token.<DOMAIN-NAME> + ``` + + - `<DOMAIN-NAME>` is the GitLab instance URL `gitlab.com` or `gitlab.example.com`. + - `<personal_access_token>` with the scope set to `read_api`. + +1. If you are on a GitLab self-managed instance, add `gitlab-domains` to `composer.json`. + + ```shell + composer config gitlab-domains gitlab01.example.com gitlab02.example.com + ``` + + Result in the `composer.json` file: + + ```json + { + ... + "repositories": [ + { "type": "composer", "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/" } + ], + "config": { + ... + "gitlab-domains": ["gitlab01.example.com", "gitlab02.example.com"] + }, + "require": { + ... + "<package_name>": "<version>" + }, + ... + } + ``` + + You can unset this with the command: + + ```shell + composer config --unset gitlab-domains + ``` + + NOTE: **Note:** + On GitLab.com, Composer uses the GitLab token from `auth.json` as a private token by default. + Without the `gitlab-domains` definition in `composer.json`, Composer uses the GitLab token + as basic-auth, with the token as a username and a blank password. This results in a 401 error. + +Output indicates that the package has been successfully installed. CAUTION: **Important:** -Make sure to never commit the `auth.json` file to your repository. To install packages from a CI job, +Never commit the `auth.json` file to your repository. To install packages from a CI/CD job, consider using the [`composer config`](https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#authentication) tool with your personal access token stored in a [GitLab CI/CD environment variable](../../../ci/variables/README.md) or in -[Hashicorp Vault](../../../ci/secrets/index.md). +[HashiCorp Vault](../../../ci/secrets/index.md). diff --git a/doc/user/packages/conan_repository/img/conan_package_view.png b/doc/user/packages/conan_repository/img/conan_package_view.png Binary files differdeleted file mode 100644 index 742fb4195da..00000000000 --- a/doc/user/packages/conan_repository/img/conan_package_view.png +++ /dev/null diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index 7c3082e0f83..10c820a86be 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -4,153 +4,150 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab Conan Repository +# Conan packages in the Package Registry > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8248) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.6. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab Conan Repository, every -project can have its own space to store Conan packages. +Publish Conan packages in your project’s Package Registry. Then install the +packages whenever you need to use them as a dependency. - +To publish Conan packages to the Package Registry, add the +Package Registry as a remote and authenticate with it. -## Enabling the Conan Repository +Then you can run `conan` commands and publish your package to the Package Registry. -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Conan Repository](../../../administration/packages/index.md). - -After the Conan Repository is enabled, it's available for all new projects -by default. To enable it for existing projects, or if you want to disable it: - -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. - -You should then be able to see the **Packages & Registries** section on the left sidebar. +## Build a Conan package -## Getting started +This section explains how to install Conan and build a package for your C/C++ project. -This section covers installing Conan and building a package for your C/C++ project. This is a quickstart if you're new -to Conan. If you already are using Conan and understand how to build your own packages, move on to the [next section](#adding-the-gitlab-package-registry-as-a-conan-remote). +If you already use Conan and know how to build your own packages, go to the [next section](#add-the-package-registry-as-a-conan-remote). -### Installing Conan +### Install Conan -Follow the instructions at [conan.io](https://conan.io/downloads.html) to download the Conan package manager to your local development environment. +Download the Conan package manager to your local development environment by following +the instructions at [conan.io](https://conan.io/downloads.html). -Once installation is complete, verify you can use Conan in your terminal by running +When installation is complete, verify you can use Conan in your terminal by running: ```shell conan --version ``` -You should see the Conan version printed in the output: +The Conan version is printed in the output: ```plaintext Conan version 1.20.5 ``` -### Installing CMake +### Install CMake + +When you develop with C++ and Conan, you have a range of compilers to choose from. +This example uses the CMake compiler. + +To install CMake: + +- For Mac, use [homebrew](https://brew.sh/) and run `brew install cmake`. +- For other operating systems, follow the instructions at [cmake.org](https://cmake.org/install/). -When developing with C++ and Conan, you have a wide range of options for compilers. This tutorial walks through using the cmake -compiler. In your terminal, run the command +When installation is complete, verify you can use CMake in your terminal by running: ```shell cmake --version ``` -You should see the cmake version printed in the output. If you see something else, you may have to install cmake. +The CMake version is printed in the output. -On a Mac, you can use [homebrew](https://brew.sh/) to install cmake by running `brew install cmake`. Otherwise, follow -instructions at [cmake.org](https://cmake.org/install/) for your operating system. +### Create a project -### Creating a project +To test the Package Registry, you need a C++ project. If you don't already have one, you can clone the +Conan [hello world starter project](https://github.com/conan-io/hello). -Understanding what is needed to create a valid and compilable C++ project is out of the scope of this guide, but if you're new to C++ and want to try out the GitLab -package registry, Conan.io has a great [hello world starter project](https://github.com/conan-io/hello) that you can clone to get started. +### Build a package -Clone the repository and it can be used for the rest of the tutorial if you don't have your own C++ project. +To build a package: -### Building a package +1. Open a terminal and navigate to your project's root folder. +1. Generate a new recipe by running `conan new` with a package name and version: -In your terminal, navigate to the root folder of your project. Generate a new recipe by running `conan new` and providing it with a -package name and version: + ```shell + conan new Hello/0.1 -t + ``` -```shell -conan new Hello/0.1 -t -``` - -Next, create a package for that recipe by running `conan create` providing the Conan user and channel: +1. Create a package for the recipe by running `conan create` with the Conan user and channel: -```shell -conan create . mycompany/beta -``` + ```shell + conan create . mycompany/beta + ``` -NOTE: **Note:** -If you are using the [instance level remote](#instance-level-remote), a specific [naming convention](#package-recipe-naming-convention-for-instance-level-remote) must be followed. + NOTE: **Note:** + If you use an [instance remote](#add-a-remote-for-your-instance), you must follow a specific [naming convention](#package-recipe-naming-convention-for-instance-remotes). -These two example commands generate a final package with the recipe `Hello/0.1@mycompany/beta`. +A package with the recipe `Hello/0.1@mycompany/beta` is created. -For more advanced details on creating and managing your packages, refer to the [Conan docs](https://docs.conan.io/en/latest/creating_packages.html). +For more details on creating and managing Conan packages, see the [Conan docs](https://docs.conan.io/en/latest/creating_packages.html). -You are now ready to upload your package to the GitLab registry. To get started, first you need to set GitLab as a remote. Then you need to add a Conan user for that remote to authenticate your requests. +## Add the Package Registry as a Conan remote -## Adding the GitLab Package Registry as a Conan remote +To run `conan` commands, you must add the Package Registry as a Conan remote for your project or instance. -You can add the GitLab Package Registry as a Conan remote at the project or instance level. - -### Project level remote +### Add a remote for your project > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11679) in GitLab 13.4. -The project level remote allows you to work with packages within a given project. -The advantage of using the project level remote is there are no restrictions to your -package name, however all GitLab Conan packages require a full recipe -with the user and channel (`package_name/version@user/channel`). +Set a remote so you can work with packages in a project without +having to specify the remote name in every command. + +When you set a remote for a project, there are no restrictions to your package names. +However, your commands must include the full recipe, including the user and channel, +for example, `package_name/version@user/channel`. To add the remote: -```shell -conan remote add gitlab https://gitlab.example.com/api/v4/projects/<project_id>/packages/conan -``` +1. In your terminal, run this command: -Once the remote is set, you can use the remote when running Conan commands by adding `--remote=gitlab` to the end of your commands. + ```shell + conan remote add gitlab https://gitlab.example.com/api/v4/projects/<project_id>/packages/conan + ``` -For example: +1. Use the remote by adding `--remote=gitlab` to the end of your Conan command. -```shell -conan search Hello* --all --remote=gitlab -``` + For example: -### Instance level remote + ```shell + conan search Hello* --all --remote=gitlab + ``` -The instance level remote allows you to use a single remote to access packages accross your entire -GitLab instance. However, when using this remote, there are certain -[package naming restrictions](#package-recipe-naming-convention-for-instance-level-remote) -that must be followed. +### Add a remote for your instance -Add a new remote to your Conan configuration: +Use a single remote to access packages across your entire GitLab instance. -```shell -conan remote add gitlab https://gitlab.example.com/api/v4/packages/conan -``` +However, when using this remote, you must follow these +[package naming restrictions](#package-recipe-naming-convention-for-instance-remotes). + +To add the remote: -Once the remote is set, you can use the remote when running Conan commands by adding `--remote=gitlab` to the end of your commands. +1. In your terminal, run this command: -For example: + ```shell + conan remote add gitlab https://gitlab.example.com/api/v4/packages/conan + ``` -```shell -conan search 'Hello*' --remote=gitlab -``` +1. Use the remote by adding `--remote=gitlab` to the end of your Conan command. -#### Package recipe naming convention for instance level remote + For example: -The standard Conan recipe convention looks like `package_name/version@user/channel`, -but if you're using the [instance level remote](#instance-level-remote), the recipe + ```shell + conan search 'Hello*' --remote=gitlab + ``` + +#### Package recipe naming convention for instance remotes + +The standard Conan recipe convention is `package_name/version@user/channel`, +but if you're using an [instance remote](#add-a-remote-for-your-instance), the recipe `user` must be the plus sign (`+`) separated project path. -The following table shows some example recipes you can give your package based on -the project name and path. +Example recipe names: | Project | Package | Supported | | ---------------------------------- | ----------------------------------------------- | --------- | @@ -159,179 +156,202 @@ the project name and path. | `gitlab-org/gitlab-ce` | `my-package/1.0.0@gitlab-org+gitlab-ce/stable` | Yes | | `gitlab-org/gitlab-ce` | `my-package/1.0.0@foo/stable` | No | -NOTE: **Note:** -[Project level remotes](#project-level-remote) allow for more flexible package names. +[Project remotes](#add-a-remote-for-your-project) have a more flexible naming convention. -## Authenticating to the GitLab Conan Repository +## Authenticate to the Package Registry -You need a personal access token or deploy token. +To authenticate to the Package Registry, you need either a personal access token or deploy token. -For repository authentication: +- If you use a [personal access token](../../../user/profile/personal_access_tokens.md), set the scope to `api`. +- If you use a [deploy token](./../../project/deploy_tokens/index.md), set the scope to `read_package_registry`, `write_package_registry`, or both. -- You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. -- You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. +### Add your credentials to the GitLab remote -### Adding a Conan user to the GitLab remote +Associate your token with the GitLab remote, so that you don't have to explicitly +add a token to every Conan command. -Once you have a personal access token and have [set your Conan remote](#adding-the-gitlab-package-registry-as-a-conan-remote), you can associate the token with the remote so you don't have to explicitly add them to each Conan command you run: +Prerequisites: + +- You must have an authentication token. +- The Conan remote [must be set](#add-the-package-registry-as-a-conan-remote). + +In a terminal, run this command. In this example, the remote name is `gitlab`. Use the name of your remote. ```shell conan user <gitlab_username or deploy_token_username> -r gitlab -p <personal_access_token or deploy_token> ``` -NOTE: **Note:** -If you named your remote something other than `gitlab`, your remote name should be used in this command instead of `gitlab`. - -From now on, when you run commands using `--remote=gitlab`, your username and password is automatically included in the requests. - -NOTE: **Note:** -The personal access token is not stored locally at any moment. Conan uses JSON Web Tokens (JWT), so when you run this command, Conan requests an expirable token from GitLab using your token. The JWT does expire on a regular basis, so you need to re-enter your personal access token when that happens. +Now when you run commands with `--remote=gitlab`, your username and password are automatically included in the requests. -Alternatively, you could explicitly include your credentials in any given command. -For example: +Alternately, you can explicitly include your credentials in any given command. For example: ```shell CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@mycompany/beta --all --remote=gitlab ``` -### Setting a default remote to your project (optional) +NOTE: **Note:** +Your authentication with GitLab expires on a regular basis, +so occasionally you may need to re-enter your personal access token. + +### Set a default remote for your project (optional) + +If you want to interact with the GitLab Package Registry without having to specify a remote, +you can tell Conan to always use the Package Registry for your packages. -If you'd like Conan to always use GitLab as the registry for your package, you can tell Conan to always reference the GitLab remote for a given package recipe: +In a terminal, run this command. ```shell conan remote add_ref Hello/0.1@mycompany/beta gitlab ``` NOTE: **Note:** -The package recipe does include the version, so setting the default remote for `Hello/0.1@user/channel` will not work for `Hello/0.2@user/channel`. -This functionality is best suited for when you want to consume or install packages from the GitLab registry without having to specify a remote. +The package recipe includes the version, so the default remote for `Hello/0.1@user/channel` does not work for `Hello/0.2@user/channel`. -The rest of the example commands in this documentation assume that you've added a Conan user with your credentials to the `gitlab` remote and will not include the explicit credentials or remote option. With that said, be aware that any of the commands could be run without having added a user or default remote: +If you do not set a default user or remote, you can still include the user and remote in your commands: ```shell `CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> <conan command> --remote=gitlab ``` -## Uploading a package +## Publish a Conan package -First you need to [create your Conan package locally](https://docs.conan.io/en/latest/creating_packages/getting_started.html). +Publish a Conan package to the Package Registry, so that anyone who can access the project can use the package as a dependency. -NOTE: **Note:** -If you are using the [instance level remote](#instance-level-remote), a specific [naming convention](#package-recipe-naming-convention-for-instance-level-remote) must be followed. +Prerequisites: + +To publish a Conan package, you need: -Ensure you have a project created on GitLab and that the personal access token you're using has the correct permissions for write access to the container registry by selecting the `api` [scope](../../../user/profile/personal_access_tokens.md#limiting-scopes-of-a-personal-access-token). +- The Package Registry [set as a remote](#add-the-package-registry-as-a-conan-remote). +- [Authentication](#authenticate-to-the-package-registry) set up with the Package Registry. +- A local [Conan package](https://docs.conan.io/en/latest/creating_packages/getting_started.html). + - For an instance remote, the package must meet the [naming convention](#package-recipe-naming-convention-for-instance-remotes). +- A project ID, which is on the project's homepage. -You can upload your package to the GitLab Package Registry using the `conan upload` command: +To publish the package, use the `conan upload` command: ```shell conan upload Hello/0.1@mycompany/beta --all ``` -## Installing a package +## Publish a Conan package by using CI/CD -Conan packages are commonly installed as dependencies using the `conanfile.txt` file. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11678) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.7. -In your project where you would like to install the Conan package as a dependency, open `conanfile.txt` or create -an empty file named `conanfile.txt` in the root of your project. +To work with Conan commands in [GitLab CI/CD](./../../../ci/README.md), you can use +`CI_JOB_TOKEN` in place of the personal access token in your commands. -Add the Conan recipe to the `[requires]` section of the file: +You can provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each +Conan command in your `.gitlab-ci.yml` file. For example: -```ini - [requires] - Hello/0.1@mycompany/beta +```yaml +image: conanio/gcc7 - [generators] - cmake +create_package: + stage: deploy + script: + - conan remote add gitlab https://gitlab.example.com/api/v4/packages/conan + - conan new <package-name>/0.1 -t + - conan create . <group-name>+<project-name>/stable + - CONAN_LOGIN_USERNAME=ci_user CONAN_PASSWORD=${CI_JOB_TOKEN} conan upload <package-name>/0.1@<group-name>+<project-name>/stable --all --remote=gitlab ``` -Next, create a build directory from the root of your project and navigate to it: +Additional Conan images to use as the basis of your CI file are available +in the [Conan docs](https://docs.conan.io/en/latest/howtos/run_conan_in_docker.html#available-docker-images). -```shell -mkdir build && cd build -``` +## Install a Conan package -Now you can install the dependencies listed in `conanfile.txt`: +Install a Conan package from the Package Registry so you can use it as a dependency. -```shell -conan install .. -``` +Conan packages are often installed as dependencies by using the `conanfile.txt` file. + +Prerequisites: + +To install a Conan package, you need: + +- The Package Registry [set as a remote](#add-the-package-registry-as-a-conan-remote). +- [Authentication](#authenticate-to-the-package-registry) set up with the Package Registry. + +1. In the project where you want to install the package as a dependency, open `conanfile.txt`. + Or, in the root of your project, create a file called `conanfile.txt`. + +1. Add the Conan recipe to the `[requires]` section of the file: + + ```plaintext + [requires] + Hello/0.1@mycompany/beta + + [generators] + cmake + ``` + +1. At the root of your project, create a `build` directory and change to that directory: + + ```shell + mkdir build && cd build + ``` + +1. Install the dependencies listed in `conanfile.txt`: + + ```shell + conan install <options> + ``` NOTE: **Note:** -If you're trying to install the package you just created in this tutorial, not much will happen since that package -already exists on your local machine. +If you try to install the package you just created in this tutorial, the package +already exists on your local machine, so this command has no effect. -## Removing a package +## Remove a Conan package There are two ways to remove a Conan package from the GitLab Package Registry. -- **Using the Conan client in the command line:** +- From the command line, using the Conan client: ```shell conan remove Hello/0.2@user/channel --remote=gitlab ``` - You need to explicitly include the remote in this command, otherwise the package is only removed from your + You must explicitly include the remote in this command, otherwise the package is only removed from your local system cache. NOTE: **Note:** This command removes all recipe and binary package files from the Package Registry. -- **GitLab project interface**: in the packages view of your project page, you can delete packages by clicking the red trash icons. +- From the GitLab user interface: -## Searching the GitLab Package Registry for Conan packages + Go to your project's **Packages & Registries > Package Registry**. Remove the package by clicking the red trash icon. -The `conan search` command can be run searching by full or partial package name, or by exact recipe. +## Search for Conan packages in the Package Registry -To search using a partial name, use the wildcard symbol `*`, which should be placed at the end of your search (for example, `my-packa*`): +To search by full or partial package name, or by exact recipe, run the `conan search` command. -```shell -conan search Hello --all --remote=gitlab -conan search He* --all --remote=gitlab -conan search Hello/0.1@mycompany/beta --all --remote=gitlab -``` +- To search for all packages with a specific package name: + + ```shell + conan search Hello --remote=gitlab + ``` -The scope of your search includes all projects you have permission to access, this includes your private projects as well as all public projects. +- To search for a partial name, like all packages starting with `He`: -## Fetching Conan package information from the GitLab Package Registry + ```shell + conan search He* --remote=gitlab + ``` -The `conan info` command returns information about a given package: +The scope of your search includes all projects you have permission to access. This includes your private projects as well as all public projects. + +## Fetch Conan package information from the Package Registry + +The `conan info` command returns information about a package: ```shell conan info Hello/0.1@mycompany/beta ``` -## List of supported CLI commands +## Supported CLI commands The GitLab Conan repository supports the following Conan CLI commands: -- `conan upload`: Upload your recipe and package files to the GitLab Package Registry. -- `conan install`: Install a conan package from the GitLab Package Registry, this includes using the `conanfile.txt` file. -- `conan search`: Search the GitLab Package Registry for public packages, and private packages you have permission to view. -- `conan info`: View the information on a given package from the GitLab Package Registry. -- `conan remove`: Delete the package from the GitLab Package Registry. - -## Using GitLab CI with Conan packages - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11678) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.7. - -To work with Conan commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token in your commands. - -It's easiest to provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each -Conan command in your `.gitlab-ci.yml` file. For example: - -```yaml -image: conanio/gcc7 - -create_package: - stage: deploy - script: - - conan remote add gitlab https://gitlab.example.com/api/v4/packages/conan - - conan new <package-name>/0.1 -t - - conan create . <group-name>+<project-name>/stable - - CONAN_LOGIN_USERNAME=ci_user CONAN_PASSWORD=${CI_JOB_TOKEN} conan upload <package-name>/0.1@<group-name>+<project-name>/stable --all --remote=gitlab - -``` - -You can find additional Conan images to use as the base of your CI file -in the [Conan docs](https://docs.conan.io/en/latest/howtos/run_conan_in_docker.html#available-docker-images). +- `conan upload`: Upload your recipe and package files to the Package Registry. +- `conan install`: Install a Conan package from the Package Registry, this includes using the `conanfile.txt` file. +- `conan search`: Search the Package Registry for public packages, and private packages you have permission to view. +- `conan info`: View the information on a given package from the Package Registry. +- `conan remove`: Delete the package from the Package Registry. diff --git a/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_1.png b/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_1.png Binary files differdeleted file mode 100644 index bbbba44eb9b..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_1.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_hover_path_13_4.png b/doc/user/packages/container_registry/img/container_registry_hover_path_13_4.png Binary files differnew file mode 100644 index 00000000000..2d16c11fe61 --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_hover_path_13_4.png diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_v13_1.png b/doc/user/packages/container_registry/img/container_registry_repositories_v13_1.png Binary files differdeleted file mode 100644 index 13a6d1a4470..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_v13_1.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_1.png b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_1.png Binary files differdeleted file mode 100644 index 35a02182a77..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_1.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png b/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png Binary files differdeleted file mode 100644 index 088e80221de..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png +++ /dev/null diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 077666bc036..baadd3c91a7 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -9,235 +9,186 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/4040) in GitLab 8.8. > - Docker Registry manifest `v1` support was added in GitLab 8.9 to support Docker > versions earlier than 1.10. -> - Starting from GitLab 8.12, if you have 2FA enabled in your account, you need -> to pass a [personal access token](../../profile/personal_access_tokens.md) instead of your password in order to -> login to GitLab's Container Registry. -> - Multiple level image names support was added in GitLab 9.1. -> - The group level Container Registry was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23315) in GitLab 12.10. +> - Starting in GitLab 8.12, if you have [two-factor authentication](../../profile/account/two_factor_authentication.md) enabled in your account, you need +> to pass a [personal access token](../../profile/personal_access_tokens.md) instead of your password to +> sign in to the Container Registry. +> - Support for multiple level image names was added in GitLab 9.1. +> - The group-level Container Registry was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23315) in GitLab 12.10. > - Searching by image repository name was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31322) in GitLab 13.0. -NOTE: **Note:** -This document is the user guide. To learn how to enable GitLab Container -Registry across your GitLab instance, visit the -[administrator documentation](../../../administration/packages/container_registry.md). - -With the Docker Container Registry integrated into GitLab, every project can +With the Docker Container Registry integrated into GitLab, every GitLab project can have its own space to store its Docker images. You can read more about Docker Registry at <https://docs.docker.com/registry/introduction/>. - - -## Enable the Container Registry for your project - -CAUTION: **Warning:** -The Container Registry follows the visibility settings of the project. If the project is public, so is the Container Registry. - -If you cannot find the **Packages & Registries > Container Registry** entry under your -project's sidebar, it is not enabled in your GitLab instance. Ask your -administrator to enable GitLab Container Registry following the -[administration documentation](../../../administration/packages/container_registry.md). +This document is the user guide. To learn how to enable the Container +Registry for your GitLab instance, visit the +[administrator documentation](../../../administration/packages/container_registry.md). -If you are using GitLab.com, this is enabled by default so you can start using -the Registry immediately. Currently there is a soft (10GB) size restriction for -Registry on GitLab.com, as part of the [repository size limit](../../project/repository/index.md). +## View the Container Registry -Once enabled for your GitLab instance, to enable Container Registry for your -project: +You can view the Container Registry for a project or group. -1. Go to your project's **Settings > General** page. -1. Expand the **Visibility, project features, permissions** section - and enable the **Container Registry** feature on your project. For new - projects this might be enabled by default. For existing projects - (prior GitLab 8.8), enable it explicitly. -1. Press **Save changes** for the changes to take effect. You should now be able - to see the **Packages & Registries > Container Registry** link in the sidebar. +1. Go to your project or group. +1. Go to **Packages & Registries > Container Registry**. -## Control Container Registry from within GitLab +You can search, sort, filter, and [delete](#delete-images-from-within-gitlab) containers on this page. -GitLab offers a simple Container Registry management panel. This management panel is available -for both projects and groups. +Only members of the project or group can access a private project's Container Registry. -### Control Container Registry for your project +If a project is public, so is the Container Registry. -Navigate to your project's **{package}** **Packages & Registries > Container Registry**. +## Use images from the Container Registry - +To download and run a container image hosted in the GitLab Container Registry: -This view allows you to: +1. Copy the link to your container image: + - Go to your project or group's **Packages & Registries > Container Registry** + and find the image you want. + - Next to the image name, click the **Copy** button. -- Show all the image repositories that belong to the project. -- Filter image repositories by their name. -- [Delete](#delete-images-from-within-gitlab) one or more image repository. -- Navigate to the image repository details page. -- Show a **Quick start** dropdown with the most common commands to log in, build and push. -- Show a banner if the optional [cleanup policy](#cleanup-policy) is enabled for this project. +  -### Control Container Registry for your group +1. Use `docker run` with the image link: -Navigate to your group's **{package}** **Packages & Registries > Container Registry**. + ```shell + docker run [options] registry.example.com/group/project/image [arguments] + ``` - +For more information on running Docker containers, visit the +[Docker documentation](https://docs.docker.com/engine/userguide/intro/). -This view allows you to: +## Image naming convention -- Show all the image repositories of the projects that belong to this group. -- [Delete](#delete-images-from-within-gitlab) one or more image repositories. -- Navigate to a specific image repository details page. +Images follow this naming convention: -### Image Repository details page +```plaintext +<registry URL>/<namespace>/<project>/<image> +``` -Clicking on the name of any image repository navigates to the details. +If your project is `gitlab.example.com/mynamespace/myproject`, for example, +then your image must be named `gitlab.example.com/mynamespace/myproject/my-app` at a minimum. - +You can append additional names to the end of an image name, up to three levels deep. -NOTE: **Note:** -The following page has the same functionalities both in the **Group level container registry** -and in the **Project level container registry**. +For example, these are all valid image names for images within the project named `myproject`: -This view: +```plaintext +registry.example.com/mynamespace/myproject:some-tag +``` -- Shows all the image repository details. -- Shows all the tags of the image repository. -- Allows you to quickly copy the tag path (by clicking on the clipboard button near the tag name). -- Allows you to [delete one or more tags](#delete-images-from-within-gitlab). +```plaintext +registry.example.com/mynamespace/myproject/image:latest +``` -## Use images from GitLab Container Registry +```plaintext +registry.example.com/mynamespace/myproject/my/image:rc1 +``` -To download and run a container from images hosted in GitLab Container Registry, -use `docker run`: +## Build and push images by using Docker commands -```shell -docker run [options] registry.example.com/group/project/image [arguments] -``` +To build and push to the Container Registry, you can use Docker commands. -For more information on running Docker containers, visit the -[Docker documentation](https://docs.docker.com/engine/userguide/intro/). +### Authenticate with the Container Registry -## Authenticating to the GitLab Container Registry +Before you can build and push images, you must authenticate with the Container Registry. -If you visit the **Packages & Registries > Container Registry** link under your project's -menu, you can see the explicit instructions to login to the Container Registry -using your GitLab credentials. +To authenticate, you can use: -For example if the Registry's URL is `registry.example.com`, then you should be -able to login with: +- A [personal access token](../../profile/personal_access_tokens.md). +- A [deploy token](../../project/deploy_tokens/index.md). -```shell -docker login registry.example.com -``` +Both of these require the minimum scope to be: -NOTE: **Note:** -If you have [2 Factor Authentication](../../profile/account/two_factor_authentication.md) -enabled in your account, you need to pass a -[personal access token](../../profile/personal_access_tokens.md) instead -of your password in order to login to GitLab's Container Registry. +- For read (pull) access, `read_registry`. +- For write (push) access, `write_registry`. -Credentials must be provided for authorization to any non-public registry. Only project members can access private, -GitLab-hosted registries. +To authenticate, run the `docker` command. For example: -There are two ways to authenticate: + ```shell + docker login registry.example.com -u <username> -p <token> + ``` -- By using a [personal access token](../../profile/personal_access_tokens.md). -- By using a [deploy token](../../project/deploy_tokens/index.md). +### Build and push images by using Docker commands -The minimum scope needed for both of them is `read_registry`. +To build and push to the Container Registry: -Example of using a token: +1. Authenticate with the Container Registry. -```shell -docker login registry.example.com -u <username> -p <token> -``` +1. Run the command to build or push. For example, to build: -## Build and push images from your local machine + ```shell + docker build -t registry.example.com/group/project/image . + ``` -Building and publishing images should be a straightforward process. Just make -sure that you are using the Registry URL with the namespace and project name -that is hosted on GitLab: + Or to push: -```shell -docker build -t registry.example.com/group/project/image . -docker push registry.example.com/group/project/image -``` + ```shell + docker push registry.example.com/group/project/image + ``` -Your image is named after the following scheme: +You can also view these commands by going to your project's **Packages & Registries > Container Registry**. -```plaintext -<registry URL>/<namespace>/<project>/<image> -``` +## Build and push by using GitLab CI/CD -GitLab supports up to three levels of image repository names. -The following examples of image tags are valid: +Use [GitLab CI/CD](../../../ci/yaml/README.md) to build and push images to the +Container Registry. Use it to test, build, and deploy your project from the Docker +image you created. -```plaintext -registry.example.com/group/project:some-tag -registry.example.com/group/project/image:latest -registry.example.com/group/project/my/image:rc1 -``` +### Authenticate by using GitLab CI/CD -## Build and push images using GitLab CI/CD - -While you can build and push your images from your local machine, take -full advantage of the Container Registry by combining it with GitLab CI/CD. -You can then create workflows and automate any processes that involve testing, -building, and eventually deploying your project from the Docker image you -created. - -Before diving into the details, some things you should be aware of: - -- You must [authenticate to the container registry](#authenticating-to-the-container-registry-with-gitlab-cicd) - before running any commands. You can do this in the `before_script` if multiple - jobs depend on it. -- Using `docker build --pull` fetches any changes to base - images before building in case your cache is stale. It takes slightly - longer, but it means you don’t get stuck without security patches for base images. -- Doing an explicit `docker pull` before each `docker run` fetches - the latest image that was just built. This is especially important if you are - using multiple runners that cache images locally. Using the Git SHA in your - image tag makes this less necessary since each job is unique and you - shouldn't ever have a stale image. However, it's still possible to have a - stale image if you re-build a given commit after a dependency has changed. -- You don't want to build directly to `latest` tag in case there are multiple jobs - happening simultaneously. +Before you can build and push images by using GitLab CI/CD, you must authenticate with the Container Registry. -### Authenticating to the Container Registry with GitLab CI/CD +To use CI/CD to authenticate, you can use: -There are three ways to authenticate to the Container Registry via -[GitLab CI/CD](../../../ci/yaml/README.md): +- The `CI_REGISTRY_USER` variable. -- **Using the special `CI_REGISTRY_USER` variable**: The user specified by this variable is created for you in order to - push to the Registry connected to your project. Its password is automatically - set with the `CI_REGISTRY_PASSWORD` variable. This allows you to automate building and deploying - your Docker images and has read/write access to the Registry. This is ephemeral, - so it's only valid for one job. You can use the following example as-is: + This variable has read-write access to the Container Registry and is valid for + one job only. Its password is also automatically created and assigned to `CI_REGISTRY_PASSWORD`. ```shell docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY ``` -- **Using the GitLab Deploy Token**: You can create and use a - [special deploy token](../../project/deploy_tokens/index.md#gitlab-deploy-token) - with your projects. - Once created, you can use the special environment variables, and GitLab CI/CD - fills them in for you. You can use the following example as-is: +- A [CI job token](../../../ci/triggers/README.md#ci-job-token). ```shell - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY + docker login -u $CI_JOB_USER -p $CI_JOB_TOKEN $CI_REGISTRY ``` -- **Using a personal access token**: You can create and use a - [personal access token](../../profile/personal_access_tokens.md) - in case your project is private: +- A [deploy token](../../project/deploy_tokens/index.md#gitlab-deploy-token) with the minimum scope of: + - For read (pull) access, `read_registry`. + - For write (push) access, `write_registry`. - - For read (pull) access, the scope should be `read_registry`. - - For write (push) access, the scope should be `write_registry`. + ```shell + docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY + ``` - Replace the `<username>` and `<access_token>` in the following example: +- A [personal access token](../../profile/personal_access_tokens.md) with the minimum scope of: + - For read (pull) access, `read_registry`. + - For write (push) access, `write_registry`. ```shell docker login -u <username> -p <access_token> $CI_REGISTRY ``` +### Configure your `.gitlab-ci.yml` file + +You can configure your `.gitlab-ci.yml` file to build and push images to the Container Registry. + +- If multiple jobs require authentication, put the authentication command in the `before_script`. +- Before building, use `docker build --pull` to fetch changes to base images. It takes slightly + longer, but it ensures your image is up-to-date. +- Before each `docker run`, do an explicit `docker pull` to fetch + the image that was just built. This is especially important if you are + using multiple runners that cache images locally. + + If you use the Git SHA in your image tag, each job is unique and you + should never have a stale image. However, it's still possible to have a + stale image if you re-build a given commit after a dependency has changed. +- Don't build directly to the `latest` tag because multiple jobs may be + happening simultaneously. + ### Container Registry examples with GitLab CI/CD If you're using Docker-in-Docker on your runners, this is how your `.gitlab-ci.yml` @@ -359,15 +310,15 @@ in addition to the steps in the Below is an example of what your `.gitlab-ci.yml` should look like: ```yaml - build: - image: $CI_REGISTRY/group/project/docker:19.03.12 - services: - - name: $CI_REGISTRY/group/project/docker:19.03.12-dind - alias: docker - stage: build - script: - - docker build -t my-docker-image . - - docker run my-docker-image /script/to/run/tests +build: + image: $CI_REGISTRY/group/project/docker:19.03.12 + services: + - name: $CI_REGISTRY/group/project/docker:19.03.12-dind + alias: docker + stage: build + script: + - docker build -t my-docker-image . + - docker run my-docker-image /script/to/run/tests ``` If you forget to set the service alias, the `docker:19.03.12` image is unable to find the @@ -394,7 +345,7 @@ the deleted images. To delete images from within GitLab: -1. Navigate to your project's or group's **{package}** **Packages & Registries > Container Registry**. +1. Navigate to your project's or group's **Packages & Registries > Container Registry**. 1. From the **Container Registry** page, you can select what you want to delete, by either: @@ -406,8 +357,6 @@ To delete images from within GitLab: 1. In the dialog box, click **Remove tag**. -  - ### Delete images using the API If you want to automate the process of deleting images, GitLab provides an API. For more @@ -512,9 +461,28 @@ Cleanup policies can be run on all projects, with these exceptions: for all projects (even those created before 12.8) in [GitLab application settings](../../../api/settings.md#change-application-settings) by setting `container_expiration_policies_enable_historic_entries` to true. + Alternatively, you can execute the following command in the [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + + ```ruby + ApplicationSetting.last.update(container_expiration_policies_enable_historic_entries: true) + ``` There are performance risks with enabling it for all projects, especially if you are using an [external registry](./index.md#use-with-external-container-registries). +- For self-managed GitLab instances, you can enable or disable the cleanup policy for a specific + project. + + To enable it: + + ```ruby + Feature.enable(:container_expiration_policies_historic_entry, Project.find(<project id>)) + ``` + + To disable it: + + ```ruby + Feature.disable(:container_expiration_policies_historic_entry, Project.find(<project id>)) + ``` ### How the cleanup policy works @@ -636,7 +604,7 @@ you can use the Container Registry to store Helm Charts. However, due to the way and stored by Docker, it is not possible for GitLab to parse this data and meet performance standards. [This epic](https://gitlab.com/groups/gitlab-org/-/epics/2313) updates the architecture of the Container Registry to support Helm Charts. -You can read more about the above challenges [here](https://gitlab.com/gitlab-org/gitlab/-/issues/38047#note_298842890). +[Read more about the above challenges](https://gitlab.com/gitlab-org/gitlab/-/issues/38047#note_298842890). ## Limitations @@ -647,6 +615,19 @@ Container Registry, you must delete all existing images. - Prior to GitLab 12.10, any tags that use the same image ID as the `latest` tag are not deleted by the cleanup policy. +## Disable the Container Registry for a project + +The Container Registry is enabled by default. + +You can, however, remove the Container Registry for a project: + +1. Go to your project's **Settings > General** page. +1. Expand the **Visibility, project features, permissions** section + and disable **Container Registry**. +1. Click **Save changes**. + +The **Packages & Registries > Container Registry** entry is removed from the project's sidebar. + ## Troubleshooting the GitLab Container Registry ### Docker connection error @@ -661,6 +642,21 @@ To get around this, you can [change the group path](../../group/index.md#changin [change the project path](../../project/settings/index.md#renaming-a-repository) or change the branch name. +You may also get a `404 Not Found` or `Unknown Manifest` message if you are using +a Docker Engine version earlier than 17.12. Later versions of Docker Engine use +[the v2 API](https://docs.docker.com/registry/spec/manifest-v2-2/). + +The images in your GitLab Container Registry must also use the Docker v2 API. +For information on how to update your images, see the [Docker help](https://docs.docker.com/registry/spec/deprecated-schema-v1). + +### `Blob unknown to registry` error when pushing a manifest list + +When [pushing a Docker manifest list](https://docs.docker.com/engine/reference/commandline/manifest/#create-and-push-a-manifest-list) to the GitLab Container Registry, you may receive the error `manifest blob unknown: blob unknown to registry`. This issue occurs when the individual child manifests referenced in the manifest list were not pushed to the same repository. + +For example, you may have two individual images, one for `amd64` and another for `arm64v8`, and you want to build a multi-arch image with them. The `amd64` and `arm64v8` images must be pushed to the same repository where you want to push the multi-arch image. + +As a workaround, you should include the architecture in the tag name of individual images. For example, use `mygroup/myapp:1.0.0-amd64` instead of using sub repositories, like `mygroup/myapp/amd64:1.0.0`. You can then tag the manifest list with `mygroup/myapp:1.0.0`. + ### Troubleshoot as a GitLab server admin Troubleshooting the GitLab Container Registry, most of the times, requires diff --git a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png b/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png Binary files differdeleted file mode 100644 index e550d296d5a..00000000000 --- a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png +++ /dev/null diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index e3ee8909165..3fa21ef486b 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -8,80 +8,80 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7934) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.11. -NOTE: **Note:** -This is the user guide. In order to use the dependency proxy, an administrator -must first [configure it](../../../administration/packages/dependency_proxy.md). +The GitLab Dependency Proxy is a local proxy you can use for your frequently-accessed +upstream images. -For many organizations, it is desirable to have a local proxy for frequently used -upstream images/packages. In the case of CI/CD, the proxy is responsible for -receiving a request and returning the upstream image from a registry, acting -as a pull-through cache. +In the case of CI/CD, the Dependency Proxy receives a request and returns the +upstream image from a registry, acting as a pull-through cache. -The dependency proxy is available in the group level. To access it, navigate to -a group's **Packages & Registries > Dependency Proxy**. +## Prerequisites - +To use the Dependency Proxy: -## Supported dependency proxies +- Your group must be public. Authentication for private groups is [not supported yet](https://gitlab.com/gitlab-org/gitlab/-/issues/11582). -NOTE: **Note:** -For a list of the upcoming additions to the proxies, visit the -[direction page](https://about.gitlab.com/direction/package/dependency_proxy/#top-vision-items). +### Supported images and packages -The following dependency proxies are supported. +The following images and packages are supported. -| Dependency proxy | GitLab version | +| Image/Package | GitLab version | | ---------------- | -------------- | | Docker | 11.11+ | -## Using the Docker dependency proxy +For a list of planned additions, view the +[direction page](https://about.gitlab.com/direction/package/dependency_proxy/#top-vision-items). + +## View the Dependency Proxy + +To view the Dependency Proxy: + +- Go to your group's **Packages & Registries > Dependency Proxy**. + +The Dependency Proxy is not available for projects. + +## Use the Dependency Proxy for Docker images + +You can use GitLab as a source for your Docker images. -With the Docker dependency proxy, you can use GitLab as a source for a Docker image. -To get a Docker image into the dependency proxy: +Prerequisites: -1. Find the proxy URL on your group's page under **Packages & Registries > Dependency Proxy**, - for example `gitlab.com/groupname/dependency_proxy/containers`. -1. Trigger GitLab to pull the Docker image you want (e.g., `alpine:latest` or - `linuxserver/nextcloud:latest`) and store it in the proxy storage by using - one of the following ways: +- Your images must be stored on [Docker Hub](https://hub.docker.com/). +- Docker Hub must be available. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/241639) + for progress on accessing images when Docker Hub is down. - - Manually pulling the Docker image: +To store a Docker image in Dependency Proxy storage: + +1. Go to your group's **Packages & Registries > Dependency Proxy**. +1. Copy the **Dependency Proxy URL**. +1. Use one of these commands. In these examples, the image is `alpine:latest`. + + - Add the URL to your [`.gitlab-ci.yml`](../../../ci/yaml/README.md#image) file: ```shell - docker pull gitlab.com/groupname/dependency_proxy/containers/alpine:latest + image: gitlab.example.com/groupname/dependency_proxy/containers/alpine:latest ``` - - From a `Dockerfile`: + - Manually pull the Docker image: ```shell - FROM gitlab.com/groupname/dependency_proxy/containers/alpine:latest + docker pull gitlab.example.com/groupname/dependency_proxy/containers/alpine:latest ``` - - In [`.gitlab-ci.yml`](../../../ci/yaml/README.md#image): + - Add the URL to a `Dockerfile`: ```shell - image: gitlab.com/groupname/dependency_proxy/containers/alpine:latest + FROM gitlab.example.com/groupname/dependency_proxy/containers/alpine:latest ``` GitLab pulls the Docker image from Docker Hub and caches the blobs on the GitLab server. The next time you pull the same image, GitLab gets the latest -information about the image from Docker Hub but serves the existing blobs +information about the image from Docker Hub, but serves the existing blobs from the GitLab server. -The blobs are kept forever, and there is no hard limit on how much data can be -stored. - -## Clearing the cache +## Clear the Dependency Proxy cache -It is possible to use the GitLab API to purge the dependency proxy cache for a -given group to gain back disk space that may be taken up by image blobs that -are no longer needed. See the [dependency proxy API documentation](../../../api/dependency_proxy.md) -for more details. - -## Limitations - -The following limitations apply: +Blobs are kept forever on the GitLab server, and there is no hard limit on how much data can be +stored. -- Only [public groups are supported](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) (authentication is not supported yet). -- Only Docker Hub is supported. -- This feature requires Docker Hub being available. +To reclaim disk space used by image blobs that are no longer needed, use +the [Dependency Proxy API](../../../api/dependency_proxy.md). diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md new file mode 100644 index 00000000000..0c961b5c86f --- /dev/null +++ b/doc/user/packages/generic_packages/index.md @@ -0,0 +1,139 @@ +--- +stage: Package +group: Package +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# GitLab Generic Packages Repository **(CORE)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4209) in GitLab 13.5. +> - It's [deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. +> - It's enabled on GitLab.com. +> - It's able to be enabled or disabled per-project. +> - It's recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-generic-packages-in-the-package-registry). + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +Publish generic files, like release binaries, in your project’s Package Registry. Then, install the packages whenever you need to use them as a dependency. + +## Authenticate to the Package Registry + +To authenticate to the Package Registry, you need either a [personal access token](../../../api/README.md#personalproject-access-tokens) +or [CI job token](../../../api/README.md#gitlab-ci-job-token). + +## Publish a package file + +When you publish a package file, if the package does not exist, it is created. + +Prerequisites: + +- You need to [authenticate with the API](../../../api/README.md#authentication). + +```plaintext +PUT /projects/:id/packages/generic/:package_name/:package_version/:file_name +``` + +| Attribute | Type | Required | Description | +| -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | +| `package_name` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). +| `package_version` | string | yes | The package version. It can contain only numbers (`0-9`), and dots (`.`). Must be in the format of `X.Y.Z`, i.e. should match `/\A\d+\.\d+\.\d+\z/` regular expresion. +| `file_name` | string | yes | The file name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), dots (`.`), hyphens (`-`), or underscores (`_`). + +Provide the file context in the request body. + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + --upload-file path/to/file.txt \ + https://gitlab.example.com/api/v4/projects/24/generic/my_package/0.0.1/file.txt +``` + +Example response: + +```json +{ + "message":"201 Created" +} +``` + +## Download package file + +Install a package file. + +Prerequisites: + +- You need to [authenticate with the API](../../../api/README.md#authentication). + +```plaintext +GET /projects/:id/packages/generic/:package_name/:package_version/:file_name +``` + +| Attribute | Type | Required | Description | +| -------------------| --------------- | ---------| ------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/README.md#namespaced-path-encoding). | +| `package_name` | string | yes | The package name. | +| `package_version` | string | yes | The package version. | +| `file_name` | string | yes | The file name. | + +The file context is served in the response body. The response content type will be `application/octet-stream`. + +Example request that uses a personal access token: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + https://gitlab.example.com/api/v4/projects/24/generic/my_package/0.0.1/file.txt +``` + +## Publish a generic package by using CI/CD + +To work with generic packages in [GitLab CI/CD](./../../../ci/README.md), you can use +`CI_JOB_TOKEN` in place of the personal access token in your commands. + +For example: + +```yaml +image: curlimages/curl:latest + +stages: + - upload + - download + +upload: + stage: upload + script: + - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file path/to/file.txt ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/my_package/0.0.1/file.txt' + +download: + stage: download + script: + - 'wget --header="JOB-TOKEN: $CI_JOB_TOKEN" ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/my_package/0.0.1/file.txt' +``` + +### Enable or disable generic packages in the Package Registry + +Support for generic packages is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +# For the instance +Feature.enable(:generic_packages) +# For a single project +Feature.enable(:generic_packages, Project.find(<project id>)) +``` + +To disable it: + +```ruby +# For the instance +Feature.disable(:generic_packages) +# For a single project +Feature.disable(:generic_packages, Project.find(<project id>)) +``` diff --git a/doc/user/packages/go_proxy/index.md b/doc/user/packages/go_proxy/index.md index edf1528a751..bd3b5b49ebd 100644 --- a/doc/user/packages/go_proxy/index.md +++ b/doc/user/packages/go_proxy/index.md @@ -50,7 +50,7 @@ Feature.disable(:go_proxy, Project.find(2)) ### Enable the Package Registry The Package Registry is enabled for new projects by default. If you cannot find -the **{package}** **Packages > List** entry under your project's sidebar, verify +the **Packages > List** entry under your project's sidebar, verify the following: 1. Your GitLab administrator has diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 92d31c31986..53ba3d01b3e 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -23,6 +23,7 @@ The Package Registry supports the following formats: <tr><td><a href="https://docs.gitlab.com/ee/user/packages/npm_registry/index.html">NPM</a></td><td>11.7+</td></tr> <tr><td><a href="https://docs.gitlab.com/ee/user/packages/nuget_repository/index.html">NuGet</a></td><td>12.8+</td></tr> <tr><td><a href="https://docs.gitlab.com/ee/user/packages/pypi_repository/index.html">PyPI</a></td><td>12.10+</td></tr> +<tr><td><a href="https://docs.gitlab.com/ee/user/packages/generic_packages/index.html">Generic packages</a></td><td>13.5+</td></tr> </table> </div> </div> diff --git a/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png b/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png Binary files differdeleted file mode 100644 index 92cefc26660..00000000000 --- a/doc/user/packages/maven_repository/img/maven_package_view_v12_6.png +++ /dev/null diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index 7329725a643..d4a8ff0cdb4 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -4,44 +4,24 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab Maven Repository +# Maven packages in the Package Repository > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5811) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.3. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab [Maven](https://maven.apache.org) Repository, every -project can have its own space to store its Maven artifacts. +Publish [Maven](https://maven.apache.org) artifacts in your project’s Package Registry. +Then, install the packages whenever you need to use them as a dependency. - +## Build a Maven package -## Enabling the Maven Repository +This section explains how to install Maven and build a package. -NOTE: **Note:** -This option is available only if your GitLab administrator has -[enabled support for the Maven repository](../../../administration/packages/index.md). - -After the Packages feature is enabled, the Maven Repository is available for -all new projects by default. To enable it for existing projects, or if you want -to disable it: - -1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. -1. Find the Packages feature and enable or disable it. -1. Click on **Save changes** for the changes to take effect. - -You should then be able to see the **Packages & Registries** section on the left sidebar. -Next, you must configure your project to authorize with the GitLab Maven -repository. +If you already use Maven and know how to build your own packages, go to the +[next section](#add-the-package-registry-as-a-maven-remote). -## Getting Started with Maven +Maven repositories work well with Gradle, too. To set up a Gradle project, see [get started with Gradle](#use-gradle-to-create-a-java-project). -This section covers installing Maven and building a package. This is a -quickstart to help if you're new to building Maven packages. If you're already -using Maven and understand how to build your own packages, move onto the -[next section](#adding-the-gitlab-package-registry-as-a-maven-remote). - -Maven repositories work well with Gradle, too. Move onto [getting started with Gradle](#getting-started-with-gradle) if you want to setup a Gradle project. - -### Installing Maven +### Install Maven The required minimum versions are: @@ -66,7 +46,7 @@ Default locale: en_GB, platform encoding: UTF-8 OS name: "mac os x", version: "10.15.2", arch: "x86_64", family: "mac" ``` -### Creating a project +### Create a project Understanding how to create a full Java project is outside the scope of this guide but you can follow the steps below to create a new project that can be @@ -105,14 +85,14 @@ your project has been set up successfully: You should see a new directory where you ran this command matching your `DartifactId` parameter (in this case it should be `my-project`). -## Getting started with Gradle +## Use Gradle to create a Java project + +This section explains how to install Gradle and initialize a Java project. -This section covers installing Gradle and initializing a Java project. This is a -quickstart to help if you're new to Gradle. If you're already -using Gradle and understand how to build your own packages, move onto the -[next section](#adding-the-gitlab-package-registry-as-a-maven-remote). +If you already use Gradle and know how to build your own packages, go to the +[next section](#add-the-package-registry-as-a-maven-remote). -### Installing Gradle +### Install Gradle Installation is needed only if you want to create a new Gradle project. Follow instructions at [gradle.org](https://gradle.org/install/) to download and install @@ -145,7 +125,7 @@ JVM: 11.0.5 (Oracle Corporation 11.0.5+10) OS: Windows 10 10.0 amd64 ``` -### Creating a project in Gradle +### Create a Java project Understanding how to create a full Java project in Gradle is outside the scope of this guide, but you can follow the steps below to create a new project that can be @@ -209,23 +189,23 @@ Project name (default: test): Enter a project name or hit enter to use the directory name as project name. -## Adding the GitLab Package Registry as a Maven remote +## Add the Package Registry as a Maven remote The next step is to add the GitLab Package Registry as a Maven remote. If a project is private or you want to upload Maven artifacts to GitLab, credentials must be provided for authorization too. Support is available -for [personal access tokens](#authenticating-with-a-personal-access-token), -[CI job tokens](#authenticating-with-a-ci-job-token), and +for [personal access tokens](#authenticate-with-a-personal-access-token), +[CI job tokens](#authenticate-with-a-ci-job-token), and [deploy tokens](../../project/deploy_tokens/index.md) only. Regular username/password credentials do not work. -### Authenticating with a personal access token +### Authenticate with a personal access token To authenticate with a [personal access token](../../profile/personal_access_tokens.md), set the scope to `api` when creating one, and add it to your Maven or Gradle configuration files. -#### Authenticating with a personal access token in Maven +#### Authenticate with a personal access token in Maven Add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: @@ -248,7 +228,7 @@ Add a corresponding section to your </settings> ``` -#### Authenticating with a personal access token in Gradle +#### Authenticate with a personal access token in Gradle Create a file `~/.gradle/gradle.properties` with the following content: @@ -278,12 +258,12 @@ repositories { You should now be able to upload Maven artifacts to your project. -### Authenticating with a CI job token +### Authenticate with a CI job token If you're using GitLab CI/CD, a CI job token can be used instead of a personal access token. -#### Authenticating with a CI job token in Maven +#### Authenticate with a CI job token in Maven To authenticate with a CI job token, add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: @@ -309,7 +289,7 @@ To authenticate with a CI job token, add a corresponding section to your You can read more on [how to create Maven packages using GitLab CI/CD](#creating-maven-packages-with-gitlab-cicd). -#### Authenticating with a CI job token in Gradle +#### Authenticate with a CI job token in Gradle To authenticate with a CI job token, add a repositories section to your [`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) @@ -331,7 +311,7 @@ repositories { } ``` -### Authenticating with a deploy token +### Authenticate with a deploy token > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. @@ -339,7 +319,7 @@ To authenticate with a [deploy token](./../../project/deploy_tokens/index.md), set the scope to `api` when creating one, and add it to your Maven or Gradle configuration files. -#### Authenticating with a deploy token in Maven +#### Authenticate with a deploy token in Maven Add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: @@ -362,7 +342,7 @@ Add a corresponding section to your </settings> ``` -#### Authenticating with a deploy token in Gradle +#### Authenticate with a deploy token in Gradle To authenticate with a deploy token, add a repositories section to your [`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) @@ -441,7 +421,7 @@ repositories { ``` The `id` must be the same with what you -[defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). +[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). Replace `PROJECT_ID` with your project ID which can be found on the home page of your project. @@ -452,7 +432,7 @@ domain name. NOTE: **Note:** For retrieving artifacts, you can use either the [URL encoded](../../../api/README.md#namespaced-path-encoding) path of the project -(e.g., `group%2Fproject`) or the project's ID (e.g., `42`). However, only the +(such as `group%2Fproject`) or the project's ID (such as `42`). However, only the project's ID can be used for uploading. ### Group level Maven endpoint @@ -505,7 +485,7 @@ repositories { ``` The `id` must be the same with what you -[defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). +[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). Replace `my-group` with your group name and `PROJECT_ID` with your project ID which can be found on the home page of your project. @@ -516,7 +496,7 @@ domain name. NOTE: **Note:** For retrieving artifacts, you can use either the [URL encoded](../../../api/README.md#namespaced-path-encoding) path of the group -(e.g., `group%2Fsubgroup`) or the group's ID (e.g., `12`). +(such as `group%2Fsubgroup`) or the group's ID (such as `12`). ### Instance level Maven endpoint @@ -571,7 +551,7 @@ repositories { ``` The `id` must be the same with what you -[defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). +[defined in `settings.xml`](#add-the-package-registry-as-a-maven-remote). Replace `PROJECT_ID` with your project ID which can be found on the home page of your project. @@ -582,12 +562,12 @@ domain name. NOTE: **Note:** For retrieving artifacts, you can use either the [URL encoded](../../../api/README.md#namespaced-path-encoding) path of the project -(e.g., `group%2Fproject`) or the project's ID (e.g., `42`). However, only the +(such as `group%2Fproject`) or the project's ID (such as `42`). However, only the project's ID can be used for uploading. ## Uploading packages -Once you have set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) +Once you have set up the [remote and authentication](#add-the-package-registry-as-a-maven-remote) and [configured your project](#configuring-your-project-to-use-the-gitlab-maven-repository-url), test to upload a Maven artifact from a project of yours. @@ -661,7 +641,7 @@ artifacts or even delete them. ## Installing a package Installing a package from the GitLab Package Registry requires that you set up -the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) +the [remote and authentication](#add-the-package-registry-as-a-maven-remote) as above. Once this is completed, there are two ways to install a package. ### Install using Maven with `mvn install` @@ -708,7 +688,7 @@ Package details page, allowing for quick and easy installation. ### Install using Gradle -Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to build.gradle in the dependencies section: +Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to `build.gradle` in the dependencies section: ```groovy dependencies { @@ -802,7 +782,7 @@ Docker container), and Maven uses the configured CI The example below shows how to create a new package each time the `master` branch is updated: -1. Make sure you use the Job-Token authentication as described in ["Authenticating with a CI job token in Gradle"](#authenticating-with-a-ci-job-token-in-gradle). +1. Make sure you use the Job-Token authentication as described in ["Authenticating with a CI job token in Gradle"](#authenticate-with-a-ci-job-token-in-gradle). 1. Add a `deploy` job to your `.gitlab-ci.yml` file: diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index 2a1c12c2afd..f15b31d8b67 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -286,6 +286,22 @@ By default, when an NPM package is not found in the GitLab NPM Registry, the req Administrators can disable this behavior in the [Continuous Integration settings](../../admin_area/settings/continuous_integration.md). +### Installing packages from other organizations + +You can route package requests to organizations and users outside of GitLab. + +To do this, add lines to your `.npmrc` file, replacing `my-org` with the namespace or group that owns your project's repository. The name is case-sensitive and must match the name of your group or namespace exactly. + +```shell +@foo:registry=https://gitlab.example.com/api/v4/packages/npm/ +//gitlab.com/api/v4/packages/npm/:_authToken= "<your_token>" +//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" + +@my-other-org:registry=https://gitlab.example.com/api/v4/packages/npm/ +//gitlab.com/api/v4/packages/npm/:_authToken= "<your_token>" +//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken= "<your_token>" +``` + ## Removing a package In the packages view of your project page, you can delete packages by clicking @@ -340,6 +356,13 @@ with your personal access token or deploy token): //gitlab.com/api/v4/projects/:_authToken=<your_token> ``` +You can also use `yarn config` instead of `npm config` when setting your auth-token dynamically: + +```shell +yarn config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_authToken' "<your_token>" +yarn config set '//gitlab.com/api/v4/packages/npm/:_authToken' "<your_token>" +``` + ### `npm publish` targets default NPM registry (`registry.npmjs.org`) Ensure that your package scope is set consistently in your `package.json` and `.npmrc` files. @@ -418,5 +441,8 @@ npm dist-tag rm @scope/package@version my-tag # Delete a tag from the package npm install @scope/package@my-tag # Install a specific tag ``` +NOTE: **Note:** +You cannot use your `CI_JOB_TOKEN` or deploy token with the `npm dist-tag` commands. View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/258835) for details. + CAUTION: **Warning:** Due to a bug in NPM 6.9.0, deleting dist tags fails. Make sure your NPM version is greater than 6.9.1. diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index 9fb50ce71fb..22e1a95649d 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -19,7 +19,7 @@ The GitLab NuGet Repository works with: ## Setting up your development environment -[NuGet CLI 5.2 or later](https://www.nuget.org/downloads) is required. Earlier versions have not been tested +[NuGet CLI 5.1 or later](https://www.nuget.org/downloads) is required. Earlier versions have not been tested against the GitLab NuGet Repository and might not work. If you have [Visual Studio](https://visualstudio.microsoft.com/vs/), NuGet CLI is probably already installed. @@ -34,7 +34,7 @@ nuget help You should see something similar to: ```plaintext -NuGet Version: 5.2.0.6090 +NuGet Version: 5.1.0.6013 usage: NuGet <command> [args] [options] Type 'NuGet help <command>' for help on a specific command. @@ -44,7 +44,7 @@ Available commands: ``` NOTE: **Note:** -GitLab currently only supports NuGet v3. Earlier versions are not supported. +GitLab currently only supports NuGet's protocol version 3. Earlier versions are not supported. ### macOS support @@ -154,7 +154,7 @@ To add the GitLab NuGet Repository as a source for .NET, create a file named `nu When uploading packages, note that: -- The maximum allowed size is 50 Megabytes. +- The Package Registry on GitLab.com can store up to 500 MB of content. This limit is [configurable for self-managed GitLab instances](../../../administration/instance_limits.md#package-registry-limits). - If you upload the same package with the same version multiple times, each consecutive upload is saved as a separate file. When installing a package, GitLab serves the most recent file. - When uploading packages to GitLab, they are not displayed in the packages UI of your project @@ -250,21 +250,21 @@ is updated: 1. Add a `deploy` job to your `.gitlab-ci.yml` file: ```yaml - image: mcr.microsoft.com/dotnet/core/sdk:3.1 - - stages: - - deploy - - deploy: - stage: deploy - script: - - dotnet restore -p:Configuration=Release - - dotnet build -c Release - - dotnet pack -c Release - - dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text - - dotnet nuget push "bin/Release/*.nupkg" --source gitlab - only: - - master + image: mcr.microsoft.com/dotnet/core/sdk:3.1 + + stages: + - deploy + + deploy: + stage: deploy + script: + - dotnet restore -p:Configuration=Release + - dotnet build -c Release + - dotnet pack -c Release + - dotnet nuget add source "$CI_SERVER_URL/api/v4/projects/$CI_PROJECT_ID/packages/nuget/index.json" --name gitlab --username gitlab-ci-token --password $CI_JOB_TOKEN --store-password-in-clear-text + - dotnet nuget push "bin/Release/*.nupkg" --source gitlab + only: + - master ``` 1. Commit the changes and push it to your GitLab repository to trigger a new CI build. diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 9f954627b05..ae9ca5b2333 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -17,7 +17,7 @@ packages, which can be easily consumed as a dependency in downstream projects. You can view packages for your project or group. 1. Go to the project or group. -1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Go to **Packages & Registries > Package Registry**. You can search, sort, and filter packages on this page. @@ -31,7 +31,7 @@ authenticate with GitLab by using the `CI_JOB_TOKEN`. CI/CD templates, which you can use to get started, are in [this repo](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). -Learn more about [using CI/CD to build Maven packages](../maven_repository/index.md#creating-maven-packages-with-gitlab-cicd), [NPM packages](../npm_registry/index.md#publishing-a-package-with-cicd), [Composer packages](../composer_repository/index.md#publishing-the-package-with-cicd), [NuGet Packages](../nuget_repository/index.md#publishing-a-nuget-package-with-cicd), [Conan Packages](../conan_repository/index.md#using-gitlab-ci-with-conan-packages), and [PyPI packages](../pypi_repository/index.md#using-gitlab-ci-with-pypi-packages). +Learn more about [using CI/CD to build Maven packages](../maven_repository/index.md#creating-maven-packages-with-gitlab-cicd), [NPM packages](../npm_registry/index.md#publishing-a-package-with-cicd), [Composer packages](../composer_repository/index.md#publish-a-composer-package-by-using-cicd), [NuGet Packages](../nuget_repository/index.md#publishing-a-nuget-package-with-cicd), [Conan Packages](../conan_repository/index.md#publish-a-conan-package-by-using-cicd), [PyPI packages](../pypi_repository/index.md#using-gitlab-ci-with-pypi-packages), and [generic packages](../generic_packages/index.md#publish-a-generic-package-by-using-cicd). If you use CI/CD to build a package, extended activity information is displayed when you view the package details: @@ -45,7 +45,7 @@ user who triggered it. To download a package: -1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Go to **Packages & Registries > Package Registry**. 1. Click the name of the package you want to download. 1. In the **Activity** section, click the name of the package you want to download. @@ -60,7 +60,7 @@ You can delete packages by using [the API](../../../api/packages.md#delete-a-pro To delete a package in the UI, from your group or project: -1. Go to **{package}** **Packages & Registries > Package Registry**. +1. Go to **Packages & Registries > Package Registry**. 1. Find the name of the package you want to delete. 1. Click **Delete**. @@ -71,7 +71,7 @@ The package is permanently deleted. The Package Registry is automatically enabled. If you are using a self-managed instance of GitLab, your administrator can remove -the menu item, **{package}** **Packages & Registries**, from the GitLab sidebar. For more information, +the menu item, **Packages & Registries**, from the GitLab sidebar. For more information, see the [administration documentation](../../../administration/packages/index.md). You can also remove the Package Registry for your project specifically: @@ -81,7 +81,7 @@ You can also remove the Package Registry for your project specifically: **Packages** feature. 1. Click **Save changes**. -The **{package}** **Packages & Registries > Package Registry** entry is removed from the sidebar. +The **Packages & Registries > Package Registry** entry is removed from the sidebar. ## Package workflows diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 97f3f69d676..33c37ee6a6c 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -4,14 +4,14 @@ group: Package info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# GitLab PyPi Repository +# GitLab PyPI Repository > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/221259) to GitLab Core in 13.3. -With the GitLab PyPi Repository, every project can have its own space to store PyPi packages. +With the GitLab PyPI Repository, every project can have its own space to store PyPI packages. -The GitLab PyPi Repository works with: +The GitLab PyPI Repository works with: - [pip](https://pypi.org/project/pip/) - [twine](https://pypi.org/project/twine/) @@ -20,13 +20,13 @@ The GitLab PyPi Repository works with: You need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). -## Enabling the PyPi Repository +## Enabling the PyPI Repository NOTE: **Note:** This option is available only if your GitLab administrator has [enabled support for the Package Registry](../../../administration/packages/index.md). -After the PyPi Repository is enabled, it is available for all new projects +After the PyPI Repository is enabled, it is available for all new projects by default. To enable it for existing projects, or if you want to disable it: 1. Navigate to your project's **Settings > General > Visibility, project features, permissions**. @@ -37,8 +37,8 @@ You should then be able to see the **Packages & Registries** section on the left ## Getting started -This section covers creating a new example PyPi package to upload. This is a -quickstart to test out the **GitLab PyPi Registry**. If you already understand how +This section covers creating a new example PyPI package to upload. This is a +quickstart to test out the **GitLab PyPI Registry**. If you already understand how to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source). ### Create a project @@ -111,6 +111,11 @@ touch setup.py This file contains all the information about our package. For more information about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). +GitLab identifies packages based on +[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), +so ensure your package name meets these requirements. +See the [installation section](#install-packages) for more details. + For this guide, we don't need to extensively fill out this file, simply add the below to your `setup.py`: @@ -152,10 +157,10 @@ And confirm your output matches the below: mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz ``` -Our package is now all set up and ready to be uploaded to the **GitLab PyPi +Our package is now all set up and ready to be uploaded to the **GitLab PyPI Package Registry**. Before we do so, we next need to set up authentication. -## Adding the GitLab PyPi Repository as a source +## Adding the GitLab PyPI Repository as a source ### Authenticating with a personal access token @@ -256,7 +261,7 @@ TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username ``` If you did not follow the guide above, then you need to ensure your package -has been properly built and you [created a PyPi package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). +has been properly built and you [created a PyPI package with `setuptools`](https://packaging.python.org/tutorials/packaging-projects/). You can then upload your package using the following command: @@ -274,7 +279,7 @@ Where: Install the latest version of a package using the following command: ```shell -pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> +pip install --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> ``` Where: @@ -287,7 +292,7 @@ If you were following the guide above and want to test installing the `MyPyPiPackage` package, you can run the following: ```shell -pip install mypypipackage --no-deps --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +pip install mypypipackage --no-deps --extra-index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple ``` This should result in the following: @@ -300,6 +305,12 @@ Installing collected packages: mypypipackage Successfully installed mypypipackage-0.0.1 ``` +GitLab looks for packages using +[Python normalized names (PEP-503)](https://www.python.org/dev/peps/pep-0503/#normalized-names), +so the characters `-`, `_`, and `.` are all treated the same and repeated characters are removed. +A `pip install` request for `my.package` looks for packages that match any of +the three characters, such as `my-package`, `my_package`, and `my....package`. + ## Using GitLab CI with PyPI packages > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202012) in GitLab 13.4. diff --git a/doc/user/packages/workflows/monorepo.md b/doc/user/packages/workflows/monorepo.md index c87f181bf82..f20f3427ac5 100644 --- a/doc/user/packages/workflows/monorepo.md +++ b/doc/user/packages/workflows/monorepo.md @@ -7,17 +7,17 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Monorepo package management workflows Oftentimes, one project or Git repository may contain multiple different -subprojects or submodules that all get packaged and published individually. +sub-projects or submodules that all get packaged and published individually. ## Publishing different packages to the parent project The number and name of packages you can publish to one project is not limited. You can accomplish this by setting up different configuration files for each package. See the documentation for the package manager of your choice since -each will have its own specific files and instructions to follow to publish +each has its own specific files and instructions to follow to publish a given package. -Here, we will walk through how to do this with [NPM](../npm_registry/index.md). +Here, we take a walk through how to do this with [NPM](../npm_registry/index.md). Let us say we have a project structure like so: @@ -36,27 +36,27 @@ as well as `Foo`. Following the instructions in the [GitLab NPM registry documentation](../npm_registry/index.md), publishing `MyProject` consists of modifying the `package.json` file with a -`publishConfig` section, as well as either modifying your local NPM config with +`publishConfig` section, as well as either modifying your local NPM configuration with CLI commands like `npm config set`, or saving a `.npmrc` file in the root of the -project specifying these config settings. +project specifying these configuration settings. If you follow the instructions you can publish `MyProject` by running `npm publish` from the root directory. Publishing `Foo` is almost exactly the same, you simply have to follow the steps -while in the `Foo` directory. `Foo` will need its own `package.json` file, -which can be added manually or using `npm init`. And it will need its own +while in the `Foo` directory. `Foo` needs its own `package.json` file, +which can be added manually or using `npm init`. It also needs its own configuration settings. Since you are publishing to the same place, if you used `npm config set` to set the registry for the parent project, then no -additional setup is necessary. If you used a `.npmrc` file, you will need an +additional setup is necessary. If you used a `.npmrc` file, you need an additional `.npmrc` file in the `Foo` directory (be sure to add `.npmrc` files to the `.gitignore` file or use environment variables in place of your access tokens to prevent them from being exposed). It can be identical to the one you used in `MyProject`. You can now run `npm publish` from the `Foo` -directory and you will be able to publish `Foo` separately from `MyProject` +directory and you can publish `Foo` separately from `MyProject` A similar process could be followed for Conan packages, instead of dealing with -`.npmrc` and `package.json`, you will just be dealing with `conanfile.py` in +`.npmrc` and `package.json`, you just deal with `conanfile.py` in multiple locations within the project. ## Publishing to other projects @@ -64,9 +64,9 @@ multiple locations within the project. A package is associated with a project on GitLab, but the package does not need to be associated with the code in that project. Notice when configuring NPM or Maven, you only use the `Project ID` to set the registry URL that the -package will be uploaded to. If you set this to any project that you have -access to and update any other config similarly depending on the package type, -your packages will be published to that project. This means you can publish +package is to be uploaded to. If you set this to any project that you have +access to and update any other configuration similarly depending on the package type, +your packages are published to that project. This means you can publish multiple packages to one project, even if their code does not exist in the same place. See the [project registry workflow documentation](./project_registry.md) for more details. @@ -77,7 +77,7 @@ CI pipelines open an entire world of possibilities for dealing with the patterns described in the previous sections. A common desire would be to publish specific packages only if changes were made to those directories. -Using the example project above, this `gitlab-ci.yml` file will publish +Using the example project above, this `gitlab-ci.yml` file publishes `Foo` anytime changes are made to the `Foo` directory on the `master` branch, and publish `MyPackage` anytime changes are made to anywhere _except_ the `Foo` directory on the `master` branch. diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index 571cda09e69..24437e6dfac 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -26,8 +26,8 @@ point them at the same project on GitLab. There are a few reasons you might want to publish all your packages to one project on GitLab: 1. You want to publish your packages on GitLab, but to a project that is different from where your code is stored. -1. You would like to group packages together in ways that make sense for your usage (all NPM packages in one project, - all packages being used by a specific department in one project, all private packages in one project, etc.) +1. You would like to group packages together in ways that make sense for your usage (such as all NPM packages in one project, + all packages being used by a specific department in one project, or all private packages in one project) 1. You want to use one remote for all of your packages when installing them into other projects. 1. You would like to migrate your packages to a single place on GitLab from a third-party package registry and do not want to worry about setting up separate projects for each package. @@ -44,7 +44,7 @@ Let's take a look at how you might create a public place to hold all of your pub ### Create a project First, create a new project on GitLab. It does not have to have any code or content. Make note of the project ID -displayed on the project overview page, as you will need this later. +displayed on the project overview page for use later in this process. ### Create an access token @@ -67,24 +67,24 @@ If you are using NPM, this involves creating an `.npmrc` file and adding the app to your project using your project ID, then adding a section to your `package.json` file with a similar URL. Follow -the instructions in the [GitLab NPM Registry documentation](../npm_registry/index.md#authenticating-to-the-gitlab-npm-registry). Once -you do this, you will be able to push your NPM package to your project using `npm publish`, as described in the +the instructions in the [GitLab NPM Registry documentation](../npm_registry/index.md#authenticating-to-the-gitlab-npm-registry). After +you do this, you can push your NPM package to your project using `npm publish`, as described in the [uploading packages](../npm_registry/index.md#uploading-packages) section of the docs. #### Maven If you are using Maven, this involves updating your `pom.xml` file with distribution sections, including the appropriate URL for your project, as described in the [GitLab Maven Repository documentation](../maven_repository/index.md#project-level-maven-endpoint). -Then, you need to add a `settings.xml` file and [include your access token](../maven_repository/index.md#authenticating-with-a-personal-access-token). +Then, you need to add a `settings.xml` file and [include your access token](../maven_repository/index.md#authenticate-with-a-personal-access-token). Now you can [deploy Maven packages](../maven_repository/index.md#uploading-packages) to your project. #### Conan -For Conan, first you need to add GitLab as a Conan registry remote. Follow the instructions in the [GitLab Conan Repository docs](../conan_repository/index.md#adding-the-gitlab-package-registry-as-a-conan-remote) +For Conan, first you need to add GitLab as a Conan registry remote. Follow the instructions in the [GitLab Conan Repository docs](../conan_repository/index.md#add-the-package-registry-as-a-conan-remote) to do so. Then, create your package using the plus-separated (`+`) project path as your Conan user. For example, if your project is located at `https://gitlab.com/foo/bar/my-proj`, then you can [create your Conan package](../conan_repository/index.md) -using `conan create . foo+bar+my-proj/channel`, where `channel` is your package channel (`stable`, `beta`, etc.). Once your package -is created, you are ready to [upload your package](../conan_repository/index.md#uploading-a-package) depending on your final package recipe. For example: +using `conan create . foo+bar+my-proj/channel`, where `channel` is your package channel (such as `stable` or `beta`). After your package +is created, you are ready to [upload your package](../conan_repository/index.md#publish-a-conan-package) depending on your final package recipe. For example: ```shell CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> conan upload MyPackage/1.0.0@foo+bar+my-proj/channel --all --remote=gitlab diff --git a/doc/user/permissions.md b/doc/user/permissions.md index e2baac1a962..2e9f36360c6 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -1,5 +1,7 @@ --- -description: 'Understand and explore the user permission levels in GitLab, and what features each of them grants you access to.' +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Permissions @@ -8,7 +10,7 @@ Users have different abilities depending on the access level they have in a particular group or project. If a user is both in a project's group and the project itself, the highest permission level is used. -On public and internal projects the Guest role is not enforced. All users can: +On public and internal projects, the Guest role is not enforced. All users can: - Create issues. - Leave comments. @@ -42,17 +44,17 @@ or an instance administrator, who receives all permissions. For more information The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer |Maintainer| Owner* | +| Action | Guest | Reporter | Developer |Maintainer| Owner (*10*) | |---------------------------------------------------|---------|------------|-------------|----------|--------| | Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | -| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | | View Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View licenses in Dependency list **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | +| View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | | View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View GitLab Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -63,30 +65,35 @@ The following table depicts the various user permission levels in a project. | Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | | See related issues | ✓ | ✓ | ✓ | ✓ | ✓ | | Create confidential issue | ✓ | ✓ | ✓ | ✓ | ✓ | -| View confidential issues | (*2*) | ✓ | ✓ | ✓ | ✓ | | View [Releases](project/releases/index.md) | ✓ (*6*) | ✓ | ✓ | ✓ | ✓ | | View requirements **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Merge Request analytics **(STARTER)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | | Manage user-starred metrics dashboards (*7*) | ✓ | ✓ | ✓ | ✓ | ✓ | +| View confidential issues | (*2*) | ✓ | ✓ | ✓ | ✓ | | Assign issues | | ✓ | ✓ | ✓ | ✓ | | Label issues | | ✓ | ✓ | ✓ | ✓ | | Set issue weight | | ✓ | ✓ | ✓ | ✓ | | Lock issue threads | | ✓ | ✓ | ✓ | ✓ | | Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | -| Manage related issues **(STARTER)** | | ✓ | ✓ | ✓ | ✓ | +| Manage related issues | | ✓ | ✓ | ✓ | ✓ | | Manage labels | | ✓ | ✓ | ✓ | ✓ | | Create code snippets | | ✓ | ✓ | ✓ | ✓ | | See a commit status | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | -| View project statistics | | | ✓ | ✓ | ✓ | +| View CI/CD analytics | | ✓ | ✓ | ✓ | ✓ | +| View Code Review analytics **(STARTER)** | | ✓ | ✓ | ✓ | ✓ | +| View Repository analytics | | ✓ | ✓ | ✓ | ✓ | | View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | | Create new merge request | | ✓ | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create/edit requirements **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | | Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | -| Delete [packages](packages/index.md) | | | | ✓ | ✓ | | Create/edit/delete a Cleanup policy | | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | | Create/edit/delete [Releases](project/releases/index.md)| | | ✓ | ✓ | ✓ | @@ -99,9 +106,12 @@ The following table depicts the various user permission levels in a project. | Lock merge request threads | | | ✓ | ✓ | ✓ | | Approve merge requests (*9*) | | | ✓ | ✓ | ✓ | | Manage/Accept merge requests | | | ✓ | ✓ | ✓ | +| View project statistics | | | ✓ | ✓ | ✓ | | Create new environments | | | ✓ | ✓ | ✓ | | Stop environments | | | ✓ | ✓ | ✓ | | Enable Review Apps | | | ✓ | ✓ | ✓ | +| View Pods logs | | | ✓ | ✓ | ✓ | +| Read Terraform state | | | ✓ | ✓ | ✓ | | Add tags | | | ✓ | ✓ | ✓ | | Cancel and retry jobs | | | ✓ | ✓ | ✓ | | Create or update commit status | | | ✓ (*5*) | ✓ | ✓ | @@ -116,12 +126,14 @@ The following table depicts the various user permission levels in a project. | Create vulnerability from vulnerability finding **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Resolve vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Dismiss vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Revert vulnerability to detected state **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Apply code change suggestions | | | ✓ | ✓ | ✓ | | Create and edit wiki pages | | | ✓ | ✓ | ✓ | | Rewrite/remove Git tags | | | ✓ | ✓ | ✓ | | Manage Feature Flags **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | Run CI/CD pipeline against a protected branch | | | ✓ (*5*) | ✓ | ✓ | +| Delete [packages](packages/index.md) | | | | ✓ | ✓ | | Request a CVE ID **(FREE ONLY)** | | | | ✓ | ✓ | | Use environment terminals | | | | ✓ | ✓ | | Run Web IDE's Interactive Web Terminals **(ULTIMATE ONLY)** | | | | ✓ | ✓ | @@ -132,6 +144,7 @@ The following table depicts the various user permission levels in a project. | Enable/disable tag protections | | | | ✓ | ✓ | | Edit project settings | | | | ✓ | ✓ | | Edit project badges | | | | ✓ | ✓ | +| Export project | | | | ✓ | ✓ | | Share (invite) projects with groups | | | | ✓ (*8*) | ✓ (*8*)| | Add deploy keys to project | | | | ✓ | ✓ | | Configure project hooks | | | | ✓ | ✓ | @@ -143,8 +156,6 @@ The following table depicts the various user permission levels in a project. | Remove GitLab Pages | | | | ✓ | ✓ | | Manage clusters | | | | ✓ | ✓ | | Manage Project Operations | | | | ✓ | ✓ | -| View Pods logs | | | ✓ | ✓ | ✓ | -| Read Terraform state | | | ✓ | ✓ | ✓ | | Manage Terraform state | | | | ✓ | ✓ | | Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | @@ -165,17 +176,8 @@ The following table depicts the various user permission levels in a project. | Disable notification emails | | | | | ✓ | | Force push to protected branches (*4*) | | | | | | | Remove protected branches (*4*) | | | | | | -| View CI\CD analytics | | ✓ | ✓ | ✓ | ✓ | -| View Code Review analytics **(STARTER)** | | ✓ | ✓ | ✓ | ✓ | -| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Merge Request analytics **(STARTER)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Repository analytics | | ✓ | ✓ | ✓ | ✓ | -| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -\* Owner permission is only available at the group or personal namespace level (and for instance admins) and is inherited by its projects. - -1. Guest users are able to perform this action on public and internal projects, but not private projects. +1. Guest users are able to perform this action on public and internal projects, but not private projects. This doesn't apply to [external users](#external-users) where explicit access must be given even if the project is internal. 1. Guest users can only view the confidential issues they created themselves. 1. If **Public pipelines** is enabled in **Project Settings > CI/CD**. 1. Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [Protected Branches](./project/protected_branches.md). @@ -185,6 +187,7 @@ The following table depicts the various user permission levels in a project. 1. When [Share Group Lock](./group/index.md#share-with-group-lock) is enabled the project can't be shared with other groups. It does not affect group with group sharing. 1. For information on eligible approvers for merge requests, see [Eligible approvers](project/merge_requests/merge_request_approvals.md#eligible-approvers). +1. Owner permission is only available at the group or personal namespace level (and for instance admins) and is inherited by its projects. ## Project features permissions @@ -195,7 +198,7 @@ which visibility level you select on project settings. - Disabled: disabled for everyone - Only team members: only team members can see even if your project is public or internal -- Everyone with access: everyone can see depending on your project visibility level +- Everyone with access: everyone can see depending on your project's visibility level - Everyone: enabled for everyone (only available for GitLab Pages) ### Protected branches @@ -224,7 +227,7 @@ Read through the documentation on [permissions for File Locking](project/file_lo ### Confidential Issues permissions -Confidential issues can be accessed by reporters and higher permission levels, +Confidential issues can be accessed by users with reporter and higher permission levels, as well as by guest users that create a confidential issue. To learn more, read through the documentation on [permissions and access to confidential issues](project/issues/confidential_issues.md#permissions-and-access-to-confidential-issues). @@ -240,6 +243,7 @@ group. | Action | Guest | Reporter | Developer | Maintainer | Owner | |--------------------------------------------------------|-------|----------|-----------|------------|-------| | Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| View group wiki pages **(PREMIUM)** | ✓ (6) | ✓ | ✓ | ✓ | ✓ | | View Insights charts **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View group epic **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | | Create/edit group epic **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | @@ -253,10 +257,12 @@ group. | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | | Create/edit/delete iterations | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy **(PREMIUM)** | | | ✓ | ✓ | ✓ | +| Create and edit group wiki pages **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | | Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | | Create subgroup | | | | ✓ (1) | ✓ | +| Delete group wiki pages **(PREMIUM)** | | | | ✓ | ✓ | | Edit epic comments (posted by any user) **(ULTIMATE)** | | | | ✓ (2) | ✓ (2) | | Edit group settings | | | | | ✓ | | Manage group level CI/CD variables | | | | | ✓ | @@ -270,7 +276,7 @@ group. | Disable notification emails | | | | | ✓ | | View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | | View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | | View Billing **(FREE ONLY)** | | | | | ✓ (4) | @@ -283,7 +289,8 @@ group. - The [instance level](admin_area/settings/visibility_and_access_controls.md#default-project-creation-protection). - The [group level](group/index.md#default-project-creation-level). 1. Does not apply to subgroups. -1. Developers can push commits to the default branch of a new project only if the [default branch protection](group/index.md#changing-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". +1. Developers can push commits to the default branch of a new project only if the [default branch protection](group/index.md#changing-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". +1. In addition, if your group is public or internal, all users who can see the group can also see group wiki pages. ### Subgroup permissions @@ -311,7 +318,7 @@ External users: Access can be granted by adding the user as member to the project or group. Like usual users, they receive a role in the project or group with all the abilities that are mentioned in the [permissions table above](#project-members-permissions). -For example, if an external user is added as Guest, and your project is +For example, if an external user is added as Guest, and your project is internal or private, they do not have access to the code; you need to grant the external user access at the Reporter level or above if you want them to have access to the code. You should always take into account the @@ -339,7 +346,7 @@ The **Internal users** field allows specifying an email address regex pattern to identify default internal users. New users whose email address matches the regex pattern are set to internal by default rather than an external collaborator. -The regex pattern format is Ruby, but it needs to be convertible to JavaScript, +The regex pattern format is in Ruby, but it needs to be convertible to JavaScript, and the ignore case flag is set (`/regex pattern/i`). Here are some examples: - Use `\.internal@domain\.com$` to mark email addresses ending with @@ -384,6 +391,16 @@ with the permissions described on the documentation on [auditor users permission [Read more about Auditor users.](../administration/auditor_users.md) +## Users with minimal access **(PREMIUM ONLY)** + +>[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. + +Administrators can add members with a "minimal access" role to a parent group. Such users don't +automatically have access to projects and subgroups underneath. To support such access, administrators must explicitly add these "minimal access" users to the specific subgroups/projects. + +Users with minimal access can list the group in the UI and through the API. However, they cannot see +details such as projects or subgroups. They do not have access to the group's page or list any of its subgroups or projects. + ## Project features Project features like wiki and issues can be hidden from users depending on @@ -426,7 +443,7 @@ instance and project. In addition, all admins can use the admin interface under 1. Only if the job was: - Triggered by the user - - [Since GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/35069), not run for a protected branch + - [In GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/35069) and later, not run for a protected branch ### Job permissions @@ -473,7 +490,7 @@ for details about the pipelines security model. ## LDAP users permissions -Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. +In GitLab 8.15 and later, LDAP user permissions can now be manually overridden by an admin user. Read through the documentation on [LDAP users permissions](group/index.md#manage-group-memberships-via-ldap) to learn more. ## Project aliases diff --git a/doc/user/profile/account/create_accounts.md b/doc/user/profile/account/create_accounts.md index 26c2c1bed89..09bfa7afc9e 100644 --- a/doc/user/profile/account/create_accounts.md +++ b/doc/user/profile/account/create_accounts.md @@ -31,9 +31,9 @@ You can also [create users through the API](../../../api/users.md) as an admin.  -## Create users through integrations +## Create users through authentication integrations Users will be: -- Automatically created upon first login with the [LDAP integration](../../../administration/auth/ldap/index.md). -- Created when first logging in via an [OmniAuth provider](../../../integration/omniauth.md) if the `allow_single_sign_on` setting is present. +- Automatically created upon first sign in with the [LDAP integration](../../../administration/auth/ldap/index.md). +- Created when first signing in via an [OmniAuth provider](../../../integration/omniauth.md) if the `allow_single_sign_on` setting is present. diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 894494da513..0400d9ca2e5 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -51,7 +51,14 @@ To access your profile settings: From there, you can: -- Update your personal information +- Update your personal information, including: + - Full name + - Primary email, public email, and commit email + - Social media handles + - Website URL + - Location + - Job title + - Bio - Change your [password](#changing-your-password) - Set a [custom status](#current-status) for your profile - Manage your [commit email](#commit-email) for your profile diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 1b8c16f401c..911be117716 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -14,7 +14,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w If you're unable to use [OAuth2](../../api/oauth2.md), you can use a personal access token to authenticate with the [GitLab API](../../api/README.md#personalproject-access-tokens). -You can also use personal access tokens with Git to authenticate over HTTP or SSH. Personal access tokens are required when [Two-Factor Authentication (2FA)](../account/two_factor_authentication.md) is enabled. In both cases, you can authenticate with a token in place of your password. +You can also use personal access tokens with Git to authenticate over HTTP or SSH. Personal access tokens are required when [Two-Factor Authentication (2FA)](account/two_factor_authentication.md) is enabled. In both cases, you can authenticate with a token in place of your password. Personal access tokens expire on the date you define, at midnight UTC. @@ -71,7 +71,7 @@ the following table. You can programmatically create a predetermined personal access token for use in automation or tests. You need sufficient access to run a -[Rails console session](../../administration/troubleshooting/debug.md#starting-a-rails-console-session) +[Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session) for your GitLab instance. To create a token belonging to a user with username `automation-bot`, run the @@ -101,7 +101,7 @@ The list of valid scopes and what they do can be found ## Programmatically revoking a personal access token You can programmatically revoke a personal access token. You need -sufficient access to run a [Rails console session](../../administration/troubleshooting/debug.md#starting-a-rails-console-session) +sufficient access to run a [Rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session) for your GitLab instance. To revoke a known token `token-string-here123`, run the following in the Rails diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index f84fc1ae898..61944bb9d0b 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -84,7 +84,7 @@ The default syntax theme is White, and you can choose among 5 different themes: [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in 13.0, the theme you choose also applies to the [Web IDE](../project/web_ide/index.md)'s code editor and [Snippets](../snippets.md). The themes are available only in the Web IDE file editor, except for the [dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/209808) and -the [solarized dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/219228), +the [Solarized dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/219228), which apply to the entire Web IDE screen. ## Behavior @@ -131,15 +131,9 @@ You can choose between 2 options: ### Project overview content -The project overview content setting allows you to choose what content you want to +The **Project overview content** setting allows you to choose what content you want to see on a project’s home page. -You can choose between 3 options: - -- Files and Readme (default) -- Readme -- Activity - ### Tab width You can set the displayed width of tab characters across various parts of diff --git a/doc/user/project/autocomplete_characters.md b/doc/user/project/autocomplete_characters.md index 1acdc56de54..ff9cb1c712e 100644 --- a/doc/user/project/autocomplete_characters.md +++ b/doc/user/project/autocomplete_characters.md @@ -35,6 +35,8 @@ Autocomplete characters are useful when combined with [Quick Actions](quick_acti Assume your GitLab instance includes the following users: +<!-- vale gitlab.Spelling = NO --> + | Username | Name | | :-------------- | :--- | | alessandra | Rosy Grant | @@ -43,6 +45,8 @@ Assume your GitLab instance includes the following users: | logan_gutkowski | Lee Wuckert | | shelba | Josefine Haley | +<!-- vale gitlab.Spelling = YES --> + In an Issue comment, entering `@l` results in the following popup list appearing. Note that user `shelba` is not included, because the list includes only the 5 users most relevant to the Issue. diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md index ed6e2460554..fd0287fb5fb 100644 --- a/doc/user/project/badges.md +++ b/doc/user/project/badges.md @@ -10,7 +10,7 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41174) in GitLab 10.7. Badges are a unified way to present condensed pieces of information about your -projects. They consist of a small image and additionally a URL that the image +projects. They consist of a small image and a URL that the image points to. Examples for badges can be the [pipeline status](../../ci/pipelines/settings.md#pipeline-status-badge), [test coverage](../../ci/pipelines/settings.md#test-coverage-report-badge), or ways to contact the project maintainers. diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index b3b1b51a543..65416d73f06 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -43,7 +43,7 @@ For example, the following policy document allows assuming a role whose name sta Generate an access key for the IAM user, and configure GitLab with the credentials: -1. Navigate to **Admin Area > Settings > Integrations** and expand the **Amazon EKS** section. +1. Navigate to **Admin Area > Settings > General** and expand the **Amazon EKS** section. 1. Check **Enable Amazon EKS integration**. 1. Enter the account ID and access key credentials into the respective `Account ID`, `Access key ID` and `Secret access key` fields. @@ -61,22 +61,13 @@ To create and add a new Kubernetes cluster to your project, group, or instance: - **Admin Area > Kubernetes**, for an instance-level cluster. 1. Click **Add Kubernetes cluster**. 1. Under the **Create new cluster** tab, click **Amazon EKS**. You will be provided with an - `Account ID` and `External ID` to use in the next step. -1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an EKS management IAM role. - To do so, follow the [Amazon EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html) instructions - to create a IAM role suitable for managing the AWS EKS cluster's resources on your behalf. - In addition to the policies that guide suggests, you must also include the `AmazonEKSClusterPolicy` - policy for this role in order for GitLab to manage the EKS cluster correctly. -1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an IAM role: - 1. From the left panel, select **Roles**. - 1. Click **Create role**. - 1. Under `Select type of trusted entity`, select **Another AWS account**. - 1. Enter the Account ID from GitLab into the `Account ID` field. - 1. Check **Require external ID**. - 1. Enter the External ID from GitLab into the `External ID` field. - 1. Click **Next: Permissions**. - 1. Click **Create Policy**, which will open a new window. - 1. Select the **JSON** tab, and paste in the following snippet in place of the existing content: + `Account ID` and `External ID` needed for later steps. +1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an IAM policy: + 1. From the left panel, select **Policies**. + 1. Click **Create Policy**, which opens a new window. + 1. Select the **JSON** tab, and paste the following snippet in place of the + existing content. These permissions give GitLab the ability to create + resources, but not delete them: ```json { @@ -123,15 +114,26 @@ To create and add a new Kubernetes cluster to your project, group, or instance: } ``` - NOTE: **Note:** - These permissions give GitLab the ability to create resources, but not delete them. - This means that if an error is encountered during the creation process, changes will + If an error is encountered during the creation process, changes will not be rolled back and you must remove resources manually. You can do this by deleting the relevant [CloudFormation stack](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-delete-stack.html) 1. Click **Review policy**. 1. Enter a suitable name for this policy, and click **Create Policy**. You can now close this window. - 1. Switch back to the "Create role" window, and select the policy you just created. + +1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an EKS management IAM role. + To do so, follow the [Amazon EKS cluster IAM role](https://docs.aws.amazon.com/eks/latest/userguide/service_IAM_role.html) instructions + to create a IAM role suitable for managing the AWS EKS cluster's resources on your behalf. + In addition to the policies that guide suggests, you must also include the `AmazonEKSClusterPolicy` + policy for this role in order for GitLab to manage the EKS cluster correctly. +1. In the [IAM Management Console](https://console.aws.amazon.com/iam/home), create an IAM role: + 1. From the left panel, select **Roles**. + 1. Click **Create role**. + 1. Under `Select type of trusted entity`, select **Another AWS account**. + 1. Enter the Account ID from GitLab into the `Account ID` field. + 1. Check **Require external ID**. + 1. Enter the External ID from GitLab into the `External ID` field. + 1. Click **Next: Permissions**, and select the policy you just created. 1. Click **Next: Tags**, and optionally enter any tags you wish to associate with this role. 1. Click **Next: Review**. 1. Enter a role name and optional description into the fields provided. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index 18d9fa67ee1..094f4bcf6ba 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -19,9 +19,12 @@ and learn how to spin up a Kubernetes cluster managed by Google Cloud Platform ( in a few clicks. TIP: **Tip:** -Every new Google Cloud Platform (GCP) account receives [$300 in credit upon sign up](https://console.cloud.google.com/freetrial), -and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's -Google Kubernetes Engine Integration. All you have to do is [follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) and apply for credit. +Every new Google Cloud Platform (GCP) account receives +[$300 in credit upon sign up](https://console.cloud.google.com/freetrial). +In partnership with Google, GitLab is able to offer an additional $200 for new GCP +accounts to get started with GitLab's Google Kubernetes Engine Integration. +[Follow this link](https://cloud.google.com/partners/partnercredit/?pcn_code=0014M00001h35gDQAQ#contact-form) +to apply for credit. ## Before you begin @@ -30,7 +33,7 @@ Before [adding a Kubernetes cluster](#create-new-cluster) using GitLab, you need - GitLab itself. Either: - A [GitLab.com account](https://about.gitlab.com/pricing/#gitlab-com). - A [self-managed installation](https://about.gitlab.com/pricing/#self-managed) with GitLab version - 12.5 or later. This will ensure the GitLab UI can be used for cluster creation. + 12.5 or later. This ensures the GitLab UI can be used for cluster creation. - The following GitLab access: - [Maintainer access to a project](../../permissions.md#project-members-permissions) for a project-level cluster. @@ -41,28 +44,25 @@ Before [adding a Kubernetes cluster](#create-new-cluster) using GitLab, you need ## Access controls -When creating a cluster in GitLab, you will be asked if you would like to create either: +> - Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. -- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) cluster. -- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster. +When creating a cluster in GitLab, you are asked if you would like to create either: -NOTE: **Note:** -[RBAC](#rbac-cluster-resources) is recommended and the GitLab default. +- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) + cluster, which is the GitLab default and recommended option. +- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster. GitLab creates the necessary service accounts and privileges to install and run [GitLab managed applications](index.md#installing-applications). When GitLab creates the cluster, a `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace to manage the newly created cluster. -NOTE: **Note:** -Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5. - The first time you install an application into your cluster, the `tiller` service account is created with `cluster-admin` privileges in the -`gitlab-managed-apps` namespace. This service account will be used by Helm to +`gitlab-managed-apps` namespace. This service account is used by Helm to install and run [GitLab managed applications](index.md#installing-applications). -Helm will also create additional service accounts and other resources for each +Helm also creates additional service accounts and other resources for each installed application. Consult the documentation of the Helm charts for each application for details. @@ -77,7 +77,7 @@ Note the following about access controls: - Environment-specific resources are only created if your cluster is [managed by GitLab](index.md#gitlab-managed-clusters). -- If your cluster was created before GitLab 12.2, it will use a single namespace for all project +- If your cluster was created before GitLab 12.2, it uses a single namespace for all project environments. ### RBAC cluster resources @@ -151,11 +151,12 @@ Amazon Elastic Kubernetes Service (EKS) at the project, group, or instance level ## Add existing cluster -If you have an existing Kubernetes cluster, you can add it to a project, group, or instance. +If you have an existing Kubernetes cluster, you can add it to a project, group, +or instance. -NOTE: **Note:** -Kubernetes integration is not supported for arm64 clusters. See the issue -[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838) for details. +Kubernetes integration isn't supported for arm64 clusters. See the issue +[Helm Tiller fails to install on arm64 cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838) +for details. ### Existing Kubernetes cluster @@ -181,7 +182,7 @@ To add a Kubernetes cluster to your project, group, or instance: kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}' ``` - 1. **CA certificate** (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We will use the certificate created by default. + 1. **CA certificate** (required) - A valid Kubernetes certificate is needed to authenticate to the cluster. We use the certificate created by default. 1. List the secrets with `kubectl get secrets`, and one should be named similar to `default-token-xxxxx`. Copy that token name for use below. 1. Get the certificate by running this command: @@ -190,9 +191,20 @@ To add a Kubernetes cluster to your project, group, or instance: kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode ``` - NOTE: **Note:** - If the command returns the entire certificate chain, you need copy the *root ca* - certificate at the bottom of the chain. + If the command returns the entire certificate chain, you must copy the Root CA + certificate and any intermediate certificates at the bottom of the chain. + A chain file has following structure: + + ```plaintext + -----BEGIN MY CERTIFICATE----- + -----END MY CERTIFICATE----- + -----BEGIN INTERMEDIATE CERTIFICATE----- + -----END INTERMEDIATE CERTIFICATE----- + -----BEGIN INTERMEDIATE CERTIFICATE----- + -----END INTERMEDIATE CERTIFICATE----- + -----BEGIN ROOT CERTIFICATE----- + -----END ROOT CERTIFICATE----- + ``` 1. **Token** - GitLab authenticates against Kubernetes using service tokens, which are @@ -229,10 +241,10 @@ To add a Kubernetes cluster to your project, group, or instance: kubectl apply -f gitlab-admin-service-account.yaml ``` - You will need the `container.clusterRoleBindings.create` permission + You need the `container.clusterRoleBindings.create` permission to create cluster-level roles. If you do not have this permission, you can alternatively enable Basic Authentication and then run the - `kubectl apply` command as an admin: + `kubectl apply` command as an administrator: ```shell kubectl apply -f gitlab-admin-service-account.yaml --username=admin --password=<password> @@ -257,7 +269,7 @@ To add a Kubernetes cluster to your project, group, or instance: Copy the `<authentication_token>` value from the output: - ```yaml + ```plaintext Name: gitlab-token-b5zv4 Namespace: kube-system Labels: <none> @@ -274,7 +286,7 @@ To add a Kubernetes cluster to your project, group, or instance: ``` NOTE: **Note:** - For GKE clusters, you will need the + For GKE clusters, you need the `container.clusterRoleBindings.create` permission to create a cluster role binding. You can follow the [Google Cloud documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access) @@ -283,7 +295,7 @@ To add a Kubernetes cluster to your project, group, or instance: 1. **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](index.md#gitlab-managed-clusters) for more information. 1. **Project namespace** (optional) - You don't have to fill it in; by leaving - it blank, GitLab will create one for you. Also: + it blank, GitLab creates one for you. Also: - Each project should have a unique namespace. - The project namespace is not necessarily the namespace of the secret, if you're using a secret with broader permissions, like the secret from `default`. @@ -294,21 +306,21 @@ To add a Kubernetes cluster to your project, group, or instance: 1. Finally, click the **Create Kubernetes cluster** button. -After a couple of minutes, your cluster will be ready to go. You can now proceed +After a couple of minutes, your cluster is ready. You can now proceed to install some [pre-defined applications](index.md#installing-applications). #### Disable Role-Based Access Control (RBAC) (optional) When connecting a cluster via GitLab integration, you may specify whether the -cluster is RBAC-enabled or not. This will affect how GitLab interacts with the +cluster is RBAC-enabled or not. This affects how GitLab interacts with the cluster for certain operations. If you did *not* check the **RBAC-enabled cluster** -checkbox at creation time, GitLab will assume RBAC is disabled for your cluster +checkbox at creation time, GitLab assumes RBAC is disabled for your cluster when interacting with it. If so, you must disable RBAC on your cluster for the integration to work properly. - + -NOTE: **Note:** +CAUTION: **Caution:** Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](index.md#security-implications), and may not be desirable. @@ -356,3 +368,12 @@ When removing the cluster integration, note: To learn more on automatically deploying your applications, read about [Auto DevOps](../../../topics/autodevops/index.md). + +## Troubleshooting + +### There was a problem authenticating with your cluster. Please ensure your CA Certificate and Token are valid + +If you encounter this error while adding a Kubernetes cluster, ensure you're +properly pasting the service token. Some shells may add a line break to the +service token, making it invalid. Ensure that there are no line breaks by +pasting your token into an editor and removing any additional spaces. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 8d188f00ceb..459ba144186 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -12,8 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/39840) in > GitLab 11.11 for [instances](../../instance/clusters/index.md). -## Overview - Using the GitLab project Kubernetes integration, you can: - Use [Review Apps](../../../ci/review_apps/index.md). @@ -31,6 +29,11 @@ Besides integration at the project level, Kubernetes clusters can also be integrated at the [group level](../../group/clusters/index.md) or [GitLab instance level](../../instance/clusters/index.md). +To view your project level Kubernetes clusters, navigate to **Operations > Kubernetes** +from your project. On this page, you can [add a new cluster](#adding-and-removing-clusters) +and view information about your existing clusters, such as nodes count and rough estimates +of memory and CPU usage. + ## Setting up ### Supported cluster versions @@ -49,10 +52,9 @@ Currently, GitLab supports the following Kubernetes versions: - 1.17 - 1.16 - 1.15 -- 1.14 +- 1.14 (deprecated, support ends on December 22, 2020) - 1.13 (deprecated, support ends on November 22, 2020) -NOTE: **Note:** Some GitLab features may support versions outside the range provided here. ### Adding and removing clusters @@ -192,7 +194,6 @@ To clear the cache: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24580) in GitLab 11.8. -NOTE: **Note:** You do not need to specify a base domain on cluster settings when using GitLab Serverless. The domain in that case will be specified as part of the Knative installation. See [Installing Applications](#installing-applications). @@ -220,13 +221,11 @@ Auto DevOps automatically detects, builds, tests, deploys, and monitors your applications. To make full use of Auto DevOps (Auto Deploy, Auto Review Apps, and -Auto Monitoring) you will need the Kubernetes project integration enabled. +Auto Monitoring) you will need the Kubernetes project integration enabled, but +Kubernetes clusters can be used without Auto DevOps. [Read more about Auto DevOps](../../../topics/autodevops/index.md) -NOTE: **Note:** -Kubernetes clusters can be used without Auto DevOps. - ## Deploying to a Kubernetes cluster A Kubernetes cluster can be the destination for a deployment job. If @@ -249,36 +248,51 @@ GitLab CI/CD build environment. | Variable | Description | | -------- | ----------- | | `KUBE_URL` | Equal to the API URL. | -| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](add_remove_clusters.md#access-controls). | -| `KUBE_NAMESPACE` | The namespace associated with the project's deployment service account. In the format `<project_name>-<project_id>-<environment>`. For GitLab-managed clusters, a matching namespace is automatically created by GitLab in the cluster. | +| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](add_remove_clusters.md#access-controls). Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main service account of the cluster integration. | +| `KUBE_NAMESPACE` | The namespace associated with the project's deployment service account. In the format `<project_name>-<project_id>-<environment>`. For GitLab-managed clusters, a matching namespace is automatically created by GitLab in the cluster. If your cluster was created before GitLab 12.2, the default `KUBE_NAMESPACE` is set to `<project_name>-<project_id>`. | | `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. | | `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. | -| `KUBECONFIG` | Path to a file containing `kubeconfig` for this deployment. CA bundle would be embedded if specified. This config also embeds the same token defined in `KUBE_TOKEN` so you likely will only need this variable. This variable name is also automatically picked up by `kubectl` so you won't actually need to reference it explicitly if using `kubectl`. | +| `KUBECONFIG` | Path to a file containing `kubeconfig` for this deployment. CA bundle would be embedded if specified. This configuration also embeds the same token defined in `KUBE_TOKEN` so you likely will only need this variable. This variable name is also automatically picked up by `kubectl` so you won't actually need to reference it explicitly if using `kubectl`. | | `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, this variable can be used to set a domain per cluster. See [cluster domains](#base-domain) for more information. | -NOTE: **Note:** -Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main -service account of the cluster integration. - -NOTE: **Note:** -If your cluster was created before GitLab 12.2, default `KUBE_NAMESPACE` will be set to `<project_name>-<project_id>`. - ### Custom namespace -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. - -The Kubernetes integration defaults to project-environment-specific namespaces -of the form `<project_name>-<project_id>-<environment>` (see [Deployment -variables](#deployment-variables)). - -For **non**-GitLab-managed clusters, the namespace can be customized using -[`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) -in `.gitlab-ci.yml`. - -NOTE: **Note:** -When using a [GitLab-managed cluster](#gitlab-managed-clusters), the -namespaces are created automatically prior to deployment and [can not be -customized](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. +> - An option to use project-wide namespaces [was added](https://gitlab.com/gitlab-org/gitlab/-/issues/38054) in GitLab 13.5. + +The Kubernetes integration provides a `KUBECONFIG` with an auto-generated namespace +to deployment jobs. It defaults to using project-environment specific namespaces +of the form `<prefix>-<environment>`, where `<prefix>` is of the form +`<project_name>-<project_id>`. To learn more, read [Deployment variables](#deployment-variables). + +You can customize the deployment namespace in a few ways: + +- You can choose between a **namespace per [environment](../../../ci/environments/index.md)** + or a **namespace per project**. A namespace per environment is the default and recommended + setting, as it prevents the mixing of resources between production and non-production environments. +- When using a project-level cluster, you can additionally customize the namespace prefix. + When using namespace-per-environment, the deployment namespace is `<prefix>-<environment>`, + but otherwise just `<prefix>`. +- For **non-managed** clusters, the auto-generated namespace is set in the `KUBECONFIG`, + but the user is responsible for ensuring its existence. You can fully customize + this value using + [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) + in `.gitlab-ci.yml`. + +When you customize the namespace, existing environments remain linked to their current +namespaces until you [clear the cluster cache](#clearing-the-cluster-cache). + +CAUTION: **Warning:** +By default, anyone who can create a deployment job can access any CI variable within +an environment's deployment job. This includes `KUBECONFIG`, which gives access to +any secret available to the associated service account in your cluster. +To keep your production credentials safe, consider using +[Protected Environments](../../../ci/environments/protected_environments.md), +combined with either + +- a GitLab-managed cluster and namespace per environment, +- *or*, an environment-scoped cluster per protected environment. The same cluster + can be added multiple times with multiple restricted service accounts. ### Integrations diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index afb6d016f45..2e224208eb8 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -28,7 +28,6 @@ above the log file data, depending on your configuration: <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> To learn more about the Log Explorer, see [APM - Log Explorer](https://www.youtube.com/watch?v=hWclZHA7Dgw). -NOTE: **Note:** [Learn more about Kubernetes + GitLab](https://about.gitlab.com/solutions/kubernetes/). Everything you need to build, test, deploy, and run your application at scale. diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 360b02efb69..c1e4e821efd 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -115,9 +115,7 @@ the components outlined above and the pre-loaded demo runbook. VARIABLE_VALUE = project.variables.get('PRIVATE_TOKEN').value ``` -1. To configure the operation of a runbook, create and configure variables: - - NOTE: **Note:** +1. To configure the operation of a runbook, create and configure variables. For this example, we are using the **Run SQL queries in Notebook** section in the sample runbook to query a PostgreSQL database. The first four lines of the following code block define the variables that are required for this query to function: diff --git a/doc/user/project/clusters/securing.md b/doc/user/project/clusters/securing.md index a15660051f7..bed01ff4d58 100644 --- a/doc/user/project/clusters/securing.md +++ b/doc/user/project/clusters/securing.md @@ -85,7 +85,7 @@ Host Security (Falco) are deployed with this model. To deploy GitLab Managed Apps to your cluster, you must first [add your cluster](add_remove_clusters.md) -to GitLab. Then [install](../../clusters/applications.md#installing-applications) +to GitLab. Then [install](../../clusters/applications.md#install-with-one-click) the Web Application Firewall from the project or group Kubernetes page. Note that your project doesn't have to be hosted or deployed through GitLab. You can manage a diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 543ffdbce8f..db91f78fc20 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -136,8 +136,8 @@ This example code does the following: In order to interact with your AWS account, the GitLab CI/CD pipelines require both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to be defined in your GitLab settings under **Settings > CI/CD > Variables**. For more information please see [Create a custom variable in the UI](../../../../ci/variables/README.md#create-a-custom-variable-in-the-ui). -NOTE: **Note:** - The AWS credentials you provide must include IAM policies that provision correct access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. + The AWS credentials you provide must include IAM policies that provision correct + access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. #### Deploying your function @@ -154,9 +154,7 @@ endpoints: #### Manually testing your function Running the following `curl` command should trigger your function. - -NOTE: **Note:** -Your URL should be the one retrieved from the GitLab deploy stage log. +Your URL should be the one retrieved from the GitLab deploy stage log: ```shell curl https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello @@ -222,7 +220,8 @@ the environment of the deployed function: ```yaml provider: - ... + # Other configuration omitted + # ... environment: A_VARIABLE: ${env:A_VARIABLE} ``` @@ -245,10 +244,10 @@ functions: hello: handler: src/handler.hello events: - - http: # Rewrite this part to enable CORS + - http: # Rewrite this part to enable CORS path: hello method: get - cors: true # <-- CORS here + cors: true # <-- CORS here ``` You also need to return CORS specific headers in your function response: @@ -378,7 +377,6 @@ To set these: `AWS_SECRET_ACCESS_KEY`. 1. Mask the credentials so they do not show in logs using the **Masked** toggle. -NOTE: **Note:** The AWS credentials you provide must include IAM policies that provision correct access control to AWS Lambda, API Gateway, CloudFormation, and IAM resources. diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 1157c2c5632..603c4bd73b1 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -75,8 +75,8 @@ To run Knative on GitLab, you will need: ## Installing Knative via GitLab's Kubernetes integration -NOTE: **Note:** -The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB memory. **RBAC must be enabled.** +The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.50 GB +memory. **RBAC must be enabled.** 1. [Add a Kubernetes cluster](../add_remove_clusters.md). 1. Select the **Applications** tab and scroll down to the Knative app section. Enter the domain to be used with @@ -99,22 +99,19 @@ The minimum recommended cluster size to run Knative is 3-nodes, 6 vCPUs, and 22.  -NOTE: **Note:** You can deploy either [functions](#deploying-functions) or [serverless applications](#deploying-serverless-applications) -on a given project but not both. The current implementation makes use of a `serverless.yml` file to signal a FaaS project. +on a given project, but not both. The current implementation makes use of a +`serverless.yml` file to signal a FaaS project. ## Using an existing installation of Knative > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58941) in GitLab 12.0. -NOTE: **Note:** -The "invocations" monitoring feature of GitLab serverless will not work when +The _invocations_ monitoring feature of GitLab serverless won't work when adding an existing installation of Knative. -It is also possible to use GitLab Serverless with an existing Kubernetes -cluster which already has Knative installed. - -You must do the following: +It's also possible to use GitLab Serverless with an existing Kubernetes cluster +which already has Knative installed. You must do the following: 1. Follow the steps to [add an existing Kubernetes @@ -453,16 +450,16 @@ To run a function locally: > Introduced in GitLab 11.5. +12345678901234567890123456789012345678901234567890123456789012345678901234567890 Serverless applications are an alternative to [serverless functions](#deploying-functions). -They are useful in scenarios where an existing runtime does not meet the needs of an application, -such as one written in a language that has no runtime available. Note though that serverless -applications should be stateless! - -NOTE: **Note:** -You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) to get started. +They're useful in scenarios where an existing runtime does not meet the needs of +an application, such as one written in a language that has no runtime available. +Note though that serverless applications should be stateless. -Add the following `.gitlab-ci.yml` to the root of your repository -(you may skip this step if you've previously cloned the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) mentioned above): +You can reference and import the sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app) +to get started. Add the following `.gitlab-ci.yml` to the root of your repository +(you may skip this step if you've previously cloned the previously mentioned, +sample [Knative Ruby App](https://gitlab.com/knative-examples/knative-ruby-app)): ```yaml include: @@ -561,14 +558,18 @@ Or: ## Enabling TLS for Knative services -By default, a GitLab serverless deployment will be served over `http`. In order to serve over `https` you -must manually obtain and install TLS certificates. +By default, a GitLab serverless deployment will be served over `http`. To serve +over `https`, you must manually obtain and install TLS certificates. -The simplest way to accomplish this is to -use [Certbot to manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS. +12345678901234567890123456789012345678901234567890123456789012345678901234567890 +The simplest way to accomplish this is to use Certbot to +[manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). +Certbot is a free, open source software tool for automatically using Let’s Encrypt +certificates on manually-administered websites to enable HTTPS. -NOTE: **Note:** -The instructions below relate to installing and running Certbot on a Linux server that has Python 3 installed and may not work on other operating systems or with other versions of Python. +The following instructions relate to installing and running Certbot on a Linux +server that has Python 3 installed, and may not work on other operating systems +or with other versions of Python. 1. Install Certbot by running the [`certbot-auto` wrapper script](https://certbot.eff.org/docs/install.html#certbot-auto). @@ -788,7 +789,7 @@ The instructions below relate to installing and running Certbot on a Linux serve kubectl edit gateway knative-ingress-gateway --namespace knative-serving ``` - Update the gateway to include the following tls: section and configuration: + Update the gateway to include the following `tls:` section and configuration: ```shell tls: diff --git a/doc/user/project/code_intelligence.md b/doc/user/project/code_intelligence.md index f56673e69b7..d0c5a24826a 100644 --- a/doc/user/project/code_intelligence.md +++ b/doc/user/project/code_intelligence.md @@ -58,9 +58,9 @@ relevant language. | Language | Implementation | |---|---| -| Go | [sourcegraph/lsif-go](https://github.com/sourcegraph/lsif-go) | -| JavaScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | -| TypeScript | [sourcegraph/lsif-node](https://github.com/sourcegraph/lsif-node) | +| Go | [`sourcegraph/lsif-go`](https://github.com/sourcegraph/lsif-go) | +| JavaScript | [`sourcegraph/lsif-node`](https://github.com/sourcegraph/lsif-node) | +| TypeScript | [`sourcegraph/lsif-node`](https://github.com/sourcegraph/lsif-node) | View a complete list of [available LSIF indexers](https://lsif.dev/#implementations-server) on their website and refer to their documentation to see how to generate an LSIF file for your specific language. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 730a9ada428..4ae3d5ec032 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -75,7 +75,6 @@ be used for merge request approvals: - As [merge request eligible approvers](merge_requests/merge_request_approvals.md#code-owners-as-eligible-approvers). - As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners). **(PREMIUM)** -NOTE: **Note:** Developer or higher [permissions](../permissions.md) are required in order to approve a merge request. @@ -93,12 +92,14 @@ to specify the actual owners and granular permissions. Using Code Owners in conjunction with [Protected Branches](protected_branches.md#protected-branches-approval-by-code-owners) will prevent any user who is not specified in the `CODEOWNERS` file from pushing -changes for the specified files/paths, even if their role is included in the +changes for the specified files/paths, except those included in the **Allowed to push** column. This allows for a more inclusive push strategy, as administrators don't have to restrict developers from pushing directly to the protected branch, but can restrict pushing to certain files where a review by Code Owners is required. +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5, users and groups who are allowed to push to protected branches do not require a merge request to merge their feature branches. Thus, they can skip merge request approval rules, Code Owners included. + ## The syntax of Code Owners files Files can be specified using the same kind of patterns you would use @@ -158,7 +159,7 @@ file.md @group-x @group-x/subgroup-y ### Code Owners Sections **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.2 behind a feature flag, enabled by default. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. Code Owner rules can be grouped into named sections. This allows for better organization of broader categories of Code Owner rules to be applied. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 8146f39ef87..3c6494d5f1a 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -85,7 +85,7 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ [`kubernetes`](https://docs.gitlab.com/runner/executors/kubernetes.html) executor. 1. Configure the [Kubernetes integration](clusters/index.md) in your project for the cluster. The Kubernetes namespace is of particular note as you will need it - for your deployment scripts (exposed by the `KUBE_NAMESPACE` env variable). + for your deployment scripts (exposed by the `KUBE_NAMESPACE` environment variable). 1. Ensure Kubernetes annotations of `app.gitlab.com/env: $CI_ENVIRONMENT_SLUG` and `app.gitlab.com/app: $CI_PROJECT_PATH_SLUG` are applied to the deployments, replica sets, and pods, where `$CI_ENVIRONMENT_SLUG` and @@ -106,7 +106,7 @@ To display the Deploy Boards for a specific [environment](../../ci/environments/ be done automatically and no action is necessary. If you are using GCP to manage clusters, you can see the deployment details in GCP itself by going to **Workloads > deployment name > Details**: - +  Once all of the above are set up and the pipeline has run at least once, diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index 81c9008c5b3..4f344554016 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -12,7 +12,7 @@ more repositories, by importing an SSH public key to your GitLab instance. This is useful for cloning repositories to your Continuous Integration (CI) server. By using deploy keys, you don't have to set up a -dummy user account. +fake user account. There are two types of deploy keys: diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index aa5987bf5f9..e0c4097d1c5 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -49,8 +49,8 @@ To create a Markdown file: 1. Click the `+` button next to `master` and click **New file**. 1. Add the name of your issue template to the **File name** text field next to `master`. - Make sure words are separated with underscores and that your file has the `.md` extension, for - example `feature_request.md`. + Make sure that your file has the `.md` extension, for + example `feature_request.md` or `Feature Request.md`. 1. Commit and push to your default branch. If you don't have a `.gitlab/issue_templates` directory in your repository, you'll need to create it. @@ -79,6 +79,9 @@ This will enable the `Bug` dropdown option when creating or editing issues. When to the issue description field. The 'Reset template' button will discard any changes you made after picking the template and return it to its initial status. +TIP: **Tip:** +You can create short-cut links to create an issue using a designated template. For example: `https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20proposal`. +  ## Setting a default template for merge requests and issues **(STARTER)** diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 6fd33901621..46c2e211d57 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -69,7 +69,7 @@ brew install git-lfs ``` Once installed, **open your local repository in a terminal window** and -install Git LFS in your repo. If you're sure that LFS is already installed, +install Git LFS in your repository. If you're sure that LFS is already installed, you can skip this step. If you're unsure, re-installing it won't do any harm: ```shell @@ -159,7 +159,7 @@ command line interface, file locks can be created for any file. ### View exclusively-locked files To list all the files locked with LFS locally, open a terminal window in your -repo and run: +repository and run: ```shell git lfs locks @@ -189,7 +189,7 @@ Suggested workflow for shared projects: 1. Lock the file. 1. Edit the file. 1. Commit your changes. -1. Push to the repo. +1. Push to the repository. 1. Get your changes reviewed, approved, and merged. 1. Unlock the file. diff --git a/doc/user/project/img/issue_board_default_lists_v13_4.png b/doc/user/project/img/issue_board_default_lists_v13_4.png Binary files differnew file mode 100644 index 00000000000..23cdc9b4e22 --- /dev/null +++ b/doc/user/project/img/issue_board_default_lists_v13_4.png diff --git a/doc/user/project/img/issue_board_welcome_message.png b/doc/user/project/img/issue_board_welcome_message.png Binary files differdeleted file mode 100644 index 357dff42488..00000000000 --- a/doc/user/project/img/issue_board_welcome_message.png +++ /dev/null diff --git a/doc/user/project/img/labels_key_value_v12_1.png b/doc/user/project/img/labels_key_value_v12_1.png Binary files differdeleted file mode 100644 index ccda944a647..00000000000 --- a/doc/user/project/img/labels_key_value_v12_1.png +++ /dev/null diff --git a/doc/user/project/img/labels_key_value_v13_5.png b/doc/user/project/img/labels_key_value_v13_5.png Binary files differnew file mode 100644 index 00000000000..4264eb3211e --- /dev/null +++ b/doc/user/project/img/labels_key_value_v13_5.png diff --git a/doc/user/project/img/labels_prioritized_v12_1.png b/doc/user/project/img/labels_prioritized_v12_1.png Binary files differdeleted file mode 100644 index 512c5d59a5a..00000000000 --- a/doc/user/project/img/labels_prioritized_v12_1.png +++ /dev/null diff --git a/doc/user/project/img/labels_prioritized_v13_5.png b/doc/user/project/img/labels_prioritized_v13_5.png Binary files differnew file mode 100644 index 00000000000..04ffd67a59f --- /dev/null +++ b/doc/user/project/img/labels_prioritized_v13_5.png diff --git a/doc/user/project/img/labels_subscriptions_v12_1.png b/doc/user/project/img/labels_subscriptions_v12_1.png Binary files differdeleted file mode 100644 index fa83b7db414..00000000000 --- a/doc/user/project/img/labels_subscriptions_v12_1.png +++ /dev/null diff --git a/doc/user/project/img/labels_subscriptions_v13_5.png b/doc/user/project/img/labels_subscriptions_v13_5.png Binary files differnew file mode 100644 index 00000000000..a2a4e9e50cc --- /dev/null +++ b/doc/user/project/img/labels_subscriptions_v13_5.png diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index 89130d5822f..56266718d12 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -76,6 +76,3 @@ If you've accidentally started the import process with the wrong account, follow 1. Revoke GitLab access to your Bitbucket account, essentially reversing the process in the following procedure: [Import your Bitbucket repositories](#import-your-bitbucket-repositories). 1. Sign out of the Bitbucket account. Follow the procedure linked from the previous step. - -NOTE: **Note:** -To import a repository including LFS objects from a Bitbucket server repository, use the [Repo by URL](../import/repo_by_url.md) importer. diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index d0499730bfe..ac5be2b46a4 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -37,12 +37,7 @@ Import your projects from Bitbucket Server to GitLab with minimal effort. empty changes. 1. Attachments in Markdown are currently not imported. 1. Task lists are not imported. -1. Emoji reactions are not imported. -1. [LFS objects](../../../topics/git/lfs/index.md) are not imported. - - NOTE: **Note:** - To import a repository including LFS objects from a Bitbucket server repository, use the [Repo by URL](../import/repo_by_url.md) importer. - +1. Emoji reactions are not imported 1. Project filtering does not support fuzzy search (only `starts with` or `full match strings` are currently supported) @@ -69,20 +64,43 @@ namespace that started the import process. #### User assignment by username -Alternatively, user assignment by username is available behind a `bitbucket_server_user_mapping_by_username` feature flag. -The importer will try to find a user in the GitLab user database using author's `username` or `slug` or `displayName`. -Falls back to author's `email` if user is not found by username. -Similarly to user assignment by email, if no such user is available, the project creator is set as the author. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218609) in GitLab 13.4. +> - It's [deployed behind a feature flag](../../feature_flags.md), disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to enable it. + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +If you've enabled this feature, the importer tries to find a user in the GitLab user database with +the author's: + +- `username` +- `slug` +- `displayName` + +If the user is not found by any of these properties, the search falls back to the author's +`email` address. -To enable or disable user assignment by username: +Alternatively, if there is also no email address, the project creator is set as the author. -Start a [Rails console](../../../administration/troubleshooting/debug.md#starting-a-rails-console-session). +##### Enable or disable User assignment by username + +User assignment by username is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: ```ruby -# Enable Feature.enable(:bitbucket_server_user_mapping_by_username) +``` -# Disable +To disable it: + +```ruby Feature.disable(:bitbucket_server_user_mapping_by_username) ``` diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index f21ec26bdef..2d0caa7d46e 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -96,7 +96,7 @@ back to both GitLab and GitHub when completed. The mirroring is pull-only by default, so you may create or update the file on GitHub: -  +  1. Once your file has been committed, a new pipeline will be automatically triggered if your file is valid: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 4cd0c9e02c7..6c0105aaded 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -35,25 +35,20 @@ The namespace is a user or group in GitLab, such as `gitlab.com/janedoe` or `git This process does not migrate or import any types of groups or organizations from GitHub to GitLab. -### If you're using GitLab.com +### Use cases -If you're using GitLab.com, you can alternatively import -GitHub repositories using a [personal access token](#using-a-github-token), -but we don't recommend this method because it can't associate all user activity -(such as issues and pull requests) with matching GitLab users. +The steps you take depend on whether you are importing from GitHub.com or GitHub Enterprise, as well as whether you are importing to GitLab.com or self-managed GitLab instance. -### If you're importing from GitLab Enterprise - -If you're importing from GitHub Enterprise, you must enable [GitHub integration][gh-import]. - -### If you're using a self-managed GitLab instance - -If you're an administrator of a self-managed GitLab instance, you must enable -[GitHub integration][gh-import]. - -If you're an administrator of a self-managed GitLab instance, you can also use the -[GitHub Rake task](../../../administration/raketasks/github_import.md) to import projects from -GitHub without the constraints of a Sidekiq worker. +- If you're importing to GitLab.com, you can alternatively import GitHub repositories + using a [personal access token](#using-a-github-token). We do not recommend + this method, as it does not associate all user activity (such as issues and + pull requests) with matching GitLab users. +- If you're importing to a self-managed GitLab instance, you can alternatively use the + [GitHub Rake task](../../../administration/raketasks/github_import.md) to import + projects without the constraints of a [Sidekiq](../../../development/sidekiq_style_guide.md) worker. +- If you're importing from GitHub Enterprise to your self-managed GitLab instance, you must first enable + [GitHub integration](../../../integration/github.md). However, you cannot import projects from GitHub Enterprise to GitLab.com. +- If you're importing from GitHub.com to your self-managed GitLab instance, you do not need to set up GitHub integration. ## How it works @@ -106,7 +101,7 @@ If you are using a self-managed GitLab instance or if you are importing from Git 1. From the top navigation bar, click **+** and select **New project**. 1. Select the **Import project** tab and then select **GitHub**. 1. Select the first button to **List your GitHub repositories**. You are redirected to a page on [GitHub](https://github.com) to authorize the GitLab application. -1. Click **Authorize gitlabhq**. You are redirected back to GitLab's Import page and all of your GitHub repositories are listed. +1. Click **Authorize GitlabHQ**. You are redirected back to GitLab's Import page and all of your GitHub repositories are listed. 1. Continue on to [selecting which repositories to import](#selecting-which-repositories-to-import). ### Using a GitHub token @@ -124,7 +119,7 @@ If you are not using the GitHub integration, you can still perform an authorizat 1. Go to <https://github.com/settings/tokens/new> 1. Enter a token description. -1. Select the repo scope. +1. Select the repository scope. 1. Click **Generate token**. 1. Copy the token hash. 1. Go back to GitLab and provide the token to the GitHub importer. @@ -141,10 +136,10 @@ your GitHub repositories are listed. 1. Select the **Import** button next to any number of repositories, or select **Import all repositories**. Additionally, you can filter projects by name. If filter is applied, **Import all repositories** only imports matched repositories. 1. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will - update in realtime or you can return to it later. + update in real-time or you can return to it later. 1. Once a repository has been imported, click its GitLab path to open its GitLab URL. - + ## Mirroring and pipeline status sharing @@ -154,7 +149,7 @@ your imported repository in sync with its GitHub copy. Additionally, you can configure GitLab to send pipeline status updates back GitHub with the [GitHub Project Integration](../integrations/github.md). **(PREMIUM)** -If you import your project using [CI/CD for external repo](../../../ci/ci_cd_for_external_repos/index.md), then both +If you import your project using [CI/CD for external repository](../../../ci/ci_cd_for_external_repos/index.md), then both of the above are automatically configured. **(PREMIUM)** ## Improving the speed of imports on self-managed instances diff --git a/doc/user/project/import/img/manifest_status_v13_3.png b/doc/user/project/import/img/manifest_status_v13_3.png Binary files differindex 3f0063e6715..c1a55ba1f50 100644 --- a/doc/user/project/import/img/manifest_status_v13_3.png +++ b/doc/user/project/import/img/manifest_status_v13_3.png diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 86b671c8371..a1c28cfa2b7 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -18,7 +18,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w 1. [From Perforce](perforce.md) 1. [From SVN](svn.md) 1. [From TFVC](tfvc.md) -1. [From repo by URL](repo_by_url.md) +1. [From repository by URL](repo_by_url.md) 1. [By uploading a manifest file (AOSP)](manifest.md) 1. [From Gemnasium](gemnasium.md) 1. [From Phabricator](phabricator.md) @@ -32,7 +32,7 @@ There is also the option of [connecting your external repository to get CI/CD be ## Migrating from self-managed GitLab to GitLab.com -If you only need to migrate Git repos, you can [import each project by URL](repo_by_url.md). Issues and merge requests can't be imported. +If you only need to migrate Git repositories, you can [import each project by URL](repo_by_url.md). Issues and merge requests can't be imported. If you want to retain all metadata like issues and merge requests, you can use the [import/export feature](../settings/import_export.md) to export projects from self-managed GitLab and import those projects into GitLab.com. diff --git a/doc/user/project/import/manifest.md b/doc/user/project/import/manifest.md index 60524f3cc69..ba1e2011d08 100644 --- a/doc/user/project/import/manifest.md +++ b/doc/user/project/import/manifest.md @@ -56,7 +56,7 @@ You can start the import with: 1. From your GitLab dashboard click **New project** 1. Switch to the **Import project** tab 1. Click on the **Manifest file** button -1. Provide GitLab with a manifest xml file +1. Provide GitLab with a manifest XML file 1. Select a group you want to import to (you need to create a group first if you don't have one) 1. Click **List available repositories**. At this point, you will be redirected to the import status page with projects list based on the manifest file. diff --git a/doc/user/project/import/perforce.md b/doc/user/project/import/perforce.md index dbc1c491493..4ccc34efe30 100644 --- a/doc/user/project/import/perforce.md +++ b/doc/user/project/import/perforce.md @@ -20,7 +20,7 @@ Git: it creates an integration record in their proprietary database for every file in the branch, regardless how many were actually changed. Whereas Git was implemented with a different architecture so that a single SHA acts as a pointer - to the state of the whole repo after the changes, making it very easy to branch. + to the state of the whole repository after the changes, making it very easy to branch. This is what made feature branching workflows so easy to adopt with Git. 1. Also, context switching between branches is much easier in Git. If your manager said 'You need to stop work on that new feature and fix this security diff --git a/doc/user/project/import/repo_by_url.md b/doc/user/project/import/repo_by_url.md index 9b5e43aae79..5c53b6eaf06 100644 --- a/doc/user/project/import/repo_by_url.md +++ b/doc/user/project/import/repo_by_url.md @@ -5,7 +5,7 @@ group: Import info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Import project from repo by URL +# Import project from repository by URL You can import your existing repositories by providing the Git URL: @@ -16,4 +16,4 @@ You can import your existing repositories by providing the Git URL: 1. Click **Create project** to begin the import process 1. Once complete, you will be redirected to your newly created project - + diff --git a/doc/user/project/index.md b/doc/user/project/index.md index c79f2be1d3f..a00f93bac9c 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -34,7 +34,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Protected tags](protected_tags.md): Control over who has permission to create tags, and prevent accidental update or deletion - [Repository mirroring](repository/repository_mirroring.md) - - [Signing commits](gpg_signed_commits/index.md): use GPG to sign your commits + - [Signing commits](repository/gpg_signed_commits/index.md): use GPG to sign your commits - [Deploy tokens](deploy_tokens/index.md): Manage project-based deploy tokens that allow permanent access to the repository and Container Registry. - [Web IDE](web_ide/index.md) - [CVE ID Requests](../application_security/cve_id_request.md): Request a CVE identifier to track a @@ -96,9 +96,9 @@ When you create a project in GitLab, you'll have access to a large number of - [Wiki](wiki/index.md): document your GitLab project in an integrated Wiki. - [Snippets](../snippets.md): store, share and collaborate on code snippets. -- [Value Stream Analytics](cycle_analytics.md): review your development lifecycle. +- [Value Stream Analytics](../analytics/value_stream_analytics.md): review your development lifecycle. - [Insights](insights/index.md): configure the Insights that matter for your projects. **(ULTIMATE)** -- [Security Dashboard](security_dashboard.md): Security Dashboard. **(ULTIMATE)** +- [Security Dashboard](../application_security/security_dashboard/index.md): Security Dashboard. **(ULTIMATE)** - [Syntax highlighting](highlighting.md): an alternative to customize your code blocks, overriding GitLab's default choice of language. - [Badges](badges.md): badges for the project overview. diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index 443ca11be27..bb4a5b2b97f 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -14,8 +14,8 @@ See the project homepage for further information: <https://gitlab.com/esr/irker> ## Needed setup -You will first need an Irker daemon. You can download the Irker code from its -repository on <https://gitlab.com/esr/irker>: +You will first need an Irker daemon. You can download the Irker code +[from its repository](https://gitlab.com/esr/irker): ```shell git clone https://gitlab.com/esr/irker.git @@ -55,6 +55,6 @@ case, `Aorimn` is treated as a nick and no more as a channel name. Irker can also join password-protected channels. Users need to append `?key=thesecretpassword` to the channel name. When using this feature remember to **not** put the `#` sign in front of the channel name; failing to do so will -result on irker joining a channel literally named `#chan?key=password` henceforth +result on Irker joining a channel literally named `#chan?key=password` henceforth leaking the channel key through the `/whois` IRC command (depending on IRC server -configuration). This is due to a long standing irker bug. +configuration). This is due to a long standing Irker bug. diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 3e0b6492477..b4e02bbd5f3 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -23,7 +23,7 @@ Features include: - **View a list of Jira issues directly in GitLab** **(PREMIUM)** For additional features, you can install the -[Jira Development Panel integration](../../../integration/jira_development_panel.md) **(PREMIUM)**. +[Jira Development Panel integration](../../../integration/jira_development_panel.md). This enables you to: - In a Jira issue, display relevant GitLab information in the [development panel](https://support.atlassian.com/jira-software-cloud/docs/view-development-information-for-an-issue/), including related branches, commits, and merge requests. @@ -122,6 +122,8 @@ By now you should have [configured Jira](#configuring-jira) and enabled the you should be able to reference and close Jira issues by just mentioning their ID in GitLab commits and merge requests. +Jira issue IDs must be formatted in uppercase for the integration to work. + ### Reference Jira issues When GitLab project has Jira issue tracker configured and enabled, mentioning diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 7a1f757c138..a502dfbf320 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -50,7 +50,7 @@ Click on the service links to see further configuration instructions and details | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | No | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | No | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | No | -| Packagist | Update your project on Packagist, the main Composer repository | Yes | +| Packagist | Update your projects on Packagist, the main Composer repository | Yes | | Pipelines emails | Email the pipeline status to a list of recipients | No | | [Slack Notifications](slack.md) | Send GitLab events (for example, an issue was created) to Slack as notifications | No | | [Slack slash commands](slack_slash_commands.md) **(CORE ONLY)** | Use slash commands in Slack to control GitLab | No | diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index a19b819c823..28a9afa5bb0 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -58,6 +58,43 @@ CPU and Memory consumption is monitored, but requires [naming conventions](prome The [NGINX Ingress](../clusters/index.md#installing-applications) that is deployed by GitLab to clusters, is automatically annotated for monitoring providing key response metrics: latency, throughput, and error rates. +##### Example of Kubernetes service annotations and labels + +As an example, to activate Prometheus monitoring of a service: + +1. Add at least this annotation: `prometheus.io/scrape: 'true'`. +1. Add two labels so GitLab can retrieve metrics dynamically for any environment: + - `application: ${CI_ENVIRONMENT_SLUG}` + - `release: ${CI_ENVIRONMENT_SLUG}` +1. Create a dynamic PromQL query. For example, a query like + `temperature{application="{{ci_environment_slug}}",release="{{ci_environment_slug}}"}` to either: + - Add [custom metrics](../../../operations/metrics/index.md#adding-custom-metrics). + - Add [custom dashboards](../../../operations/metrics/dashboards/index.md). + +The following is a service definition to accomplish this: + +```yaml +--- +# Service +apiVersion: v1 +kind: Service +metadata: + name: service-${CI_PROJECT_NAME}-${CI_COMMIT_REF_SLUG} + # === Prometheus annotations === + annotations: + prometheus.io/scrape: 'true' + labels: + application: ${CI_ENVIRONMENT_SLUG} + release: ${CI_ENVIRONMENT_SLUG} + # === End of Prometheus === +spec: + selector: + app: ${CI_PROJECT_NAME} + ports: + - port: ${EXPOSED_PORT} + targetPort: ${CONTAINER_PORT} +``` + ### Manual configuration of Prometheus #### Requirements @@ -136,10 +173,7 @@ one of them will be used: > - GitLab 9.3 added the [numeric comparison](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/27439) of the 30 minute averages. Developers can view the performance impact of their changes within the merge -request workflow. - -NOTE: **Note:** -Requires [Kubernetes](prometheus_library/kubernetes.md) metrics. +request workflow. This feature requires [Kubernetes](prometheus_library/kubernetes.md) metrics. When a source branch has been deployed to an environment, a sparkline and numeric comparison of the average memory consumption will appear. On the diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index e278c7eb664..70f8a55bb07 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index 712805b75f2..0fbc49ddad7 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- diff --git a/doc/user/project/integrations/prometheus_library/index.md b/doc/user/project/integrations/prometheus_library/index.md index 6f2c2477eee..35b111ab2b2 100644 --- a/doc/user/project/integrations/prometheus_library/index.md +++ b/doc/user/project/integrations/prometheus_library/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 29efe08e53d..43c2d305437 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -12,7 +12,7 @@ GitLab has support for automatically detecting and monitoring Kubernetes metrics ## Requirements -The [Prometheus](../prometheus.md) and [Kubernetes](../kubernetes.md) +The [Prometheus](../prometheus.md) and [Kubernetes](../../clusters/index.md) integration services must be enabled. ## Metrics supported diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index 0d3042463c9..1757378fb70 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 7bebe7b1e65..fdea800c410 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- @@ -8,11 +8,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22133) in GitLab 11.7. +GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the built-in Prometheus metrics included with Kubernetes NGINX Ingress controller [version 0.16.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0160) onward. + NOTE: **Note:** NGINX Ingress versions prior to 0.16.0 offer an included [VTS Prometheus metrics exporter](nginx_ingress_vts.md), which exports metrics different than the built-in metrics. -GitLab has support for automatically detecting and monitoring the Kubernetes NGINX Ingress controller. This is provided by leveraging the built-in Prometheus metrics included with Kubernetes NGINX Ingress controller [version 0.16.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0160) onward. - ## Requirements [Prometheus integration](../prometheus.md) must be active. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md index 326931e9790..ec7b1ee6d10 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: APM +group: Health info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- diff --git a/doc/user/project/integrations/services_templates.md b/doc/user/project/integrations/services_templates.md index 688643a85a7..abb072c9a0a 100644 --- a/doc/user/project/integrations/services_templates.md +++ b/doc/user/project/integrations/services_templates.md @@ -6,20 +6,52 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Service templates -Using a service template, GitLab administrators can provide default values for configuring integrations at the project level. +Using a service template, GitLab administrators can: -When you enable a service template, the defaults are applied to **all** projects that do not -already have the integration enabled or do not otherwise have custom values saved. -The values are pre-filled on each project's configuration page for the applicable integration. +- Provide default values for configuring integrations when creating new projects. +- Bulk configure all existing projects in one step. -If you disable the template, these values no longer appear as defaults, while -any values already saved for an integration remain unchanged. +When you enable a service template: + +- The defaults are applied to **all** existing projects that either: + - Don't already have the integration enabled. + - Don't have custom values stored for already enabled integrations. +- Values are populated on each project's configuration page for the applicable + integration. +- Settings are stored at the project level. + +If you disable the template: + +- GitLab default values again become the default values for integrations on + new projects. +- Projects previously configured using the template will continue to use + those settings. + +If you change the template, the revised values are applied to new projects. This feature +does not provide central administration of integration settings. + +## Central administration of project integrations + +A new set of features is being introduced in GitLab to provide more control over +how integrations are configured at the instance, group, and project level. + +[Read more about setting up project integration management](../../admin_area/settings/project_integration_management.md) +(introduced in GitLab 13.3) and [our plans for managing integrations](https://gitlab.com/groups/gitlab-org/-/epics/2137). ## Enable a service template Navigate to the **Admin Area > Service Templates** and choose the service template you wish to create. +Recommendation: + +- Test the settings on some projects individually before enabling a template. +- Copy the working settings from a project to the template. + +There is no "Test settings" option when enabling templates. If the settings do not work, +these incorrect settings will be applied to all existing projects that do not already have +the integration configured. Fixing the integration then needs to be done project-by-project. + ## Service for external issue trackers The following image shows an example service template for Redmine. diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 03ff5f845b6..e6f12c2532f 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -64,7 +64,7 @@ The following triggers are available for Slack notifications: - **Tag push**: Triggered when a new tag is pushed to the repository. - **Pipeline**: Triggered when a pipeline status changes. - **Wiki page**: Triggered when a wiki page is created or updated. -- **Deployment**: Triggered when a deployment finishes. +- **Deployment**: Triggered when a deployment starts or finishes. - **Alert**: Triggered when a new, unique alert is recorded. ## Troubleshooting diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 800eb1d3359..7adea5ebcd6 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1303,7 +1303,12 @@ Note that `commit.id` is the ID of the pipeline, not the ID of the commit. ### Deployment events -Triggered when deployment is finished/failed/canceled. +Triggered when a deployment: + +- Starts ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41214) in GitLab 13.5.) +- Succeeds +- Fails +- Is cancelled **Request Header**: diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index f8172a0f988..bce40e9a838 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -8,14 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5554) in [GitLab 8.11](https://about.gitlab.com/releases/2016/08/22/gitlab-8-11-released/#issue-board). -## Overview - The GitLab Issue Board is a software project management tool used to plan, organize, and visualize a workflow for a feature or product release. It can be used as a [Kanban](https://en.wikipedia.org/wiki/Kanban_(development)) or a [Scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)) board. -It pairs issue tracking and project management, keeping everything in the same place, +It pairs issue tracking and project management, keeping everything together, so that you don't need to jump between different platforms to organize your workflow. Issue boards build on the existing [issue tracking functionality](issues/index.md#issues-list) and @@ -26,8 +24,8 @@ Issue boards help you to visualize and manage your entire process in GitLab. You add your labels, and then create the corresponding list for your existing issues. When you're ready, you can drag your issue cards from one step to another one. -An issue board can show you what issues your team is working on, who is assigned to each, -and where in the workflow those issues are. +An issue board can show you the issues your team is working on, who is assigned to each, +and where the issues are in the workflow. To let your team members organize their own workflows, use [multiple issue boards](#use-cases-for-multiple-issue-boards). This allows creating multiple issue @@ -60,8 +58,8 @@ Here are some common use cases for issue boards. ### Use cases for a single issue board -With the GitLab Workflow you can discuss proposals in issues, categorize them -with labels, and from there, organize and prioritize them with issue boards. +With the GitLab Workflow you can discuss proposals in issues, label +them, and organize and prioritize them with issue boards. For example, let's consider this simplified development workflow: @@ -155,7 +153,7 @@ card includes: ## Permissions Users with the [Reporter and higher roles](../permissions.md) can use all the functionality of the -Issue Board feature to create or delete lists and drag issues from one list to another. +Issue Board feature to create or delete lists. They can also drag issues from one list to another. ## How GitLab orders issues in a list @@ -164,20 +162,19 @@ that order by dragging the issues. The changed order is saved, so that anybody w board later sees the reordering, with some exceptions. The first time a given issue appears in any board (that is, the first time a user -loads a board containing that issue), it is ordered in relation to other issues in that list -according to [label priority](labels.md#label-priority). +loads a board containing that issue), it is ordered in relation to other issues in that list. +The order is done according to [label priority](labels.md#label-priority). At this point, that issue is assigned a relative order value by the system, -representing its relative order with respect to the other issues in the list. Any time -you reorder that issue by dragging, its relative order value changes accordingly. +with respect to the other issues in the list. Any time +you drag and reorder the issue, its relative order value changes accordingly. -Also, any time that issue appears in any board when it's loaded by a user, -the updated relative order value is used for the ordering. It's only the first -time an issue appears that it takes from the priority order mentioned above. This means that -if issue `A` is reordered by dragging to be above issue `B` by any user in -a given board inside your GitLab instance, any time those two issues are subsequently -loaded in any board in the same instance (could be a different project board or a different group -board, for example), that ordering is maintained. +Also, any time that issue appears in any board, the ordering is done according to +the updated relative order value. It's only the first +time an issue appears that it takes from the priority order mentioned above. If a user in your GitLab instance +drags issue `A` above issue `B`, the ordering is maintained when these two issues are subsequently +loaded in any board in the same instance. This could be a different project board or a different group +board, for example. This ordering also affects [issue lists](issues/sorting_issue_lists.md). Changing the order in an issue board changes the ordering in an issue list, @@ -195,8 +192,7 @@ advanced functionality is present in [higher tiers only](https://about.gitlab.co > - Multiple issue boards per group are available in [GitLab Premium](https://about.gitlab.com/pricing/). Multiple issue boards allow for more than one issue board for a given project or group. -This is great for large projects with more than one team or in situations where a repository is used -to host the code of multiple products. +This is great for large projects with more than one team or when a repository hosts the code of multiple products. Using the search box at the top of the menu, you can filter the listed boards. @@ -230,13 +226,13 @@ To delete the currently active issue board: An issue board can be associated with a GitLab [Milestone](milestones/index.md#milestones), [Labels](labels.md), Assignee and Weight -which will automatically filter the Board issues according to these fields. +which automatically filter the board issues accordingly. This allows you to create unique boards according to your team's need.  You can define the scope of your board when creating it or by clicking the "Edit board" button. -Once a milestone, assignee or weight is assigned to an issue board, you will no longer be able to +Once a milestone, assignee or weight is assigned to an issue board, you can no longer filter through these in the search bar. In order to do that, you need to remove the desired scope (for example, milestone, assignee, or weight) from the issue board. @@ -274,24 +270,22 @@ especially in combination with [assignee lists](#assignee-lists). ### Group issue boards **(PREMIUM)** -> [Introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. +> - One group issue board per group introduced in GitLab 10.6. +> - Multiple group issue boards [introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. -Accessible at the group navigation level, a group issue board offers the same features as a project-level board, -but it can display issues from all projects in that +Accessible at the group navigation level, a group issue board offers the same features as a project-level board. +It can display issues from all projects in that group and its descendant subgroups. Similarly, you can only filter by group labels for these boards. When updating milestones and labels for an issue through the sidebar update mechanism, again only group-level objects are available. -NOTE: **Note:** -Multiple group issue boards were originally [introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0, and one group issue board per group was made available in GitLab Core 10.6. -  ### Assignee lists **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. -Like in a regular list that shows all issues with a chosen label, you can add +As in a regular list showing all issues with a chosen label, you can add an assignee list that shows all issues assigned to a user. You can have a board with both label lists and assignee lists. To add an assignee list: @@ -317,7 +311,7 @@ milestone, giving you more freedom and visibility on the issue board. To add a m 1. Select the **Milestone** tab. 1. Search and click the milestone. -Similar to the assignee lists, you're now able to [drag issues](#drag-issues-between-lists) +Like the assignee lists, you're able to [drag issues](#drag-issues-between-lists) to and from a milestone list to manipulate the milestone of the dragged issues. As in other list types, click the trash icon to remove a list. @@ -333,7 +327,7 @@ You cannot set a WIP limit on the default lists (**Open** and **Closed**). Examples: -- You have a list with four issues, and a limit of five, the header will show **4/5**. +- When you have a list with four issues and a limit of five, the header shows **4/5**. If you exceed the limit, the current number of issues is shown in red. - You have a list with five issues with a limit of five. When you move another issue to that list, the list's header displays **6/5**, with the six shown in red. @@ -374,29 +368,24 @@ If you're not able to do some of the things above, make sure you have the right ### First time using an issue board -The first time you open an issue board, you are presented with -the default lists (**Open** and **Closed**) and a welcome message that gives -you two options. You can either: +> The automatic creation of the **To Do** and **Doing** lists was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202144) in GitLab 13.5. -- Create a predefined set of labels (by default: **To Do** and **Doing**) and create their - corresponding lists to the issue board. -- Opt-out and use your own lists. +The first time you open an issue board, you are presented with the default lists +(**Open**, **To Do**, **Doing**, and **Closed**). - +If the **To Do** and **Doing** labels don't exist in the project or group, they are created, and +their lists appear as empty. If any of them already exists, the list is filled with the issues that +have that label. -If you choose to use and create the predefined lists, they will appear as empty -because the labels associated to them will not exist up until that moment, -which means the system has no way of populating them automatically. That's of -course if the predefined labels don't already exist. If any of them does exist, -the list will be created and filled with the issues that have that label. + ### Create a new list Create a new list by clicking the **Add list** button in the upper right corner of the issue board. - + -Then, choose the label or user to create the list from. The new list will be inserted +Then, choose the label or user to create the list from. The new list is inserted at the end of the lists, before **Done**. Moving and reordering lists is as easy as dragging them around. @@ -407,15 +396,15 @@ You can now choose it to create a list. ### Delete a list To delete a list from the issue board, use the small trash icon present -in the list's heading. A confirmation dialog will appear for you to confirm. +in the list's heading. A confirmation dialog appears for you to confirm. -Deleting a list doesn't have any effect in issues and labels, it's just the -list view that is removed. You can always add it back later if you need. +Deleting a list doesn't have any effect on issues and labels, as it's just the +list view that's removed. You can always restore it later if you need. ### Add issues to a list You can add issues to a list by clicking the **Add issues** button -present in the upper right corner of the issue board. This will open up a modal +present in the upper right corner of the issue board. This opens up a modal window where you can see all the issues that do not belong to any list. Select one or more issues by clicking the cards and then click **Add issues** @@ -435,7 +424,7 @@ respective label is removed. ### Filter issues You should be able to use the filters on top of your issue board to show only -the results you want. This is similar to the filtering used in the issue tracker +the results you want. It's similar to the filtering used in the issue tracker since the metadata from the issues and labels are re-used in the issue board. You can filter by author, assignee, milestone, and label. @@ -469,12 +458,12 @@ For example, you can create a list based on the label of **Frontend** and one fo worked on by the designers. Then, once they're done, all they have to do is -drag it to the next list, **Backend**, where a backend developer can +drag it to the next list, **Backend**. Then, a backend developer can eventually pick it up. Once they’re done, they move it to **Done**, to close the issue. -This process can be seen clearly when visiting an issue since with every move -to another list the label changes and a system note is recorded. +This process can be seen clearly when visiting an issue. With every move +to another list, the label changes and a system note is recorded.  diff --git a/doc/user/project/issues/crosslinking_issues.md b/doc/user/project/issues/crosslinking_issues.md index c721bef8f4d..2a75f8ad837 100644 --- a/doc/user/project/issues/crosslinking_issues.md +++ b/doc/user/project/issues/crosslinking_issues.md @@ -33,7 +33,7 @@ Of course, you can replace `gitlab.com` with the URL of your own GitLab instance NOTE: **Note:** Linking your first commit to your issue is going to be relevant -for tracking your process with [GitLab Cycle Analytics](https://about.gitlab.com/stages-devops-lifecycle/value-stream-analytics/). +for tracking your process with [GitLab Value Stream Analytics](https://about.gitlab.com/stages-devops-lifecycle/value-stream-analytics/). It will measure the time taken for planning the implementation of that issue, which is the time between creating an issue and making the first commit. diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index 7c9278c8403..6f57487fccd 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -4,14 +4,12 @@ group: Knowledge info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" --- -# Design Management +# Design Management **(CORE)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. > - Support for SVGs was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12771) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.4. > - Design Management was [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212566) to GitLab Core in 13.0. -## Overview - Design Management allows you to upload design assets (wireframes, mockups, etc.) to GitLab issues and keep them stored in one single place, accessed by the Design Management's page within an issue, giving product designers, product managers, and engineers a @@ -58,8 +56,7 @@ Support for [PDF](https://gitlab.com/gitlab-org/gitlab/issues/32811) is planned - From GitLab 13.1, Design filenames are limited to 255 characters. - Design Management data [isn't deleted when a project is destroyed](https://gitlab.com/gitlab-org/gitlab/-/issues/13429) yet. -- Design Management data [won't be moved](https://gitlab.com/gitlab-org/gitlab/-/issues/13426) - when an issue is moved, nor [deleted](https://gitlab.com/gitlab-org/gitlab/-/issues/13427) +- Design Management data [won't be deleted](https://gitlab.com/gitlab-org/gitlab/-/issues/13427) when an issue is deleted. - From GitLab 12.7, Design Management data [can be replicated](../../../administration/geo/replication/datatypes.md#limitations-on-replicationverification) by Geo but [not verified](https://gitlab.com/gitlab-org/gitlab/-/issues/32467). @@ -114,9 +111,6 @@ Designs with the same filename as an existing uploaded design will create a new of the design, and will replace the previous version. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34353) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.9, dropping a design on an existing uploaded design will also create a new version, provided the filenames are the same. -Designs cannot be added if the issue has been moved, or its -[discussion is locked](../../discussions/#lock-discussions). - ### Skipped designs Designs with the same filename as an existing uploaded design _and_ whose content has not changed will be skipped. @@ -185,9 +179,7 @@ viewed by browsing previous versions. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34382) in GitLab 13.3. -You can change the order of designs by dragging them to a new position: - - +You can change the order of designs by dragging them to a new position. ## Starting discussions on designs @@ -231,47 +223,19 @@ Note that your resolved comment pins will disappear from the Design to free up s However, if you need to revisit or find a resolved discussion, all of your resolved threads will be available in the **Resolved Comment** area at the bottom of the right sidebar. -## Add To-Do for Designs +## Add to dos for designs > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198439) in GitLab 13.4. -> - It's [deployed behind a feature flag](../../feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-the-design-to-do-button). **(CORE ONLY)** - -CAUTION: **Warning:** -This feature might not be available to you. Check the **version history** note above for details. - -Add a to-do for a design by clicking **Add a To-Do** on the design sidebar: - - - -### Enable or disable the design To-Do button **(CORE ONLY)** - -The **Add a To-Do** button for Designs is under development but ready for production use. It is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:design_management_todo_button) -``` +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/245074) in GitLab 13.5. -To disable it: +Add a to do for a design by clicking **Add a To Do** on the design sidebar: -```ruby -Feature.disable(:design_management_todo_button) -``` + ## Referring to designs in Markdown > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217160) in **GitLab 13.1**. -> - It is deployed behind a feature flag, disabled by default. -> - It is disabled on GitLab.com. -> - It is not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-design-references). **(CORE ONLY)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/258662) in **GitLab 13.5** We support referring to designs in [Markdown](../../markdown.md), which is available throughout the application, including in merge request and issue descriptions, in discussions and comments, and in wiki pages. @@ -287,25 +251,6 @@ This will be rendered as: > See [#123[homescreen.png]](https://gitlab.com/your-group/your-project/-/issues/123/designs/homescreen.png) -### Enable or disable design references **(CORE ONLY)** - -Design reference parsing is -deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:design_management_reference_filter_gfm_pipeline) -``` - -To re-enable it: - -```ruby -Feature.enable(:design_management_reference_filter_gfm_pipeline) -``` - ## Design activity records > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33051) in GitLab 13.1. diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index 55b45bf9d3d..b3ebefadef0 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -46,7 +46,7 @@ the icon and the date colored red. You can sort issues by those that are Due dates also appear in your [to-do list](../../todos.md). - + The day before an open issue is due, an email will be sent to all participants of the issue. Like the due date, the "day before the due date" is determined by the diff --git a/doc/user/project/issues/img/design_todo_button_v13_4.png b/doc/user/project/issues/img/design_todo_button_v13_4.png Binary files differdeleted file mode 100644 index 62bbecf4ed9..00000000000 --- a/doc/user/project/issues/img/design_todo_button_v13_4.png +++ /dev/null diff --git a/doc/user/project/issues/img/design_todo_button_v13_5.png b/doc/user/project/issues/img/design_todo_button_v13_5.png Binary files differnew file mode 100644 index 00000000000..970161a6097 --- /dev/null +++ b/doc/user/project/issues/img/design_todo_button_v13_5.png diff --git a/doc/user/project/issues/img/designs_reordering_v13_3.gif b/doc/user/project/issues/img/designs_reordering_v13_3.gif Binary files differdeleted file mode 100644 index 496eea532e2..00000000000 --- a/doc/user/project/issues/img/designs_reordering_v13_3.gif +++ /dev/null diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 060266a478f..434af3a4a49 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -156,12 +156,15 @@ collaborate with your team. efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. -### Related issues **(STARTER)** +### Related issues You can mark two issues as related, so that when viewing one, the other is always listed in its [Related Issues](related_issues.md) section. This can help display important context, such as past work, dependencies, or duplicates. +Users on [GitLab Starter, GitLab Bronze, and higher tiers](https://about.gitlab.com/pricing/), can +also mark issues as blocking or blocked by another issue. + ### Crosslinking issues You can [cross-link issues](crosslinking_issues.md) by referencing an issue from another diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 5356e6aeb40..003444e0ed8 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -36,7 +36,7 @@ You can find all the information for that issue on one screen. - **15.** [Edit](#edit) - **16.** [Description](#description) - **17.** [Mentions](#mentions) -- **18.** [Related Issues **(STARTER)**](#related-issues) +- **18.** [Related Issues](#related-issues) - **19.** [Related Merge Requests](#related-merge-requests) - **20.** [Award emoji](#award-emoji) - **21.** [Show all activity](#show-all-activity) @@ -80,7 +80,7 @@ The button to do this has a different label depending on whether the issue is al List or not. If the issue is: - Already on your To-Do List: The button is labeled **Mark as done**. Click the button to remove the issue from your To-Do List. -- Not on your To-Do List: The button is labeled **Add a To Do**. Click the button to add the issue to your To-Do List. +- Not on your To-Do List: The button is labeled **Add a to do**. Click the button to add the issue to your To-Do List. ### Assignee @@ -191,7 +191,7 @@ The plain text title and description of the issue fill the top center of the iss The description fully supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm), allowing many formatting options. -> [Since GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/10103), changes to an issue's description are listed in the [issue history](#issue-history).**(STARTER)** +> [In GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/10103) and later, changes to an issue's description are listed in the [issue history](#issue-history).**(STARTER)** ### Mentions @@ -208,7 +208,7 @@ TIP: **Tip:** Avoid mentioning `@all` in issues and merge requests, as it sends an email notification to all the members of that project's group, which can be interpreted as spam. -### Related Issues **(STARTER)** +### Related Issues Issues that were mentioned as [related issues](related_issues.md) are listed here. You can also click the `+` to add more related issues. @@ -242,7 +242,7 @@ and selecting either: Also: - You can mention a user or a group present in your GitLab instance with - `@username` or `@groupname` and they will be notified via To-Do items + `@username` or `@groupname` and they will be notified via to-do items and email, unless they have [disabled all notifications](#notifications) in their profile settings. - Mentions for yourself (the current logged in user), will be highlighted diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 0e0731528be..b033dc79dcc 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -127,6 +127,7 @@ field). | title | `issue[title]` | | | description | `issue[description]` | | | description template | `issuable_template` | | +| issue type | `issue[issue_type]` | Either `incident` or `issue` | | confidential | `issue[confidential]` | Parameter value must be `true` to set to confidential | Follow these examples to form your new issue URL with prefilled fields. diff --git a/doc/user/project/issues/multiple_assignees_for_issues.md b/doc/user/project/issues/multiple_assignees_for_issues.md index c11977f5bee..b1806460c08 100644 --- a/doc/user/project/issues/multiple_assignees_for_issues.md +++ b/doc/user/project/issues/multiple_assignees_for_issues.md @@ -8,8 +8,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1904) in [GitLab Starter 9.2](https://about.gitlab.com/releases/2017/05/22/gitlab-9-2-released/#multiple-assignees-for-issues). -## Overview - In large teams, where there is shared ownership of an issue, it can be difficult to track who is working on it, who already completed their contributions, who didn't even start yet. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index 954e3771722..b040bcf3b03 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -4,33 +4,40 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- -# Related issues **(STARTER)** +# Related issues **(CORE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1797) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1797) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.4. +> - The simple "relates to" relationship [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212329) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.4. Related issues are a bi-directional relationship between any two issues and appear in a block below the issue description. Issues can be across groups and projects. +You can set any issue as: + +- Related to another issue +- Blocking another issue **(STARTER)** +- Blocked by another issue **(STARTER)** + The relationship only shows up in the UI if the user can see both issues. When you try to close an issue that has open blockers, a warning is displayed. TIP: **Tip:** -To manage related issues through our API, see the [API documentation](../../../api/issue_links.md). +To manage related issues through our API, visit the [issue links API documentation](../../../api/issue_links.md). ## Adding a related issue > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2035) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. > - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/34239) to warn when attempting to close an issue that is blocked by others in [GitLab Starter](https://about.gitlab.com/pricing/) 13.0. -> When you try to close an issue with open blockers, you'll see a warning that you can dismiss. +> When you try to close an issue with open blockers, you see a warning that you can dismiss. 1. Relate one issue to another by clicking the related issues "+" button in the header of the related issue block. 1. Input the issue reference number or paste in the full URL of the issue. -1. Select whether the current issue relates to, blocks, or is blocked by the issues being entered. +1. **(STARTER)** Select whether the current issue relates to, blocks, or is blocked by the issues being entered.  @@ -42,11 +49,11 @@ in the header of the related issue block. - same group: `project#44` - different group: `group/project#44` - Valid references will be added to a temporary list that you can review. + Valid references are added to a temporary list that you can review. 1. When you have added all the related issues, click **Add** to submit. -When you have finished adding all related issues, you will be able to see +When you have finished adding all related issues, you can see them categorized so their relationships can be better understood visually.  @@ -56,7 +63,7 @@ them categorized so their relationships can be better understood visually. In the related issues block, click the "x" icon on the right-side of each issue token that you wish to remove. -Due to the bi-directional relationship, it will no longer appear in either issue. +Due to the bi-directional relationship, it no longer appears in either issue.  diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 0d02b89be7e..4f0354f86af 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Labels -## Overview - As your count of issues, merge requests, and epics grows in GitLab, it's more and more challenging to keep track of those items. Especially as your organization grows from just a few people to hundreds or thousands. This is where labels come in. They help you organize and tag your work @@ -32,18 +30,25 @@ There are two types of labels in GitLab: ## Assign and unassign labels -Every issue, merge request and epic can be assigned any number of labels. The labels are +> Unassigning labels with the **X** button [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216881) in GitLab 13.5. + +Every issue, merge request, and epic can be assigned any number of labels. The labels are managed in the right sidebar, where you can assign or unassign labels as needed. -To assign a label to an issue, merge request or epic: +To assign or unassign a label: -1. In the label section of the sidebar, click **Edit**, then: - - In the list, click the labels you want. Each label is flagged with a checkmark. - - Find labels by entering a search query and clicking search (**{search}**), then - click on them. You can search repeatedly and add more labels. -1. Click **X** or anywhere outside the label section and the labels are applied. +1. In the **Labels** section of the sidebar, click **Edit**. +1. In the **Assign labels** list, search for labels by typing their names. + You can search repeatedly to add more labels. + The selected labels are marked with a checkmark. +1. Click the labels you want to assign or unassign. +1. To apply your changes to labels, click **X** next to **Assign labels** or anywhere outside the + label section. -You can also assign a label with the [`/label ~label1 ~label2` quick action](quick_actions.md). +Alternatively, to unassign a label, click the **X** on the label you want to unassign. + +You can also assign a label with the `/label` [quick action](quick_actions.md), +remove labels with `/unlabel`, and reassign labels (remove all and assign new ones) with `/relabel`. ## Label management @@ -52,10 +57,13 @@ and edit labels. ### Project labels -View the project labels list by going to the project and clicking **Issues > Labels**. +> Showing all inherited labels [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241990) in 13.5. + +To view the project labels list, navigate to the project and click **Issues > Labels**. The list includes all labels that are defined at the project level, as well as all -labels inherited from the immediate parent group. You can filter the list by entering a search -query at the top and clicking search (**{search}**). +labels defined by its ancestor groups. +For each label, you can see the project or group path from where it was created. +You can filter the list by entering a search query at the top and clicking search (**{search}**). To create a new project label: @@ -83,19 +91,19 @@ a label by clicking the three dots (**{ellipsis_v}**) next to the **Subscribe** and selecting **Delete**. CAUTION: **Caution:** -If you delete a label, it is permanently deleted. You will not be able to undo the deletion, and all references to the label will be removed from the system. +If you delete a label, it is permanently deleted. All references to the label are removed from the system and you cannot undo the deletion. #### Promote a project label to a group label If you previously created a project label and now want to make it available for other projects within the same group, you can promote it to a group label. -If other projects in the same group have a label with the same title, they will all be -merged with the new group label. If a group label with the same title exists, it will -also be merged. +If other projects in the same group have a label with the same title, they are all +merged with the new group label. If a group label with the same title exists, it is +also merged. All issues, merge requests, issue board lists, issue board filters, and label subscriptions -with the old labels will be assigned to the new group label. +with the old labels are assigned to the new group label. CAUTION: **Caution:** Promoting a label is a permanent action, and cannot be reversed. @@ -108,7 +116,7 @@ To promote a project label to a group label: ### Group labels -View the group labels list by going to the group and clicking **Issues > Labels**. +To view the group labels list, navigate to the group and click **Issues > Labels**. The list includes all labels that are defined at the group level only. It does not list any labels that are defined in projects. You can filter the list by entering a search query at the top and clicking search (**{search}**). @@ -118,15 +126,15 @@ follow the same process as [creating a project label](#project-labels). #### Create group labels from epics **(ULTIMATE)** -You can create group labels from the Epic sidebar. The labels you create will +You can create group labels from the epic sidebar. The labels you create belong to the immediate group to which the epic belongs. The process is the same as creating a [project label from an issue or merge request](#project-labels). ### Generate default labels If a project or group has no labels, you can generate a default set of project or group -labels from the label list page. The page will show a **Generate a default set of labels** -button if the list is empty, and clicking it will add the following default labels +labels from the label list page. The page shows a **Generate a default set of labels** +button if the list is empty. Select the button to add the following default labels to the project: - `bug` @@ -149,13 +157,13 @@ by preventing certain labels from being used together. A label is scoped when it uses a special double-colon (`::`) syntax in the label’s title, for example: - + An issue, merge request or epic cannot have two scoped labels, of the form `key::value`, -with the same `key`. Adding a new label with the same `key`, but a different `value` will -cause the previous `key` label to be replaced with the new label. +with the same `key`. Adding a new label with the same `key`, but a different `value` +causes the previous `key` label to be replaced with the new label. -Example use case: +For example: 1. An issue is identified as being low priority, and a `priority::low` project label is added to it. @@ -188,7 +196,7 @@ This functionality is demonstrated in a video regarding ### Scoped labels with nested scopes You can create a label with a nested scope by using multiple double colons `::` when creating -it. In this case, everything before the last `::` will be the scope. +it. In this case, everything before the last `::` is the scope. For example, `workflow::backend::review` and `workflow::backend::development` are valid scoped labels, but they **can't** exist on the same issue at the same time, as they @@ -202,13 +210,13 @@ both have different scopes, `workflow::frontend` and `workflow::backend`. From the project label list page and the group label list page, you can click **Subscribe** to the right of any label to enable [notifications](../profile/notifications.md) for that -label. You will be notified whenever the label is assigned to an epic, +label. You are notified whenever the label is assigned to an epic, issue, or merge request. If you are subscribing to a group label from within a project, you can select to subscribe to label notifications for the project only, or the whole group. - + ## Label priority @@ -222,7 +230,7 @@ from the group label list. From the project label list page, star a label to indicate that it has a priority. - + Drag starred labels up and down the list to change their priority, where higher in the list means higher priority. diff --git a/doc/user/project/merge_requests/accessibility_testing.md b/doc/user/project/merge_requests/accessibility_testing.md index f3a0aac9ff4..a07a155745e 100644 --- a/doc/user/project/merge_requests/accessibility_testing.md +++ b/doc/user/project/merge_requests/accessibility_testing.md @@ -55,7 +55,7 @@ include: ``` creates an `a11y` job in your CI/CD pipeline, runs -Pa11y against the webpages defined in `a11y_urls`, and builds an HTML report for each. +Pa11y against the web pages defined in `a11y_urls`, and builds an HTML report for each. The report for each URL is saved as an artifact that can be [viewed directly in your browser](../../../ci/pipelines/job_artifacts.md#browsing-artifacts). diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md index 60d247ccc19..4ba0b50a3cf 100644 --- a/doc/user/project/merge_requests/allow_collaboration.md +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -28,11 +28,12 @@ source project and only lasts while the merge request is open. Once enabled, upstream members will also be able to retry the pipelines and jobs of the merge request: -1. Enable the contribution while creating or editing a merge request. +1. While creating or editing a merge request, select the checkbox **Allow + commits from members who can merge to the target branch**.  -1. Once the merge request is created, you'll see that commits from members who +1. Once the merge request is created, you can see that commits from members who can merge to the target branch are allowed.  diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index a0be32e0708..462b8f68ece 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -111,6 +111,48 @@ It is also possible to manage multiple assignees: - When creating a merge request. - Using [quick actions](../quick_actions.md#quick-actions-for-issues-merge-requests-and-epics). +## Reviewer + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216054) in GitLab 13.5. +> - It's [deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-merge-request-reviewers). **(CORE ONLY)** + +CAUTION: **Warning:** +This feature might not be available to you. Check the **version history** note above for details. + +Requesting a code review is an important part of contributing code. However, deciding who should review +your code and asking for a review are no easy tasks. Using the "assignee" field for both authors and +reviewers makes it hard for others to determine who's doing what on a merge request. + +GitLab's Merge Request Reviewers easily allow authors to request a review as well as see the status of the +review. By selecting one or more users from the **Reviewers** field in the merge request's right-hand +sidebar, the assigned reviewers will receive a notification of the request to review the merge request. + +This makes it easy to determine the relevant roles for the users involved in the merge request, as well as formally requesting a review from a peer. + +To request it, open the **Reviewers** drop-down box to search for the user you wish to get a review from. + +### Enable or disable Merge Request Reviewers **(CORE ONLY)** + +Merge Request Reviewers is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it. + +To enable it: + +```ruby +Feature.enable(:merge_request_reviewers) +``` + +To disable it: + +```ruby +Feature.disable(:merge_request_reviewers) +``` + ### Merge requests to close issues If the merge request is being created to resolve an issue, you can diff --git a/doc/user/project/merge_requests/img/commit_nav_v13_4.png b/doc/user/project/merge_requests/img/commit_nav_v13_4.png Binary files differnew file mode 100644 index 00000000000..0ae6ce32693 --- /dev/null +++ b/doc/user/project/merge_requests/img/commit_nav_v13_4.png diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index daebd71e14f..2675f509eed 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -164,8 +164,8 @@ For example: 1. Capture the dynamic URL and save it into a `.env` file, e.g. `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. 1. Set the `.env` file to be a [job artifact](../../../ci/pipelines/job_artifacts.md#job-artifacts). 1. In the `load_performance` job: - 1. Set it to depend on the review job, so it inherits the env file. - 1. Set the `K6_DOCKER_OPTIONS` variable with the [Docker cli option for env files](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file), for example `--env-file review.env`. + 1. Set it to depend on the review job, so it inherits the environment file. + 1. Set the `K6_DOCKER_OPTIONS` variable with the [Docker CLI option for environment files](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file), for example `--env-file review.env`. 1. Configure the k6 test script to use the environment variable in it's steps. Your `.gitlab-ci.yml` file might be similar to: diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index 185ab0e6298..4b4c930c7af 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -5,13 +5,16 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, concepts --- -# Merge Request Approvals +# Merge Request Approvals **(CORE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/580) in GitLab Enterprise Edition 7.2. Available in GitLab Core and higher tiers. +> - Redesign [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1979) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.8 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/10685) in 12.0. Code review is an essential practice of every successful project, and giving your approval once a merge request is in good shape is an important part of the review process, as it clearly communicates the ability to merge the change. -## Optional Approvals **(CORE ONLY)** +## Optional Approvals > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27426) in GitLab 13.2. @@ -335,26 +338,6 @@ of your security team when a vulnerability would be introduced by a merge reques For more information, see [Security approvals in merge requests](../../application_security/index.md#security-approvals-in-merge-requests). -### Enabling the new approvals interface - -Since [GitLab v12.0](https://gitlab.com/gitlab-org/gitlab/-/issues/10685), an updated approvals -interface is available by default. In versions older than 12.0, the updated interface is not -available unless the `approval_rules` feature flag is enabled, which can be done from -the Rails console by instance administrators. - -Use these commands to start the Rails console: - -```shell -# Omnibus GitLab -gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console -e production -``` - -Then run `Feature.enable(:approval_rules)` to enable the updated interface. - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index bc4fee749e8..6b302b0ff02 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -14,7 +14,7 @@ by clicking the **Revert** button in merge requests and commit details. NOTE: **Note:** The **Revert** button will only be available for merge requests -created since GitLab 8.5. However, you can still revert a merge request +created in GitLab 8.5 and later. However, you can still revert a merge request by reverting the merge commit from the list of Commits page. NOTE: **Note:** diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index 3a18cacde64..aef68e0e771 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -84,7 +84,7 @@ Click **Expand file** on any file to view the changes for that file. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-file-by-file-diff-navigation). For larger merge requests it might sometimes be useful to review single files at a time. To enable, -from your avatar on the top-right navbar, click **Settings**, and go to **Preferences** on the left +from your avatar on the top-right navigation bar, click **Settings**, and go to **Preferences** on the left sidebar. Scroll down to the **Behavior** section and select **Show one file at a time on merge request's Changes tab**. Click **Save changes** to apply. @@ -122,6 +122,8 @@ the commits to open the single-commit view. From there, you can navigate among t by clicking the **Prev** and **Next** buttons on the top-right of the page or by using the <kbd>X</kbd> and <kbd>C</kbd> keyboard shortcuts. + + ### Incrementally expand merge request diffs By default, the diff shows only the parts of a file which are changed. diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index 69a0dd6e84f..68f5478038a 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -65,8 +65,8 @@ meaningful commit messages and: ## Enabling squash for a merge request Anyone who can create or edit a merge request can choose for it to be squashed -on the merge request form. Users can select or unselect the checkbox at the moment -they are creating the merge request: +on the merge request form. Users can select or clear the check box when they +create the merge request:  diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 56b5774f15b..bded1839f97 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -5,13 +5,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, howto --- -# Test Coverage Visualization **(CORE ONLY)** +# Test Coverage Visualization > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - [Feature flag enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/211410) in GitLab 13.4. -> - It's enabled on GitLab.com. -> - It can be disabled per-project. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enabling-the-feature). **(CORE ONLY)** +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249811) in GitLab 13.5. With the help of [GitLab CI/CD](../../../ci/README.md), you can collect the test coverage information of your favorite testing or coverage-analysis tool, and visualize @@ -58,7 +55,9 @@ NOTE: **Note:** The Cobertura XML parser currently does not support the `sources` element and ignores it. It is assumed that the `filename` of a `class` element contains the full path relative to the project root. -## Example test coverage configuration +## Example test coverage configurations + +### JavaScript example The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example uses [Mocha](https://mochajs.org/) JavaScript testing and [NYC](https://github.com/istanbuljs/nyc) coverage-tooling to @@ -74,26 +73,84 @@ test: cobertura: coverage/cobertura-coverage.xml ``` -## Enabling the feature +### Java examples + +#### Maven example -This feature comes with the `:coverage_report_view` feature flag enabled by -default. It is enabled on GitLab.com. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it for your instance. Test coverage visualization can be enabled or disabled per-project. +The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for Java uses [Maven](https://maven.apache.org/) +to build the project and [Jacoco](https://www.eclemma.org/jacoco/) coverage-tooling to +generate the coverage artifact. +You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. -To enable it: +GitLab expects the artifact in the Cobertura format, so you have to execute a few +scripts before uploading it. The `test-jdk11` job tests the code and generates an +XML artifact. The `coverage-jdk-11` job converts the artifact into a Cobertura report: -```ruby -# Instance-wide -Feature.enable(:coverage_report_view) -# or by project -Feature.enable(:coverage_report_view, Project.find(<project id>)) +```yaml +test-jdk11: + stage: test + image: maven:3.6.3-jdk-11 + script: + - 'mvn $MAVEN_CLI_OPTS clean org.jacoco:jacoco-maven-plugin:prepare-agent test jacoco:report' + artifacts: + paths: + - target/site/jacoco/jacoco.xml + +coverage-jdk11: + # Must be in a stage later than test-jdk11's stage. + # The `visualize` stage does not exist by default. + # Please define it first, or chose an existing stage like `deploy`. + stage: visualize + image: haynes/jacoco2cobertura:1.0.3 + script: + # convert report from jacoco to cobertura + - 'python /opt/cover2cover.py target/site/jacoco/jacoco.xml src/main/java > target/site/cobertura.xml' + # read the <source></source> tag and prepend the path to every filename attribute + - 'python /opt/source2filename.py target/site/cobertura.xml' + needs: ["test-jdk11"] + dependencies: + - test-jdk11 + artifacts: + reports: + cobertura: target/site/cobertura.xml ``` -To disable it: +#### Gradle example -```ruby -# Instance-wide -Feature.disable(:coverage_report_view) -# or by project -Feature.disable(:coverage_report_view, Project.find(<project id>)) +The following [`gitlab-ci.yml`](../../../ci/yaml/README.md) example for Java uses [Gradle](https://gradle.org/) +to build the project and [Jacoco](https://www.eclemma.org/jacoco/) coverage-tooling to +generate the coverage artifact. +You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. + +GitLab expects the artifact in the Cobertura format, so you have to execute a few +scripts before uploading it. The `test-jdk11` job tests the code and generates an +XML artifact. The `coverage-jdk-11` job converts the artifact into a Cobertura report: + +```yaml +test-jdk11: + stage: test + image: gradle:6.6.1-jdk11 + script: + - 'gradle test jacocoTestReport' # jacoco must be configured to create an xml report + artifacts: + paths: + - build/jacoco/jacoco.xml + +coverage-jdk11: + # Must be in a stage later than test-jdk11's stage. + # The `visualize` stage does not exist by default. + # Please define it first, or chose an existing stage like `deploy`. + stage: visualize + image: haynes/jacoco2cobertura:1.0.3 + script: + # convert report from jacoco to cobertura + - 'python /opt/cover2cover.py build/jacoco/jacoco.xml src/main/java > build/cobertura.xml' + # read the <source></source> tag and prepend the path to every filename attribute + - 'python /opt/source2filename.py build/cobertura.xml' + needs: ["test-jdk11"] + dependencies: + - test-jdk11 + artifacts: + reports: + cobertura: build/cobertura.xml ``` diff --git a/doc/user/project/merge_requests/work_in_progress_merge_requests.md b/doc/user/project/merge_requests/work_in_progress_merge_requests.md index d1833318a85..e7abf6e5fb6 100644 --- a/doc/user/project/merge_requests/work_in_progress_merge_requests.md +++ b/doc/user/project/merge_requests/work_in_progress_merge_requests.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference, concepts --- -# Draft merge requests +# Draft merge requests **(CORE)** If a merge request is not yet ready to be merged, perhaps due to continued development or open threads, you can prevent it from being accepted before it's ready by flagging @@ -16,10 +16,12 @@ being merged, and it will stay disabled until the "Draft" flag has been removed. ## Adding the "Draft" flag to a merge request -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32692) in GitLab 13.2, Work-In-Progress (WIP) merge requests were renamed to **Draft**. Support for using **WIP** will be removed in GitLab 14.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32692) in GitLab 13.2, Work-In-Progress (WIP) merge requests were renamed to **Draft**. Support for using **WIP** will be removed in GitLab 14.0. +> - **Mark as draft** and **Mark as ready** buttons [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227421) in GitLab 13.5. There are several ways to flag a merge request as a Draft: +- Click the **Mark as draft** button on the top-right corner of the merge request's page. - Add `[Draft]`, `Draft:` or `(Draft)` to the start of the merge request's title. Clicking on **Start the title with Draft:**, under the title box, when editing the merge request's description will have the same effect. @@ -28,15 +30,16 @@ There are several ways to flag a merge request as a Draft: - Add the `/wip` [quick action](../quick_actions.md#quick-actions-for-issues-merge-requests-and-epics) in a comment in the merge request. This is a toggle, and can be repeated to change the status back. Note that any other text in the comment will be discarded. -- Add `draft:` or `Draft:` to the start of a commit message targeting the merge request's - source branch. This is not a toggle, and doing it again in another commit will have - no effect. +- Add `draft:`, `Draft:`, `fixup!`, or `Fixup!` to the beginning of a commit message targeting the + merge request's source branch. This is not a toggle, and doing it again in another + commit will have no effect. ## Removing the "Draft" flag from a merge request Similar to above, when a Merge Request is ready to be merged, you can remove the `Draft` flag in several ways: +- Click the **Mark as ready** button on the top-right corner of the merge request's page. - Remove `[Draft]`, `Draft:` or `(Draft)` from the start of the merge request's title. Clicking on **Remove the Draft: prefix from the title**, under the title box, when editing the merge request's description, will have the same effect. diff --git a/doc/user/project/milestones/burndown_and_burnup_charts.md b/doc/user/project/milestones/burndown_and_burnup_charts.md new file mode 100644 index 00000000000..327a52a05ab --- /dev/null +++ b/doc/user/project/milestones/burndown_and_burnup_charts.md @@ -0,0 +1,142 @@ +--- +type: reference +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Burndown and burnup charts **(STARTER)** + +[Burndown](#burndown-charts) and [burnup](#burnup-charts) charts show the progress of completing a milestone. + + + +## Burndown charts + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1540) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1 for project milestones. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5354) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.8 for group milestones. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6495) to [GitLab Starter](https://about.gitlab.com/pricing/) 11.2 for group milestones. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6903) [fixed burndown charts](#fixed-burndown-charts) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.5. + +Burndown charts show the number of issues over the course of a milestone. + + + +At a glance, you see the current state for the completion a given milestone. +Without them, you would have to organize the data from the milestone and plot it +yourself to have the same sense of progress. + +GitLab plots it for you and presents it in a clear and beautiful chart. + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, check the video demonstration on [Mapping work versus time with burndown charts](https://www.youtube.com/watch?v=zJU2MuRChzs). + +To view a project's burndown chart: + +1. In a project, navigate to **Issues > Milestones**. +1. Select a milestone from the list. + +To view a group's burndown chart: + +1. In a group, navigate to **Issues > Milestones**. +1. Select a milestone from the list. + +### Use cases for burndown charts + +Burndown charts are generally used for tracking and analyzing the completion of +a milestone. Therefore, their use cases are tied to the +[use you are assigning your milestone to](index.md). + +For example, suppose you lead a team of developers in a large company, +and you follow this workflow: + +- Your company set the goal for the quarter to deliver 10 new features for your app + in the upcoming major release. +- You create a milestone, and remind your team to assign that milestone to every new issue + and merge request that's part of the launch of your app. +- Every week, you open the milestone, visualize the progress, identify the gaps, + and help your team to get their work done. +- Every month, you check in with your supervisor, and show the progress of that milestone + from the burndown chart. +- By the end of the quarter, your team successfully delivered 100% of that milestone, as + it was taken care of closely throughout the whole quarter. + +### How burndown charts work + +A burndown chart is available for every project or group milestone that has been attributed a **start +date** and a **due date**. + +NOTE: **Note:** +You're able to [promote project](index.md#promoting-project-milestones-to-group-milestones) to group milestones and still see the **burndown chart** for them, respecting license limitations. + +The chart indicates the project's progress throughout that milestone (for issues assigned to it). + +In particular, it shows how many issues were or are still open for a given day in the +milestone's corresponding period. + +The burndown chart can also be toggled to display the cumulative open issue +weight for a given day. When using this feature, make sure issue weights have +been properly assigned, since an open issue with no weight adds zero to the +cumulative value. + +### Fixed burndown charts + +For milestones created before GitLab 13.5, burndown charts have an additional toggle to +switch between Legacy and Fixed views. + +| Legacy | Fixed | +| ----- | ----- | +|  |  | + +**Fixed burndown** charts track the full history of milestone activity, from its creation until the +milestone expires. After the milestone due date passes, issues removed from the milestone no longer +affect the chart. + +**Legacy burndown** charts track when issues were created and when they were last closed, not their +full history. For each day, a legacy burndown chart takes the number of open issues and the issues +created that day, and subtracts the number of issues closed that day. +Issues that were created and assigned a milestone before its start date (and remain open as of the +start date) are considered as having been opened on the start date. +Therefore, when the milestone start date is changed, the number of opened issues on each day may +change. +Reopened issues are considered as having been opened on the day after they were last closed. + +## Burnup charts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6903) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.5. + +Burnup charts show the assigned and completed work for a milestone. + + + +To view a project's burnup chart: + +1. In a project, navigate to **Issues > Milestones**. +1. Select a milestone from the list. + +To view a group's burnup chart: + +1. In a group, navigate to **Issues > Milestones**. +1. Select a milestone from the list. + +### How burnup charts work + +Burnup charts have separate lines for total work and completed work. The total line +shows when scope is reduced or added to a milestone. The completed work is a count +of issues closed. + +Burnup charts can show either the total number of issues or total weight for each +day of the milestone. Use the toggle above the charts to switch between total +and weight. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/project/milestones/burndown_charts.md b/doc/user/project/milestones/burndown_charts.md index 0c8bba831a9..5aa5534dd37 100644 --- a/doc/user/project/milestones/burndown_charts.md +++ b/doc/user/project/milestones/burndown_charts.md @@ -1,89 +1,5 @@ --- -type: reference -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +redirect_to: './burndown_and_burnup_charts.md' --- -# Burndown Charts **(STARTER)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1540) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1 for project milestones. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5354) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.8 for group milestones. -> - [Added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6495) to [GitLab Starter](https://about.gitlab.com/pricing/) 11.2 for group milestones. -> - Closed or reopened issues prior to GitLab 9.1 won't have a `closed_at` -> value, so the burndown chart considers them as closed on the milestone -> `start_date`. In that case, a warning will be displayed. - -## Overview - -Burndown Charts are visual representations of the progress of completing a milestone. - - - -At a glance, you see the current state for the completion a given milestone. -Without them, you would have to organize the data from the milestone and plot it -yourself to have the same sense of progress. - -GitLab Starter plots it for you and presents it in a clear and beautiful chart. - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview, check the video demonstration on [Mapping work versus time with Burndown Charts](https://www.youtube.com/watch?v=zJU2MuRChzs). - -## Use cases - -Burndown Charts are generally used for tracking and analyzing the completion of -a milestone. Therefore, their use cases are tied to the -[use you are assigning your milestone to](index.md). - -For example, suppose you lead a team of developers in a large company, -and you follow this workflow: - -- Your company set the goal for the quarter to deliver 10 new features for your app - in the upcoming major release. -- You create a milestone, and remind your team to assign that milestone to every new issue - and merge request that's part of the launch of your app. -- Every week, you open the milestone, visualize the progress, identify the gaps, - and help your team to get their work done. -- Every month, you check in with your supervisor, and show the progress of that milestone - from the Burndown Chart. -- By the end of the quarter, your team successfully delivered 100% of that milestone, as - it was taken care of closely throughout the whole quarter. - -## How it works - -A Burndown Chart is available for every project or group milestone that has been attributed a **start -date** and a **due date**. - -Find your project's **Burndown Chart** under **Project > Issues > Milestones**, -and select a milestone from your current ones, while for group's, access the **Groups** dashboard, -select a group, and go through **Issues > Milestones** on the sidebar. - -NOTE: **Note:** -You're able to [promote project](index.md#promoting-project-milestones-to-group-milestones) to group milestones and still see the **Burndown Chart** for them, respecting license limitations. - -The chart indicates the project's progress throughout that milestone (for issues assigned to it). - -In particular, it shows how many issues were or are still open for a given day in the -milestone's corresponding period. - -The Burndown Chart tracks when issues were created and when they were last closed—not their full history. For each day, it takes the number of issues still open and issues created that day and subtracts the number of issues closed that day. -**Issues that were created and assigned a milestone before its start date—and remain open as of the start date—are considered as having been opened on the start date**. Therefore, when the milestone start date is changed the number of opened issues on each day may change. -Reopened issues are -considered as having been opened on the day after they were last closed. - -The Burndown Chart can also be toggled to display the cumulative open issue -weight for a given day. When using this feature, make sure issue weights have -been properly assigned, since an open issue with no weight adds zero to the -cumulative value. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +This document was moved to [another location](./burndown_and_burnup_charts.md). diff --git a/doc/user/project/milestones/img/burndown_and_burnup_charts_v13_5.png b/doc/user/project/milestones/img/burndown_and_burnup_charts_v13_5.png Binary files differnew file mode 100644 index 00000000000..8d6ba1d4fa7 --- /dev/null +++ b/doc/user/project/milestones/img/burndown_and_burnup_charts_v13_5.png diff --git a/doc/user/project/milestones/img/burndown_chart_fixed_v13_5.png b/doc/user/project/milestones/img/burndown_chart_fixed_v13_5.png Binary files differnew file mode 100644 index 00000000000..a532bfeeca0 --- /dev/null +++ b/doc/user/project/milestones/img/burndown_chart_fixed_v13_5.png diff --git a/doc/user/project/milestones/img/burndown_chart_legacy_v13_5.png b/doc/user/project/milestones/img/burndown_chart_legacy_v13_5.png Binary files differnew file mode 100644 index 00000000000..5824fc59ce5 --- /dev/null +++ b/doc/user/project/milestones/img/burndown_chart_legacy_v13_5.png diff --git a/doc/user/project/milestones/img/burndown_chart.png b/doc/user/project/milestones/img/burndown_chart_v13_5.png Binary files differindex e06b24f9907..e06b24f9907 100644 --- a/doc/user/project/milestones/img/burndown_chart.png +++ b/doc/user/project/milestones/img/burndown_chart_v13_5.png diff --git a/doc/user/project/milestones/img/burnup_chart_v13_5.png b/doc/user/project/milestones/img/burnup_chart_v13_5.png Binary files differnew file mode 100644 index 00000000000..a850caba348 --- /dev/null +++ b/doc/user/project/milestones/img/burnup_chart_v13_5.png diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 9d02a22f91e..8cbed3de1c6 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -7,8 +7,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Milestones -## Overview - Milestones in GitLab are a way to track issues and merge requests created to achieve a broader goal in a certain period of time. Milestones allow you to organize issues and merge requests into a cohesive group, with an optional start date and an optional due date. @@ -152,7 +150,7 @@ There are also tabs below these that show the following: For project milestones in [GitLab Starter](https://about.gitlab.com/pricing/), a [burndown chart](burndown_charts.md) is in the milestone view, showing the progress of completing a milestone. - + ### Group Burndown Charts **(STARTER)** diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index c3825371030..a7a72ca4d82 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -223,7 +223,7 @@ This is how an example usage can look like: ```yaml test: script: - - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - docker pull $CI_REGISTRY/group/other-project:latest - docker run $CI_REGISTRY/group/other-project:latest ``` @@ -236,5 +236,4 @@ to projects and their project permissions. ### API -GitLab API cannot be used via `CI_JOB_TOKEN` but there is a [proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/35067) -to support it. +GitLab API can be used via `CI_JOB_TOKEN`, see [the relevant documentation](../../api/README.md#gitlab-ci-job-token). diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md index 735d27ec04d..810538ab460 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md @@ -30,6 +30,8 @@ to do it for you. To help you out, we've gathered some instructions on how to do that for the most popular hosting services: +<!-- vale gitlab.Spelling = NO --> + - [Amazon](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html) - [Bluehost](https://www.bluehost.com/help/article/dns-management-add-edit-or-delete-dns-entries) - [Cloudflare](https://support.cloudflare.com/hc/en-us/articles/201720164-Creating-a-Cloudflare-account-and-adding-a-website) @@ -41,6 +43,8 @@ for the most popular hosting services: - [Media Temple](https://mediatemple.net/community/products/dv/204403794/how-can-i-change-the-dns-records-for-my-domain) - [Microsoft](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-2000-server/bb727018(v=technet.10)) +<!-- vale gitlab.Spelling = YES --> + If your hosting service is not listed above, you can just try to search the web for `how to add dns record on <my hosting service>`. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index badafa478ef..9b43dd58afe 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -61,7 +61,6 @@ according to the type of domain you want to use with your Pages site: - [For subdomains](#for-subdomains), `subdomain.example.com`. - [For both](#for-both-root-and-subdomains). -NOTE: **Note:** You can [configure IPv6 on self-managed instances](../../../../administration/pages/index.md#advanced-configuration), but IPv6 is not currently configured for Pages on GitLab.com. Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214718) for details. @@ -250,8 +249,8 @@ You can use any certificate satisfying the following requirements: - **A private key**, it's an encrypted key which validates your PEM against your domain. -NOTE: **Note:** -[Cloudflare certificates](https://about.gitlab.com/blog/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/), for example, meet these requirements. +For example, [Cloudflare certificates](https://about.gitlab.com/blog/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/) +meet these requirements. #### Steps @@ -269,7 +268,6 @@ NOTE: **Note:** just jumping a line between them. 1. Copy your private key and paste it in the last field. -NOTE: **Note:** **Do not** open certificates or encryption keys in regular text editors. Always use code editors (such as Sublime Text, Atom, Dreamweaver, Brackets, etc). @@ -290,8 +288,8 @@ To enable this setting: 1. Navigate to your project's **Settings > Pages**. 1. Tick the checkbox **Force HTTPS (requires valid certificates)**. -NOTE: **Note:** -If you use Cloudflare CDN in front of GitLab Pages, make sure to set the SSL connection setting to `full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). +If you use Cloudflare CDN in front of GitLab Pages, make sure to set the SSL connection setting to +`full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). <!-- ## Troubleshooting diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 4b4b430b663..24b202dfdbd 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -25,7 +25,7 @@ This feature covers only certificates for **custom domains**, not the wildcard c Before you can enable automatic provisioning of an SSL certificate for your domain, make sure you have: -- Created a [project](../getting_started_part_two.md) in GitLab +- Created a [project](../index.md#getting-started) in GitLab containing your website's source code. - Acquired a domain (`example.com`) and added a [DNS entry](index.md) pointing it to your Pages website. @@ -33,7 +33,6 @@ Before you can enable automatic provisioning of an SSL certificate for your doma and verified your ownership. - Verified your website is up and running, accessible through your custom domain. -NOTE: **Note:** GitLab's Let's Encrypt integration is enabled and available on GitLab.com. For **self-managed** GitLab instances, make sure your administrator has [enabled it](../../../../administration/pages/index.md#lets-encrypt-integration). diff --git a/doc/user/project/pages/getting_started/new_or_existing_website.md b/doc/user/project/pages/getting_started/new_or_existing_website.md index 86f36447b93..f19334a1764 100644 --- a/doc/user/project/pages/getting_started/new_or_existing_website.md +++ b/doc/user/project/pages/getting_started/new_or_existing_website.md @@ -2,4 +2,4 @@ redirect_to: 'pages_ci_cd_template.md' --- -This document was moved to [pages_ci_cd_template.md](pages_ci_cd_template.md). +This document was moved to [another location](pages_ci_cd_template.md). diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md index de9bd97b262..7dc3d2197b5 100644 --- a/doc/user/project/pages/getting_started/pages_forked_sample_project.md +++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md @@ -53,4 +53,4 @@ You can take some **optional** further steps:  - Now go to your SSG's configuration file and change the [base URL](../getting_started_part_one.md#urls-and-baseurls) - from `"project-name"` to `""`. The project name setting varies by SSG and may not be in the config file. + from `"project-name"` to `""`. The project name setting varies by SSG and may not be in the configuration file. diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 9272b1f9093..5ef0c4cc7b9 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -11,12 +11,9 @@ according to your intended website's URL. ## GitLab Pages default domain names -NOTE: **Note:** -If you use your own GitLab instance to deploy your -site with GitLab Pages, check with your sysadmin what's your -Pages wildcard domain. This guide is valid for any GitLab instance, -you just need to replace Pages wildcard domain on GitLab.com -(`*.gitlab.io`) with your own. +If you use your own GitLab instance to deploy your site with GitLab Pages, verify your Pages +wildcard domain with your sysadmin. This guide is valid for any GitLab instance, provided that you +replace the Pages wildcard domain on GitLab.com (`*.gitlab.io`) with your own. If you set up a GitLab Pages project on GitLab, it will automatically be accessible under a diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 9bc9fe97fd3..4bf5300aa13 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1 +1,5 @@ +--- +redirect_to: 'custom_domains_ssl_tls_certification/index.md' +--- + This document was moved to [another location](custom_domains_ssl_tls_certification/index.md). diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 6c3b911d033..4f389716f08 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -124,3 +124,24 @@ If you are running a self-managed instance of GitLab (GitLab Community Edition a [follow the administration steps](../../../administration/pages/index.md) to configure Pages. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a [video tutorial](https://www.youtube.com/watch?v=dD8c7WNcc6s) about how to get started with GitLab Pages administration. + +## Security for GitLab Pages + +If your username is `foo`, your GitLab Pages website is located at `foo.gitlab.io`. +GitLab allows usernames to contain a `.`, so a user named `bar.foo` could create +a GitLab Pages website `bar.foo.gitlab.io` that effectively is a subdomain of your +`foo.gitlab.io` website. Be careful if you use JavaScript to set cookies for your website. +The safe way to manually set cookies with JavaScript is to not specify the `domain` at all: + +```javascript +// Safe: This cookie is only visible to foo.gitlab.io +document.cookie = "key=value"; + +// Unsafe: This cookie is visible to foo.gitlab.io and its subdomains, +// regardless of the presence of the leading dot. +document.cookie = "key=value;domain=.foo.gitlab.io"; +document.cookie = "key=value;domain=foo.gitlab.io"; +``` + +This issue doesn't affect users with a custom domain, or users who don't set any +cookies manually with JavaScript. diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index cea6bab1a50..b97d5328c07 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -173,7 +173,7 @@ Most modern browsers support downloading files in a compressed format. This speeds up downloads by reducing the size of files. Before serving an uncompressed file, Pages will check whether the same file -exists with a `.gz` extension. If it does, and the browser supports receiving +exists with a `.br` or `.gz` extension. If it does, and the browser supports receiving compressed files, it will serve that version instead of the uncompressed one. To take advantage of this feature, the artifact you upload to the Pages should @@ -182,14 +182,17 @@ have this structure: ```plaintext public/ ├─┬ index.html +│ | index.html.br │ └ index.html.gz │ ├── css/ │ └─┬ main.css +│ | main.css.br │ └ main.css.gz │ └── js/ └─┬ main.js + | main.js.br └ main.js.gz ``` @@ -202,6 +205,7 @@ pages: script: # Build the public/ directory first - find public -type f -regex '.*\.\(htm\|html\|txt\|text\|js\|css\)$' -exec gzip -f -k {} \; + - find public -type f -regex '.*\.\(htm\|html\|txt\|text\|js\|css\)$' -exec brotli -f -k {} \; ``` By pre-compressing the files and including both versions in the artifact, Pages @@ -255,9 +259,8 @@ instead. Here are some examples of what will happen given the above Pages site: | `/other/index` | `200 OK` | `public/other/index.html` | | `/other/index.html` | `200 OK` | `public/other/index.html` | -NOTE: **Note:** -When `public/data/index.html` exists, it takes priority over the `public/data.html` -file for both the `/data` and `/data/` URL paths. +Note that when `public/data/index.html` exists, it takes priority over the `public/data.html` file +for both the `/data` and `/data/` URL paths. ## Frequently Asked Questions diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md index 708d886b352..02d1dd7898a 100644 --- a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -33,9 +33,8 @@ To follow along with this tutorial, we assume you already have: Once you have the requirements addressed, follow the instructions below to learn how to obtain the certificate. -NOTE: **Note:** -The instructions below were tested on macOS Mojave. For other -operating systems the steps might be slightly different. Follow the +Note that these instructions were tested on macOS Mojave. For other operating systems the steps +might be slightly different. Follow the [CertBot instructions](https://certbot.eff.org/) according to your OS. 1. On your computer, open a terminal and navigate to your repository's diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index b6b881b961e..b3705a5835a 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -20,11 +20,9 @@ on your GitLab instance. When enabled, only For a demonstration, see [Pages access controls](https://www.youtube.com/watch?v=tSPAr5mQYc8). 1. Navigate to your project's **Settings > General** and expand **Visibility, project features, permissions**. -1. Toggle the **Pages** button to enable the access control. - NOTE: **Note:** - If you don't see the toggle button, that means that it's not enabled. - Ask your administrator to [enable it](../../../administration/pages/index.md#access-control). +1. Toggle the **Pages** button to enable the access control. If you don't see the toggle button, + that means it isn't enabled. Ask your administrator to [enable it](../../../administration/pages/index.md#access-control). 1. The Pages access control dropdown allows you to set who can view pages hosted with GitLab Pages, depending on your project's visibility: @@ -48,9 +46,10 @@ can access the website. ## Terminating a Pages session -If you want to log out from your Pages website, -you can do so by revoking application access token for GitLab Pages: +To sign out of your GitLab Pages website, revoke the application access token +for GitLab Pages: -1. Navigate to your profile's **Settings > Applications**. -1. Find **Authorized applications** at the bottom of the page. -1. Find **GitLab Pages** and press the **Revoke** button. +1. In the top menu, select your profile, and then select **Settings**. +1. In the left sidebar, select **Applications**. +1. Scroll to the **Authorized applications** section, find the **GitLab Pages** + entry, and select its **Revoke** button. diff --git a/doc/user/project/pages/redirects.md b/doc/user/project/pages/redirects.md index ae7b1b4fa6e..60fbf368061 100644 --- a/doc/user/project/pages/redirects.md +++ b/doc/user/project/pages/redirects.md @@ -6,15 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Create redirects for GitLab Pages -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/24) in GitLab Pages 1.25.0 and GitLab 13.4. -> - It's [deployed behind a feature flag](#enable-or-disable-redirects), disabled by default. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-redirects). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/24) in GitLab Pages 1.25.0 and GitLab 13.4 behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/367) in GitLab 13.5. CAUTION: **Warning:** This feature might not be available to you. Check the **version history** note above for details. -In GitLab Pages, you can [enable](#enable-or-disable-redirects) the redirects feature to configure rules to forward one URL to another using HTTP redirects. GitLab Pages uses -[Netlify style redirects](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file). +In GitLab Pages, you can configure rules to forward one URL to another using +[Netlify style](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file) +HTTP redirects. ## Supported features @@ -22,8 +22,10 @@ GitLab Pages only supports the [`_redirects` plain text file syntax](https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file), and `.toml` files are not supported. -Redirects are only supported at a basic level, and GitLab Pages doesn't support all -[special options offered by Netlify](https://docs.netlify.com/routing/redirects/redirect-options/): +Redirects are only supported at a basic level. GitLab Pages doesn't support all +[special options offered by Netlify](https://docs.netlify.com/routing/redirects/redirect-options/). + +Note that supported paths must start with a forward slash `/`. | Feature | Supported | Example | | ------- | --------- | ------- | @@ -37,12 +39,9 @@ Redirects are only supported at a basic level, and GitLab Pages doesn't support | Redirect by country or language | **{dotted-circle}** No | `/ /anz 302 Country=au,nz` | | Redirect by role | **{dotted-circle}** No | `/admin/* 200! Role=admin` | -NOTE: **Note:** -Supported paths must start with a forward slash `/`. - ## Create redirects -To create redirects after [enabling](#enable-or-disable-redirects) the feature, +To create redirects, create a configuration file named `_redirects` in the `public/` directory of your GitLab Pages site. @@ -78,8 +77,7 @@ is ignored because `hello.html` exists: /projectname/hello.html /projectname/world.html 302 ``` -NOTE: **Note:** -GitLab does not support Netlify's +GitLab doesn't support Netlify's [force option](https://docs.netlify.com/routing/redirects/rewrites-proxies/#shadowing) to change this behavior. @@ -105,19 +103,19 @@ rule 10: valid rule 11: valid ``` -## Enable or disable redirects +## Disable redirects -Redirects in GitLab Pages is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. +Redirects in GitLab Pages is under development, and is deployed behind a feature flag +that is **enabled by default**. -For [Omnibus installations](../../../administration/pages/index.md), define the +To disable redirects, for [Omnibus installations](../../../administration/pages/index.md), define the `FF_ENABLE_REDIRECTS` environment variable in the [global settings](../../../administration/pages/index.md#global-settings). Add the following line to `/etc/gitlab/gitlab.rb` and [reconfigure the instance](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure). ```ruby -gitlab_pages['env']['FF_ENABLE_REDIRECTS'] = 'true' +gitlab_pages['env']['FF_ENABLE_REDIRECTS'] = 'false' ``` For [source installations](../../../administration/pages/source.md), define the @@ -125,6 +123,6 @@ For [source installations](../../../administration/pages/source.md), define the [restart GitLab](../../../administration/restart_gitlab.md#installations-from-source): ```shell -export FF_ENABLE_REDIRECTS="true" +export FF_ENABLE_REDIRECTS="false" /path/to/pages/bin/gitlab-pages -config gitlab-pages.conf ``` diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 3d0cb1bf3a5..7265fd330e3 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -48,7 +48,7 @@ that the `master` branch is protected by default. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5081) in GitLab 8.11. -Since GitLab 8.11, we added another layer of branch protection which provides +In GitLab 8.11 and later, we added another layer of branch protection which provides more granular management of protected branches. The "Developers can push" option was replaced by an "Allowed to push" setting which can be set to allow/prohibit Maintainers and/or Developers to push to a protected branch. @@ -185,6 +185,8 @@ When enabled, all merge requests targeting these branches will require approval by a Code Owner per matched rule before they can be merged. Additionally, direct pushes to the protected branch are denied if a rule is matched. +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5, users and groups who are allowed to push to protected branches do not require a merge request to merge their feature branches. Thus, they can skip merge request approval rules. + ## Running pipelines on protected branches The permission to merge or push to protected branches is used to define if a user can diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 518cf472b50..2f1b05f481e 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -39,22 +39,22 @@ The following quick actions are applicable to descriptions, discussions and thre | `/copy_metadata <!merge_request>` | ✓ | ✓ | | Copy labels and milestone from another merge request in the project. | | `/copy_metadata <#issue>` | ✓ | ✓ | | Copy labels and milestone from another issue in the project. | | `/create_merge_request <branch name>` | ✓ | | | Create a new merge request starting from the current issue. | -| `/done` | ✓ | ✓ | ✓ | Mark To-Do as done. | +| `/done` | ✓ | ✓ | ✓ | Mark to do as done. | | `/due <date>` | ✓ | | | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | | `/duplicate <#issue>` | ✓ | | | Close this issue and mark as a duplicate of another issue. **(CORE)** Also, mark both as related. **(STARTER)** | | `/epic <epic>` | ✓ | | | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. **(PREMIUM)** | | `/estimate <<W>w <DD>d <hh>h <mm>m>` | ✓ | ✓ | | Set time estimate. For example, `/estimate 1w 3d 2h 14m`. | | `/iteration *iteration:"iteration name"` | ✓ | | | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced in GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/196795)). **(STARTER)** | | `/label ~label1 ~label2` | ✓ | ✓ | ✓ | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | -| `/lock` | ✓ | ✓ | | Lock the thread. | +| `/lock` | ✓ | ✓ | | Lock the discussions. | | `/merge` | | ✓ | | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), adding to a [Merge Train](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md), etc. | | `/milestone %milestone` | ✓ | ✓ | | Set milestone. | | `/move <path/to/project>` | ✓ | | | Move this issue to another project. | | `/parent_epic <epic>` | | | ✓ | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/-/issues/10556)). **(ULTIMATE)** | | `/promote` | ✓ | | | Promote issue to epic. **(PREMIUM)** | | `/publish` | ✓ | | | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced in GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906)) **(ULTIMATE)** | -| `/reassign @user1 @user2` | ✓ | ✓ | | Change assignee. **(STARTER)** | -| `/relabel ~label1 ~label2` | ✓ | ✓ | ✓ | Replace existing labels with those specified. | +| `/reassign @user1 @user2` | ✓ | ✓ | | Replace current assignees with those specified. **(STARTER)** | +| `/relabel ~label1 ~label2` | ✓ | ✓ | ✓ | Replace current labels with those specified. | | `/relate #issue1 #issue2` | ✓ | | | Mark issues as related. **(STARTER)** | | `/remove_child_epic <epic>` | | | ✓ | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab/-/issues/7330)). **(ULTIMATE)** | | `/remove_due_date` | ✓ | | | Remove due date. | @@ -74,11 +74,12 @@ The following quick actions are applicable to descriptions, discussions and thre | `/tableflip <comment>` | ✓ | ✓ | ✓ | Append the comment with `(╯°□°)╯︵ ┻━┻`. | | `/target_branch <local branch name>` | | ✓ | | Set target branch. | | `/title <new title>` | ✓ | ✓ | ✓ | Change title. | -| `/todo` | ✓ | ✓ | ✓ | Add a To-Do. | +| `/todo` | ✓ | ✓ | ✓ | Add a to do. | | `/unassign @user1 @user2` | ✓ | ✓ | | Remove specific assignees. **(STARTER)** | | `/unassign` | ✓ | ✓ | | Remove all assignees. | -| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | ✓ | ✓ | ✓ | Remove all or specific labels. | -| `/unlock` | ✓ | ✓ | | Unlock the thread. | +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | ✓ | ✓ | ✓ | Remove specified labels. | +| `/unlabel` or `/remove_label` | ✓ | ✓ | ✓ | Remove all labels. | +| `/unlock` | ✓ | ✓ | | Unlock the discussions. | | `/unsubscribe` | ✓ | ✓ | ✓ | Unsubscribe from notifications. | | `/weight <value>` | ✓ | | | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. **(STARTER)** | | `/wip` | | ✓ | | Toggle the Work In Progress status. | diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 6c8aacd12b3..962d612cac1 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -47,22 +47,20 @@ To view a list of releases: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32812) in GitLab 12.9. Releases can be created directly in the GitLab UI. -NOTE: **Note:** -Only users with Developer permissions or higher can create releases. -Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). - You can create a release in the user interface, or by using the [Releases API](../../../api/releases/index.md#create-a-release). We recommend using the API to create releases as one of the last steps in your CI/CD pipeline. +Only users with Developer permissions or higher can create releases. +Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). + To create a new release through the GitLab UI: 1. Navigate to **Project overview > Releases** and click the **New release** button. 1. In the [**Tag name**](#tag-name) box, enter a name. - NOTE: **Note:** Creating a release based on an existing tag using the user interface is not yet supported. However, this is possible using the [Releases API](../../../api/releases/index.md#create-a-release). @@ -88,7 +86,6 @@ release tag. When the `released_at` date and time has passed, the badge is autom > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26016) in GitLab 12.6. Asset link editing was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9427) in GitLab 12.10. -NOTE: **Note:** Only users with Developer permissions or higher can edit releases. Read more about [Release permissions](../../../user/permissions.md#project-members-permissions). @@ -225,7 +222,6 @@ The release title can be customized using the **Release title** field when creating or editing a release. If no title is provided, the release's tag name is used instead. -NOTE: **Note:** Guest users of private projects are allowed to view the **Releases** page but are _not_ allowed to view details about the Git repository (in particular, tag names). Because of this, release titles are replaced with a generic @@ -254,7 +250,6 @@ Every release has a description. You can add any text you like, but we recommend including a changelog to describe the content of your release. This helps users quickly scan the differences between each release you publish. -NOTE: **Note:** [Git's tagging messages](https://git-scm.com/book/en/v2/Git-Basics-Tagging) and Release note descriptions are unrelated. Description supports [Markdown](../../markdown.md). @@ -334,8 +329,7 @@ generate release evidence for an existing release. Because of this, each release can have multiple release evidence snapshots. You can view the release evidence and its details on the Releases page. -NOTE: **Note:** -When the issue tracker is disabled, release evidence [cannot be downloaded](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). +When the issue tracker is disabled, release evidence [can't be downloaded](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). Here is an example of a release evidence object: @@ -431,10 +425,13 @@ ruby: junit: rspec.xml ``` -If the pipeline ran successfully, when you create your release, the `rspec.xml` file is saved as release evidence. +If the pipeline ran successfully, when you create your release, the `rspec.xml` file is saved as +release evidence. -NOTE: **Note:** -If you [schedule release evidence collection](#schedule-release-evidence-collection), some artifacts may already be expired by the time of evidence collection. To avoid this you can use the [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) keyword. Learn more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). +If you [schedule release evidence collection](#schedule-release-evidence-collection), +some artifacts may already be expired by the time of evidence collection. To avoid this you can use +the [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) +keyword. Learn more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/222351). ### Schedule release evidence collection @@ -449,21 +446,6 @@ In the API: - If you do not specify a `released_at` date, release evidence is collected on the date the release is created. -### Disable release evidence display **(CORE ONLY)** - -The `:release_evidence_collection` feature flag is enabled by default in GitLab -self-managed instances. To turn it off, ask a GitLab administrator with Rails console -access to run the following command: - -```ruby -Feature.disable(:release_evidence_collection) -``` - -NOTE: **Note:** -Release evidence is collected regardless of this feature flag, -which only enables or disables the display of the data on the -Releases page. - ## GitLab Releaser > [Introduced](https://gitlab.com/gitlab-org/gitlab-releaser/-/merge_requests/6) in GitLab 12.10. diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index e90f0a7354c..b0aa7569579 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -14,7 +14,7 @@ can create a fork. A fork is a personal copy of the repository and all its branches, which you create in a namespace of your choice. This way you can make changes in your own fork and -submit them through a merge request to the repo you don't have access to. +submit them through a merge request to the repository you don't have access to. ## Creating a fork diff --git a/doc/user/project/repository/img/repository_mirroring_push_settings.png b/doc/user/project/repository/img/repository_mirroring_push_settings.png Binary files differindex d055cc580c4..9fc25dd3b25 100644 --- a/doc/user/project/repository/img/repository_mirroring_push_settings.png +++ b/doc/user/project/repository/img/repository_mirroring_push_settings.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 536cae263b8..5473439a162 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -86,7 +86,7 @@ according to the markup language. | [reStructuredText](https://docutils.sourceforge.io/rst.html) | `rst` | | [AsciiDoc](../../asciidoc.md) | `adoc`, `ad`, `asciidoc` | | [Textile](https://textile-lang.com/) | `textile` | -| [rdoc](http://rdoc.sourceforge.net/doc/index.html) | `rdoc` | +| [Rdoc](http://rdoc.sourceforge.net/doc/index.html) | `rdoc` | | [Org mode](https://orgmode.org/) | `org` | | [creole](http://www.wikicreole.org/) | `creole` | | [MediaWiki](https://www.mediawiki.org/wiki/MediaWiki) | `wiki`, `mediawiki` | @@ -234,7 +234,7 @@ lock your files to prevent any conflicting changes. ## Repository's API -You can access your repos via [repository API](../../../api/repositories.md). +You can access your repositories via [repository API](../../../api/repositories.md). ## Clone in Apple Xcode diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 28fdda07b05..ad79fd8a8f9 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -19,7 +19,7 @@ over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and [BFG](https://rtyley.github.io/bfg-repo-cleaner/). DANGER: **Danger:** -Rewriting repository history is a destructive operation. Make sure to backup your repository before +Rewriting repository history is a destructive operation. Make sure to back up your repository before you begin. The best way back up a repository is to [export the project](../settings/import_export.md#exporting-a-project-and-its-data). @@ -230,6 +230,7 @@ This will: - Run `git gc` against the repository to remove unreferenced objects. Repacking your repository will temporarily cause the size of your repository to increase significantly, because the old pack files are not removed until the new pack files have been created. +- Unlink any unused LFS objects currently attached to your project, freeing up storage space. - Recalculate the size of your repository on disk. You will receive an email notification with the recalculated repository size after the cleanup has completed. @@ -266,21 +267,20 @@ You can still: - Create new issues. - Clone the project. -If you exceed the repository size limit, you might try to: +If you exceed the repository size limit, you can: 1. Remove some data. 1. Make a new commit. 1. Push back to the repository. -Perhaps you might also: +If these actions are insufficient, you can also: - Move some blobs to LFS. - Remove some old dependency updates from history. -Unfortunately, this workflow won't work. Deleting files in a commit doesn't actually reduce the size -of the repository because the earlier commits and blobs still exist. - -What you need to do is rewrite history. We recommend the open-source community-maintained tool +Unfortunately, this workflow doesn't work. Deleting files in a commit doesn't actually reduce the +size of the repository, because the earlier commits and blobs still exist. Instead, you must rewrite +history. We recommend the open-source community-maintained tool [`git filter-repo`](https://github.com/newren/git-filter-repo). NOTE: **Note:** diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index e1d2c20850b..188699e0c77 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -56,6 +56,7 @@ The following are some possible use cases for repository mirroring: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/249) in GitLab Enterprise Edition 8.7. > - [Moved to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18715) in 10.8. +> - [LFS support over HTTPS added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40137) in 13.5 For an existing project, you can set up push mirroring as follows: @@ -181,7 +182,7 @@ To set up a mirror from GitLab to AWS CodeCommit: not confuse it with the IAM user ID or AWS keys of this user. 1. Copy or download special Git HTTPS user ID and password. -1. In the AWS CodeCommit console, create a new repository to mirror from your GitLab repo. +1. In the AWS CodeCommit console, create a new repository to mirror from your GitLab repository. 1. Open your new repository and click **Clone URL > Clone HTTPS** (not **Clone HTTPS (GRC)**). 1. In GitLab, open the repository to be push-mirrored. 1. Click **Settings > Repository** and expand **Mirroring repositories**. @@ -192,7 +193,7 @@ To set up a mirror from GitLab to AWS CodeCommit: ``` Replace `<your_aws_git_userid>` with the AWS **special HTTPS Git user ID** from the IAM Git - credentials created earlier. Replace `<your_codecommit_repo>` with the name of your repo in CodeCommit. + credentials created earlier. Replace `<your_codecommit_repo>` with the name of your repository in CodeCommit. 1. For **Mirror direction**, select **Push**. 1. For **Authentication method**, select **Password** and fill in the **Password** field with the special IAM Git clone user ID **password** created earlier in AWS. diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index 972a46ee7a3..9b420d84f50 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -26,7 +26,7 @@ For a commit or tag to be *verified* by GitLab: [Omnibus install custom public certificates](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). - The signing time has to be within the time range of the [certificate validity](https://www.rfc-editor.org/rfc/rfc5280.html#section-4.1.2.5) which is usually up to three years. -- The signing time is equal or later then commit time. +- The signing time is equal or later than commit time. NOTE: **Note:** Certificate revocation lists are checked on a daily basis via background worker. diff --git a/doc/user/project/requirements/img/requirement_create_v13_5.png b/doc/user/project/requirements/img/requirement_create_v13_5.png Binary files differnew file mode 100644 index 00000000000..ef1bab6e6d2 --- /dev/null +++ b/doc/user/project/requirements/img/requirement_create_v13_5.png diff --git a/doc/user/project/requirements/img/requirement_view_v13_5.png b/doc/user/project/requirements/img/requirement_view_v13_5.png Binary files differnew file mode 100644 index 00000000000..7fcb24a5e3b --- /dev/null +++ b/doc/user/project/requirements/img/requirement_view_v13_5.png diff --git a/doc/user/project/requirements/img/requirements_list_v13_1.png b/doc/user/project/requirements/img/requirements_list_v13_1.png Binary files differdeleted file mode 100644 index 0ebda571928..00000000000 --- a/doc/user/project/requirements/img/requirements_list_v13_1.png +++ /dev/null diff --git a/doc/user/project/requirements/img/requirements_list_v13_5.png b/doc/user/project/requirements/img/requirements_list_v13_5.png Binary files differnew file mode 100644 index 00000000000..19516e5e66e --- /dev/null +++ b/doc/user/project/requirements/img/requirements_list_v13_5.png diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index 9d7d3914905..f533f8807d2 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -7,7 +7,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Requirements Management **(ULTIMATE)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2703) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2703) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +> - The ability to add and edit a requirement's long description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/224622) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. With requirements, you can set criteria to check your products against. They can be based on users, stakeholders, system, software, or anything else you find important to capture. @@ -22,7 +23,7 @@ When a feature is no longer necessary, you can [archive the related requirement] <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [GitLab 12.10 Introduces Requirements Management](https://www.youtube.com/watch?v=uSS7oUNSEoU). - + ## Create a requirement @@ -32,29 +33,43 @@ can create a new requirement. To create a requirement: 1. From your project page, go to **{requirements}** **Requirements**. -1. Click **New requirement**. -1. Enter a descriptive title and click **Create requirement**. +1. Select **New requirement**. +1. Enter a title and description and select **Create requirement**. -You will see the newly created requirement on the top of the list, as the requirements -list is sorted by creation date in descending order. + + +You can see the newly created requirement on the top of the list, with the requirements +list being sorted by creation date, in descending order. + +## View a requirement + +You can view a requirement from the list by selecting it. + + + +To edit a requirement while viewing it, select the **Edit** icon (**{pencil}**) +next to the requirement title. ## Edit a requirement +> The ability to mark a requirement as Satisfied [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218607) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. + You can edit a requirement (if you have the necessary privileges) from the requirements list page. To edit a requirement: -1. From the requirements list, click **Edit** (**{pencil}**). -1. Update the title in text input field. -1. Click **Save changes**. +1. From the requirements list, select the **Edit** icon (**{pencil}**). +1. Update the title and description in text input field. You can also mark a + requirement as satisfied in the edit form by using the check box **Satisfied**. +1. Select **Save changes**. ## Archive a requirement You can archive an open requirement (if you have the necessary privileges) while you're in the **Open** tab. -To archive a requirement, click **Archive** (**{archive}**). +To archive a requirement, select **Archive** (**{archive}**). As soon as a requirement is archived, it no longer appears in the **Open** tab. @@ -64,7 +79,7 @@ You can view the list of archived requirements in the **Archived** tab.  -To reopen an archived requirement, click **Reopen**. +To reopen an archived requirement, select **Reopen**. As soon as a requirement is reopened, it no longer appears in the **Archived** tab. @@ -80,7 +95,7 @@ You can search for a requirement from the requirements list page based on the fo To search for a requirement: 1. In a project, go to **{requirements}** **Requirements > List**. -1. Click the **Search or filter results** field. A dropdown menu appears. +1. Select the **Search or filter results** field. A dropdown menu appears. 1. Select the requirement author from the dropdown or enter plain text to search by requirement title. 1. Press <kbd>Enter</kbd> on your keyboard to filter the list. @@ -97,7 +112,7 @@ You can also sort the requirements list by: GitLab supports [requirements test reports](../../../ci/pipelines/job_artifacts.md#artifactsreportsrequirements) now. You can add a job to your CI pipeline that, when triggered, marks all existing -requirements as Satisfied. +requirements as Satisfied (you may manually satisfy a requirement in the edit form [edit a requirement](#edit-a-requirement)). ### Add the manual job to CI diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index e9b07f54b91..0a1b81a6359 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -10,8 +10,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/214839) to [GitLab Starter](https://about.gitlab.com/pricing/) in 13.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/215364) to [GitLab Core](https://about.gitlab.com/pricing/) in 13.2. -## Overview - Service Desk is a module that allows your team to connect directly with any external party through email right inside of GitLab; no external tools required. An ongoing conversation right where your software is built ensures that user feedback ends @@ -129,7 +127,7 @@ in the email, `%{ISSUE_PATH}` placeholder which will be replaced by You can customize the email display name. Emails sent from Service Desk will have this name in the `From` header. The default display name is `GitLab Support Bot`. -### Using custom email address +### Using custom email address **(CORE ONLY)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 395d4bf30c5..8fdcf58b5aa 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -8,7 +8,7 @@ type: reference, index, howto # Project settings NOTE: **Note:** -Only project Maintainers and Admin users have the [permissions](../../permissions.md#project-members-permissions) +Only project maintainers and administrators have the [permissions](../../permissions.md#project-members-permissions) to access a project settings. You can adjust your [project](../index.md) settings by navigating @@ -145,7 +145,7 @@ Here you can run housekeeping, archive, rename, transfer, [remove a fork relatio Archiving a project makes it read-only for all users and indicates that it's no longer actively maintained. Projects that have been archived can also be -unarchived. Only project Owners and Admin users have the +unarchived. Only project owners and administrators have the [permissions](../../permissions.md#project-members-permissions) to archive a project. When a project is archived, the repository, packages, issues, merge requests, and all @@ -162,12 +162,12 @@ To archive a project: #### Unarchiving a project Unarchiving a project removes the read-only restriction on a project, and makes it -available in project listings. Only project Owners and Admin users have the +available in project listings. Only project owners and administrators have the [permissions](../../permissions.md#project-members-permissions) to unarchive a project. To find an archived project: -1. Sign in to GitLab as a user with project Owner or Admin permissions. +1. Sign in to GitLab as a user with project owner or administrator permissions. 1. If you: - Have the project's URL, open the project's page in your browser. - Don't have the project's URL: @@ -186,7 +186,7 @@ Next, to unarchive the project: #### Renaming a repository NOTE: **Note:** -Only project Maintainers and Admin users have the [permissions](../../permissions.md#project-members-permissions) to rename a +Only project maintainers and administrators have the [permissions](../../permissions.md#project-members-permissions) to rename a repository. Not to be confused with a project's name where it can also be changed from the [general project settings](#general-project-settings). @@ -207,7 +207,7 @@ old URL won't be able to push or pull. Read more about what happens with the #### Transferring an existing project into another namespace NOTE: **Note:** -Only project Owners and Admin users have the [permissions](../../permissions.md#project-members-permissions) +Only project owners and administrators have the [permissions](../../permissions.md#project-members-permissions) to transfer a project. You can transfer an existing project into a [group](../../group/index.md) if: @@ -229,13 +229,13 @@ read what happens with the [redirects from the old project to the new one](../index.md#redirects-when-changing-repository-paths). NOTE: **Note:** -GitLab administrators can use the admin interface to move any project to any +GitLab administrators can use the administration interface to move any project to any namespace if needed. #### Delete a project NOTE: **Note:** -Only project owners and admins have [permissions](../../permissions.md#project-members-permissions) to delete a project. +Only project owners and administrators have [permissions](../../permissions.md#project-members-permissions) to delete a project. To delete a project: @@ -247,7 +247,7 @@ This action: - Deletes a project including all associated resources (issues, merge requests etc). - From [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) on [Premium or Silver](https://about.gitlab.com/pricing/) or higher tiers, -group admins can [configure](../../group/index.md#enabling-delayed-project-removal) projects within a group +group administrators can [configure](../../group/index.md#enabling-delayed-project-removal) projects within a group to be deleted after a delayed period. When enabled, actual deletion happens after number of days specified in [instance settings](../../admin_area/settings/visibility_and_access_controls.md#default-deletion-delay). @@ -270,7 +270,7 @@ To restore a project marked for deletion: Forking is a great way to [contribute to a project](../repository/forking_workflow.md) of which you're not a member. If you want to use the fork for yourself and don't need to send -[merge requests](../merge_requests.md) to the upstream project, +[merge requests](../merge_requests/index.md) to the upstream project, you can safely remove the fork relationship. CAUTION: **Caution:** diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 57cb610a2e9..b6ce21ebea6 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -1,21 +1,22 @@ --- -stage: Create -group: Source Code +stage: Manage +group: Access info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference, howto --- -# Project access tokens **(CORE ONLY)** +# Project access tokens + +NOTE: **Note:** +Project access tokens are supported for self-managed instances on Core and above. They are also supported on GitLab.com Bronze and above. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2587) in GitLab 13.0. > - It was [deployed](https://gitlab.com/groups/gitlab-org/-/epics/2587) behind a feature flag, disabled by default. > - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/218722) in GitLab 13.3. -> - It's disabled on GitLab.com. -> - It can be enabled or disabled by project. +> - [Became available on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/235765) in 13.5. > - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can [disable it](#enable-or-disable-project-access-tokens). -Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). You can also use project access tokens with Git to authenticate over HTTP or SSH. +Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). You can also use project access tokens with Git to authenticate over HTTP. Project access tokens expire on the date you define, at midnight UTC. @@ -48,12 +49,12 @@ API calls made with a project access token are associated with the corresponding These users will appear in **Members** but can not be modified. Furthermore, the bot user can not be added to any other project. -When the project access token is [revoked](#revoking-a-project-access-token) the bot user will be deleted and all -records will be moved to a system-wide user with the username "Ghost User". For more information, -see [Associated Records](../../profile/account/delete_account.md#associated-records). +- The username is set to `project_{project_id}_bot` for the first access token, such as `project_123_bot`. +- The username is set to `project_{project_id}_bot{bot_count}` for further access tokens, such as `project_123_bot1`. + +When the project access token is [revoked](#revoking-a-project-access-token) the bot user is then deleted and all records are moved to a system-wide user with the username "Ghost User". For more information, see [Associated Records](../../profile/account/delete_account.md#associated-records). -Project bot users are a [GitLab-created service account](../../../subscriptions/self_managed/index.md#choose-the-number-of-users), but count as a licensed seat. -These users will not count against your licensed seat in the future when [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/223695) is resolved. +Project bot users are [GitLab-created service accounts](../../../subscriptions/self_managed/index.md#choose-the-number-of-users) and do not count as licensed seats. ## Revoking a project access token @@ -74,33 +75,3 @@ the following table. | `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | | `read_repository` | Allows read-only access (pull) to the repository. | | `write_repository` | Allows read-write access (pull, push) to the repository. | - -### Enable or disable project access tokens - -Project access tokens are deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable it for your instance, globally or by project. - -To disable it globally: - -```ruby -Feature.disable(:resource_access_token) -``` - -To disable it for a specific project: - -```ruby -Feature.disable(:resource_access_token, project) -``` - -To enable it globally: - -```ruby -Feature.enable(:resource_access_token) -``` - -To enable it for a specific project: - -```ruby -Feature.enable(:resource_access_token, project) -``` diff --git a/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png b/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png Binary files differnew file mode 100644 index 00000000000..89864858ed3 --- /dev/null +++ b/doc/user/project/static_site_editor/img/front_matter_ui_v13_4.png diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index ce14cefba92..8a2f62ec7a2 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -6,50 +6,43 @@ type: reference, how-to description: "The static site editor enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture or Git commands." --- -# Static Site Editor +# Static Site Editor **(CORE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. > - WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214559) in GitLab 13.0. -> - Support for adding images through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216640) in GitLab 13.1. -> - Markdown front matter hidden on the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216834) in GitLab 13.1. -> - Support for `*.md.erb` files [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223171) in GitLab 13.2. > - Non-Markdown content blocks uneditable on the WYSIWYG mode [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216836) in GitLab 13.3. -DANGER: **Danger:** -In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) -to the URL structure of the Static Site Editor. Follow the instructions in this -[snippet](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/snippets/1976539) -to update your project with the latest changes. - -Static Site Editor enables users to edit content on static websites without +Static Site Editor (SSE) enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture, or Git commands. A contributor to your project can quickly edit a Markdown page and submit the changes for review. ## Use cases -The Static Site Editors allows collaborators to submit changes to static site +The Static Site Editor allows collaborators to submit changes to static site files seamlessly. For example: -- Non-technical collaborators can easily edit a page directly from the browser; they don't need to know Git and the details of your project to be able to contribute. +- Non-technical collaborators can easily edit a page directly from the browser; + they don't need to know Git and the details of your project to be able to contribute. - Recently hired team members can quickly edit content. -- Temporary collaborators can jump from project to project and quickly edit pages instead of having to clone or fork every single project they need to submit changes to. +- Temporary collaborators can jump from project to project and quickly edit pages instead + of having to clone or fork every single project they need to submit changes to. ## Requirements - In order use the Static Site Editor feature, your project needs to be -pre-configured with the [Static Site Editor Middleman template](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman). -- The editor needs to be logged into GitLab and needs to be a member of the -project (with Developer or higher permission levels). + pre-configured with the [Static Site Editor Middleman template](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman). +- You need to be logged into GitLab and be a member of the + project (with Developer or higher permission levels). ## How it works -The Static Site Editor is in an early stage of development and only works for +The Static Site Editor is in an early stage of development and only supports Middleman sites for now. You have to use a specific site template to start using it. The project template is configured to deploy a [Middleman](https://middlemanapp.com/) static website with [GitLab Pages](../pages/index.md). -Once your website is up and running, you'll see a button **Edit this page** on +Once your website is up and running, an **Edit this page** button displays on the bottom-left corner of its pages:  @@ -65,44 +58,128 @@ creates a new branch, commits their changes, and opens a merge request. The editor lands directly on the merge request, and then they can assign it to a colleague for review. -## Getting started +## Set up your project First, set up the project. Once done, you can use the Static Site Editor to -easily edit your content. - -### Set up your project - -1. To get started, create a new project from the -[Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) -template. You can either [fork it](../repository/forking_workflow.md#creating-a-fork) -or [create a new project from a template](../../../gitlab-basics/create-project.md#built-in-templates). -1. Edit the `data/config.yml` file adding your project's path. -1. Editing the file triggers a CI/CD pipeline to deploy your project with GitLab Pages. +easily [edit your content](#edit-content). + +1. To get started, create a new project from the [Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) + template. You can either [fork it](../repository/forking_workflow.md#creating-a-fork) + or [create a new project from a template](../../../gitlab-basics/create-project.md#built-in-templates). +1. Edit the [`data/config.yml`](#configuration-files) configuration file + to replace `<username>` and `<project-name>` with the proper values for + your project's path. This triggers a CI/CD pipeline to deploy your project + with GitLab Pages. 1. When the pipeline finishes, from your project's left-side menu, go to **Settings > Pages** to find the URL of your new website. 1. Visit your website and look at the bottom-left corner of the screen to see the new **Edit this page** button. -Anyone satisfying the [requirements](#requirements) will be able to edit the +Anyone satisfying the [requirements](#requirements) can edit the content of the pages without prior knowledge of Git or of your site's codebase. -NOTE: **Note:** -From GitLab 13.1 onwards, the YAML front matter of Markdown files is hidden on the -WYSIWYG editor to avoid unintended changes. To edit it, use the Markdown editing mode, the regular -GitLab file editor, or the Web IDE. +## Edit content + +> - Support for modifying the default merge request title and description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216861) in GitLab 13.5. + +After setting up your project, you can start editing content directly from the Static Site Editor. + +To edit a file: + +1. Visit the page you want to edit. +1. Click the **Edit this page** button. +1. The file is opened in the Static Site Editor in **WYSIWYG** mode. If you + wish to edit the raw Markdown instead, you can toggle the **Markdown** mode + in the bottom-right corner. +1. When you're done, click **Submit changes...**. +1. (Optional) Adjust the default title and description of the merge request that will be submitted with your changes. +1. Click **Submit changes**. +1. A new merge request is automatically created and you can assign a colleague for review. + +### Text + +> Support for `*.md.erb` files [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223171) in GitLab 13.2. + +The Static Site Editors supports Markdown files (`.md`, `.md.erb`) for editing text. + +### Images + +> - Support for adding images through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216640) in GitLab 13.1. + +You can add image files on the WYSIWYG mode by clicking the image icon (**{doc-image}**). +From there, link to a URL, add optional [ALT text](https://moz.com/learn/seo/alt-text), +and you're done. The link can reference images already hosted in your project, an asset hosted +externally on a content delivery network, or any other external URL. The editor renders thumbnail previews +so you can verify the correct image is included and there aren't any references to missing images. +default directory (`source/images/`). + +### Videos -### Use the Static Site Editor to edit your content +> - Support for embedding YouTube videos through the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216642) in GitLab 13.5. -For instance, suppose you are a recently hired technical writer at a large -company and a new feature has been added to the company product. +You can embed YouTube videos on the WYSIWYG mode by clicking the video icon (**{live-preview}**). +The following URL/ID formats are supported: -1. You are assigned the task of updating the documentation. -1. You visit a page and see content that needs to be edited. -1. Click the **Edit this page** button on the production site. -1. The file is opened in the Static Site Editor in **WYSIWYG** mode. If you wish to edit the raw Markdown - instead, you can toggle the **Markdown** mode in the bottom-right corner. -1. You edit the file right there and click **Submit changes**. -1. A new merge request is automatically created and you assign it to your colleague for review. +- YouTube watch URL (e.g. `https://www.youtube.com/watch?v=0t1DgySidms`) +- YouTube embed URL (e.g. `https://www.youtube.com/embed/0t1DgySidms`) +- YouTube video ID (e.g. `0t1DgySidms`) + +### Front matter + +> - Markdown front matter hidden on the WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216834) in GitLab 13.1. +> - Ability to edit page front matter [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235921) in GitLab 13.5. + +Front matter is a flexible and convenient way to define page-specific variables in data files +intended to be parsed by a static site generator. It is commonly used for setting a page's +title, layout template, or author, but can be used to pass any kind of metadata to the +generator as the page renders out to HTML. Included at the very top of each data file, the +front matter is often formatted as YAML or JSON and requires consistent and accurate syntax. + +To edit the front matter from the Static Site Editor you can use the GitLab's regular file editor, +the Web IDE, or easily update the data directly from the WYSIWYG editor: + +1. Click the **Page settings** button on the bottom-right to reveal a web form with the data you + have on the page's front matter. The form is populated with the current data: + +  + +1. Update the values as you wish and close the panel. +1. When you're done, click **Submit changes...**. +1. Describe your changes (add a commit message). +1. Click **Submit changes**. +1. Click **View merge request** and GitLab will take you there. + +Note that support for adding new attributes to the page's front matter from the form is not supported +yet. You can do so by editing the file locally, through the GitLab regular file editor, or through the Web IDE. Once added, the form will load the new fields. + +## Configuration files + +The Static Site Editor uses Middleman's configuration file, `data/config.yml` +to customize the behavior of the project itself and to control the **Edit this +page** button, rendered through the file [`layout.erb`](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/-/blob/master/source/layouts/layout.erb). + +To [configure the project template to your own project](#set-up-your-project), +you must replace the `<username>` and `<project-name>` in the `data/config.yml` +file with the proper values for your project's path. + +[Other Static Site Generators](#using-other-static-site-generators) used with +the Static Site Editor may use different configuration files or approaches. + +## Using Other Static Site Generators + +Although Middleman is the only Static Site Generator currently officially supported +by the Static Site Editor, you can configure your project's build and deployment +to use a different Static Site Generator. In this case, use the Middleman layout +as an example, and follow a similar approach to properly render an **Edit this page** +button in your Static Site Generator's layout. + +## Upgrade from GitLab 12.10 to 13.0 + +In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) +to the URL structure of the Static Site Editor. Follow the instructions in this +[snippet](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/snippets/1976539) +to update your project with the 13.0 changes. ## Limitations -- The Static Site Editor still cannot be quickly added to existing Middleman sites. Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. +- The Static Site Editor still cannot be quickly added to existing Middleman sites. + Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 821b42af049..4da9b5f88ff 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -32,7 +32,7 @@ file path fragments to start seeing results. ## Syntax highlighting As expected from an IDE, syntax highlighting for many languages within -the Web IDE will make your direct editing even easier. +the Web IDE makes your direct editing even easier. The Web IDE currently provides: @@ -79,7 +79,7 @@ which apply to the entire Web IDE screen. > - Support for validation based on custom schemas [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/226982) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. The Web IDE provides validation support for certain JSON and YAML files using schemas -based on the [JSON Schema Store](https://www.schemastore.org/json/). +based on the [JSON Schema Store](https://www.schemastore.org/json/). ### Predefined schemas @@ -143,7 +143,7 @@ The Web IDE supports configuration of certain editor settings by using [`.editorconfig` files](https://editorconfig.org/). When opening a file, the Web IDE looks for a file named `.editorconfig` in the current directory and all parent directories. If a configuration file is found and has settings -that match the file's path, these settings will be enforced on the opened file. +that match the file's path, these settings are enforced on the opened file. The Web IDE currently supports the following `.editorconfig` settings: @@ -166,7 +166,7 @@ review the list of changed files. Once you have finalized your changes, you can add a commit message, commit the changes and directly create a merge request. In case you don't have write -access to the selected branch, you will see a warning, but still be able to create +access to the selected branch, you see a warning, but can still create a new branch and start a merge request. To discard a change in a particular file, click the **Discard changes** button on that @@ -201,8 +201,7 @@ left. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19318) in [GitLab Core](https://about.gitlab.com/pricing/) 11.0. To switch between your authored and assigned merge requests, click the -dropdown in the top of the sidebar to open a list of merge requests. You will -need to commit or discard all your changes before switching to a different merge +dropdown in the top of the sidebar to open a list of merge requests. You need to commit or discard all your changes before switching to a different merge request. ## Switching branches @@ -211,7 +210,7 @@ request. To switch between branches of the current project repository, click the dropdown in the top of the sidebar to open a list of branches. -You will need to commit or discard all your changes before switching to a +You need to commit or discard all your changes before switching to a different branch. ## Markdown editing @@ -226,7 +225,7 @@ supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-g You can also upload any local images by pasting them directly in the Markdown file. The image is uploaded to the same directory and is named `image.png` by default. If another file already exists with the same name, a numeric suffix is automatically -added to the file name. +added to the filename. ## Live Preview @@ -249,7 +248,7 @@ The Live Preview feature needs to be enabled in the GitLab instances admin settings. Live Preview is enabled for all projects on GitLab.com - + Once you have done that, you can preview projects with a `package.json` file and a `main` entry point inside the Web IDE. An example `package.json` is shown @@ -292,7 +291,7 @@ to work: [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). **(ULTIMATE ONLY)** If you have the terminal open and the job has finished with its tasks, the -terminal will block the job from finishing for the duration configured in +terminal blocks the job from finishing for the duration configured in [`[session_server].session_timeout`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section) until you close the terminal window. @@ -308,15 +307,15 @@ In order to enable the Web IDE terminals you need to create the file file is fairly similar to the [CI configuration file](../../../ci/yaml/README.md) syntax but with some restrictions: -- No global blocks can be defined (ie: `before_script` or `after_script`) +- No global blocks can be defined (i.e., `before_script` or `after_script`) - Only one job named `terminal` can be added to this file. - Only the keywords `image`, `services`, `tags`, `before_script`, `script`, and `variables` are allowed to be used to configure the job. - To connect to the interactive terminal, the `terminal` job must be still alive - and running, otherwise the terminal won't be able to connect to the job's session. + and running, otherwise the terminal cannot connect to the job's session. By default the `script` keyword has the value `sleep 60` to prevent the job from ending and giving the Web IDE enough time to connect. This means - that, if you override the default `script` value, you'll have to add a command + that, if you override the default `script` value, you have to add a command which would keep the job running, like `sleep`. In the code below there is an example of this configuration file: @@ -333,40 +332,39 @@ terminal: NODE_ENV: "test" ``` -Once the terminal has started, the console will be displayed and we could access +Once the terminal has started, the console is displayed and we could access the project repository files. **Important**. The terminal job is branch dependent. This means that the -configuration file used to trigger and configure the terminal will be the one in +configuration file used to trigger and configure the terminal is the one in the selected branch of the Web IDE. -If there is no configuration file in a branch, an error message will be shown. +If there is no configuration file in a branch, an error message is shown. ### Running interactive terminals in the Web IDE -If Interactive Terminals are available for the current user, the **Terminal** button -will be visible in the right sidebar of the Web IDE. Click this button to open +If Interactive Terminals are available for the current user, the **Terminal** button is visible in the right sidebar of the Web IDE. Click this button to open or close the terminal tab. -Once open, the tab will show the **Start Web Terminal** button. This button may +Once open, the tab shows the **Start Web Terminal** button. This button may be disabled if the environment is not configured correctly. If so, a status -message will describe the issue. Here are some reasons why **Start Web Terminal** +message describes the issue. Here are some reasons why **Start Web Terminal** may be disabled: - `.gitlab/.gitlab-webide.yml` does not exist or is set up incorrectly. - No active private runners are available for the project. -If active, clicking the **Start Web Terminal** button will load the terminal view +If active, clicking the **Start Web Terminal** button loads the terminal view and start connecting to the runner's terminal. At any time, the **Terminal** tab -can be closed and reopened and the state of the terminal will not be affected. +can be closed and reopened and the state of the terminal is not affected. When the terminal is started and is successfully connected to the runner, then the -runner's shell prompt will appear in the terminal. From here, you can enter -commands that will be executed within the runner's environment. This is similar +runner's shell prompt appears in the terminal. From here, you can enter +commands executed within the runner's environment. This is similar to running commands in a local terminal or through SSH. While the terminal is running, it can be stopped by clicking **Stop Terminal**. -This will disconnect the terminal and stop the runner's terminal job. From here, +This disconnects the terminal and stops the runner's terminal job. From here, click **Restart Terminal** to start a new terminal session. ### File syncing to web terminal @@ -408,14 +406,14 @@ terminal: more information. - `$CI_PROJECT_DIR` is a [predefined environment variable](../../../ci/variables/predefined_variables.md) - for GitLab Runner. This is where your project's repository will be. + for GitLab Runners. This is where your project's repository resides. Once you have configured the web terminal for file syncing, then when the web -terminal is started, a **Terminal** status will be visible in the status bar. +terminal is started, a **Terminal** status is visible in the status bar.  -Changes made to your files via the Web IDE will sync to the running terminal +Changes made to your files via the Web IDE sync to the running terminal when: - <kbd>Ctrl</kbd> + <kbd>S</kbd> (or <kbd>Cmd</kbd> + <kbd>S</kbd> on Mac) @@ -425,9 +423,12 @@ when: ### Limitations -Interactive Terminals is in a beta phase and will continue to be improved upon in upcoming -releases. In the meantime, please note that the user is limited to having only one -active terminal at a time. +The Web IDE has a few limitations: + +- Interactive Terminals is in a beta phase and continues to be improved in upcoming releases. In the meantime, please note that the user is limited to having only one + active terminal at a time. + +- LFS files can be rendered and displayed but they cannot be updated and committed using the Web IDE. If an LFS file is modified and pushed to the repository, the LFS pointer in the repository will be overwritten with the modified LFS file content. ### Troubleshooting diff --git a/doc/user/project/wiki/img/wiki_sidebar.png b/doc/user/project/wiki/img/wiki_sidebar.png Binary files differdeleted file mode 100644 index ff39c861a73..00000000000 --- a/doc/user/project/wiki/img/wiki_sidebar.png +++ /dev/null diff --git a/doc/user/project/wiki/img/wiki_sidebar_v13_5.png b/doc/user/project/wiki/img/wiki_sidebar_v13_5.png Binary files differnew file mode 100644 index 00000000000..0f445d61d71 --- /dev/null +++ b/doc/user/project/wiki/img/wiki_sidebar_v13_5.png diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 40ef5e216fd..64608b9a915 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -5,7 +5,7 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, how-to --- -# Wiki +# Wiki **(CORE)** A separate system for documentation called Wiki, is built right into each GitLab project. It is enabled by default on all new projects and you can find @@ -19,6 +19,9 @@ You can create Wiki pages in the web interface or [locally using Git](#adding-and-editing-wiki-pages-locally) since every Wiki is a separate Git repository. +[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13195) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5, +**group wikis** became available. Their usage is similar to project wikis, with a few [limitations](../../group/index.md#group-wikis). + ## First time creating the Home page The first time you visit a Wiki, you will be directed to create the Home page. @@ -127,10 +130,12 @@ be preceded by the slash (`/`) character. ## Viewing a list of all created wiki pages +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17673/) in GitLab 13.5, wiki pages are displayed as a nested tree in the sidebar and pages overview. + Every wiki has a sidebar from which a short list of the created pages can be found. The list is ordered alphabetically. - + If you have many pages, not all will be listed in the sidebar. Click on **View All Pages** to see all of them. @@ -163,48 +168,13 @@ Similar to versioned diff file views, you can see the changes made in a given Wi > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14902) in **GitLab 12.10.** > - Git events were [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216014) in **GitLab 13.0.** -> - It's enabled on GitLab.com. -> - Git access activity creation is managed by a feature flag. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-in-git). **(CORE ONLY)** +> - [Feature flag for Git events was removed](https://gitlab.com/gitlab-org/gitlab/-/issues/258665) in **GitLab 13.5** Wiki events (creation, deletion, and updates) are tracked by GitLab and displayed on the [user profile](../../profile/index.md#user-profile), [group](../../group/index.md#view-group-activity), and [project](../index.md#project-activity) activity pages. -### Enable or disable Wiki events in Git **(CORE ONLY)** - -Tracking wiki events through Git is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it for your instance. - -To enable it: - -```ruby -Feature.enable(:wiki_events_on_git_push) -``` - -To enable for just a particular project: - -```ruby -project = Project.find_by_full_path('your-group/your-project') -Feature.enable(:wiki_events_on_git_push, project) -``` - -To disable it: - -```ruby -Feature.disable(:wiki_events_on_git_push) -``` - -To disable for just a particular project: - -```ruby -project = Project.find_by_full_path('your-group/your-project') -Feature.disable(:wiki_events_on_git_push, project) -``` - ## Adding and editing wiki pages locally Since wikis are based on Git repositories, you can clone them locally and edit diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index 53ec8b35631..3152229c39f 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Enablement +group: Global Search info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference --- diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index 804d4c540ac..ed5ecc17a8b 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Editor +stage: Enablement +group: Global Search info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference --- @@ -62,6 +62,7 @@ The Advanced Search Syntax also supports the use of filters. The available filte - filename: Filters by filename. You can use the glob (`*`) operator for fuzzy matching. - path: Filters by path. You can use the glob (`*`) operator for fuzzy matching. - extension: Filters by extension in the filename. Please write the extension without a leading dot. Exact match only. +- blob: Filters by Git `object ID`. Exact match only. To use them, simply add them to your query in the format `<filter_name>:<value>` without any spaces between the colon (`:`) and the value. @@ -72,17 +73,19 @@ Examples: - Finding a file named `found_blob_spec.rb` with the text `CHANGELOG` inside of it: [`CHANGELOG filename:found_blob_spec.rb](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=CHANGELOG+filename%3Afound_blob_spec.rb&group_id=9970&project_id=278964) - Finding the text `EpicLinks` inside files with the `.rb` extension: [`EpicLinks extension:rb`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=EpicLinks+extension%3Arb&group_id=9970&project_id=278964) - Finding the text `Sidekiq` in a file, when that file is in a path that includes `elastic`: [`Sidekiq path:elastic`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=Sidekiq+path%3Aelastic&group_id=9970&project_id=278964) +- Finding the files represented by the Git object ID `998707b421c89bd9a3063333f9f728ef3e43d101`: [`* blob:998707b421c89bd9a3063333f9f728ef3e43d101`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=false&scope=blobs&repository_ref=&search=*+blob%3A998707b421c89bd9a3063333f9f728ef3e43d101&group_id=9970) - Syntax filters can be combined for complex filtering. Finding any file starting with `search` containing `eventHub` and with the `.js` extension: [`eventHub filename:search* extension:js`](https://gitlab.com/search?utf8=%E2%9C%93&snippets=&scope=blobs&repository_ref=&search=eventHub+filename%3Asearch*+extension%3Ajs&group_id=9970&project_id=278964) #### Excluding filters [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31684) in GitLab Starter 13.3. -Filters can be inversed to **filter out** results from the result set, by prefixing the filter name with a `-` (hyphen) character, such as: +Filters can be inverted to **filter out** results from the result set, by prefixing the filter name with a `-` (hyphen) character, such as: - `-filename` - `-path` - `-extension` +- `-blob` Examples: diff --git a/doc/user/search/img/basic_search.png b/doc/user/search/img/basic_search.png Binary files differnew file mode 100644 index 00000000000..234805d5a4f --- /dev/null +++ b/doc/user/search/img/basic_search.png diff --git a/doc/user/search/img/basic_search_results.png b/doc/user/search/img/basic_search_results.png Binary files differnew file mode 100644 index 00000000000..52aa2e3fe6c --- /dev/null +++ b/doc/user/search/img/basic_search_results.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 475a72385ac..412951f8a3b 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -32,7 +32,7 @@ You can also filter the results using the search and filter field, as described You'll also find shortcuts to issues and merge requests created by you or assigned to you on the search field on the top-right of your screen: - + ### Filtering issue and merge request lists @@ -129,14 +129,6 @@ characters to begin your search. For example, if you want to search for issues that have the assignee "Simone Presley", you'll need to type at least "Sim" before autocomplete gives any relevant results. -## Code search - -To search through code or other documents in a single project, you can use -the search field on the top-right of your screen while the project page is open. - - - - ## Search history You can view recent searches by clicking on the little arrow-clock icon, which is to the left of the search input. Click the search entry to run that search again. This feature is available for issues and merge requests. Searches are stored locally in your browser. @@ -155,24 +147,6 @@ Some filters can be added multiple times. These include but are not limited to a  -## Shortcut - -You'll also find a shortcut on the search field on the top-right of the project's dashboard to -quickly access issues and merge requests created or assigned to you within that project: - - - -### Autocomplete suggestions - -You can also type in this search bar to see autocomplete suggestions for: - -- Projects and groups -- Various help pages (try and type **API help**) -- Project feature pages (try and type **milestones**) -- Various settings pages (try and type **user settings**) -- Recently viewed issues (try and type some word from the title of a recently viewed issue) -- Recently viewed merge requests (try and type some word from the title of a recently merge request) - ## To-Do List Your [To-Do List](../todos.md#gitlab-to-do-list) can be searched by "to do" and "done". @@ -219,6 +193,61 @@ and **Labels**, select multiple issues to add to a list of your choice:  +## Shortcut + +You'll find a shortcut on the search field on the top-right of the project's dashboard to +quickly access issues and merge requests created or assigned to you within that project: + + + +### Autocomplete suggestions + +You can also type in this search bar to see autocomplete suggestions for: + +- Projects and groups +- Various help pages (try and type **API help**) +- Project feature pages (try and type **milestones**) +- Various settings pages (try and type **user settings**) +- Recently viewed issues (try and type some word from the title of a recently viewed issue) +- Recently viewed merge requests (try and type some word from the title of a recently viewed merge request) +- Recently viewed epics (try and type some word from the title of a recently viewed epic) + +## Basic search + +The Basic search in GitLab is a global search service that allows you to search +across the entire GitLab instance, within a group, or a single project. Basic search is +backed by the database and allows searching in: + +- Projects +- Issues +- Merge requests +- Milestones +- Users +- Epics (Group only) +- Code (Project only) +- Comments (Project only) +- Commits (Project only) +- Wiki (Project only) + +To start a search, type into the search bar on the top-right of the screen. You can always search +in all GitLab and may also see the options to search within a group or project if you are in the +group or project dashboard. + + + +Once the results are returned, you can modify the search, select a different type of data to +search, or choose a specific group or project. + + + +### Code search + +To search through code or other documents in a single project, you can use +the search field on the top-right of your screen while the project page is open. + + + + ## Advanced Search **(STARTER)** Leverage Elasticsearch for faster, more advanced code search across your entire diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 15391f034a8..2d0c9d4f943 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -92,6 +92,40 @@ be changed to `http-a-weird-filename-me` to be included in the snippet's repository. As snippets are stored by ID, changing their filenames will not break direct or embedded links to the snippet. +### Multiple files by Snippet + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2829) in GitLab 13.5. + +GitLab Snippets support multiple files in one single snippet. It can be very handy +when your code snippet is composed of multiple parts or when they relate +to a certain context. For example: + +- A snippet that includes a script and its output. +- A snippet that includes HTML, CSS, and JS code. +- A snippet with a `docker-compose.yml` file and its associated `.env` file. +- A `gulpfile.js` file coupled with a `package.json` file, which together can be used to bootstrap a project and manage its dependencies. + +Snippets support between 1 and 10 files. They can be managed via Git (since they're [versioned](#versioned-snippets) +by a Git repository), through the [Snippets API](../api/snippets.md), or within the GitLab UI. + + + +To add a new file to your snippet through the GitLab UI: + +1. Go to your snippet in the GitLab UI. +1. Click **Edit** in the top right. +1. Select **Add another file**. +1. Add your content to the file in the form fields provided. +1. Click **Save changes**. + +To delete a file from your snippet through the GitLab UI: + +1. Go to your snippet in the GitLab UI. +1. Click **Edit** in the top right. +1. Select **Delete file** alongside the file name of each file +you wish to delete. +1. Click **Save changes**. + ### Cloning snippets Snippets can be cloned as a regular Git repository using SSH or HTTPS. Click the **Clone** @@ -114,16 +148,16 @@ see the documentation on [reducing repository size](../user/project/repository/r ### Limitations - Binary files are not supported. -- Creating or deleting branches is not supported. Only a default *master*. -branch is used. +- Creating or deleting branches is not supported. Only a default *master* branch is used. - Git tags are not supported in snippet repositories. -- Snippets' repositories are limited to one file. Attempting to push more -than one file will result in an error. +- Snippets' repositories are limited to 10 files. Attempting to push more +than 10 files will result in an error. - Revisions are not *yet* visible to the user on the GitLab UI, but it's planned to be added in future iterations. See the [revisions tab issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39271) for updates. - The [maximum size for a snippet](../administration/snippets/index.md#snippets-content-size-limit) is 50 MB, by default. +- Git LFS is not supported. ## Discover snippets @@ -131,8 +165,8 @@ There are two main ways of how you can discover snippets in GitLab. For exploring all snippets that are visible to you, you can go to the Snippets dashboard of your GitLab instance via the top navigation. For GitLab.com you can -find it [here](https://gitlab.com/dashboard/snippets). This navigates you to an -overview that shows snippets you created and allows you to explore all snippets. +navigate to an [overview]((https://gitlab.com/dashboard/snippets)) that shows snippets +you created and allows you to explore all snippets. If you want to discover snippets that belong to a specific project, you can navigate to the Snippets page via the left side navigation on the project page. diff --git a/doc/user/todos.md b/doc/user/todos.md index 1fca3c0ab64..c48d2adfa45 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -15,7 +15,7 @@ spend your time. This can include taking an action, or keeping track of things do your work, being able to get started quickly is important. Your *To-Do List* offers a chronological list of items waiting for your input -(known as *to-dos*) in a single dashboard. +(known as *to do items*) in a single dashboard. The To-Do List supports tracking [actions](#what-triggers-a-to-do) related to the following: @@ -26,18 +26,18 @@ the following:  -You can access your To-Do List by clicking the **{task-done}** To-Do List icon -next to the search bar in the top navigation. If the to-do item count is: +You can access your To-Do List by clicking the To-Do List icon (**{task-done}**) +next to the search bar in the top navigation. If the to do item count is: -- *Less than 100*, the number in blue is the number of to-do items. +- *Less than 100*, the number in blue is the number of to do items. - *100 or more*, the number displays as 99+. The exact number displays in the To-Do List.  -## What triggers a to-do +## What triggers a to do -A to-do item appears on your To-Do List when: +A to do item appears on your To-Do List when: - An issue or merge request is assigned to you. - You're `@mentioned` in the description or comment of an issue or merge request @@ -51,29 +51,29 @@ A to-do item appears on your To-Do List when: - You're the author. - You're the user that set the merge request to automatically merge after a pipeline succeeds. -- [Since GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12136), a +- [In GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/12136) and later, a merge request is removed from a [merge train](../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md), and you're the user that added it. **(PREMIUM)** When several trigger actions occur for the same user on the same object (for -example, an issue), GitLab displays only the first action as a single to-do +example, an issue), GitLab displays only the first action as a single to do item. -To-do triggers aren't affected by [GitLab notification email settings](profile/notifications.md). +To do item triggers aren't affected by [GitLab notification email settings](profile/notifications.md). NOTE: **Note:** -When a user no longer has access to a resource related to a to-do (such as an -issue, merge request, project, or group), for security reasons GitLab deletes -any related to-do items within the next hour. Deletion is delayed to prevent -data loss, in the case where a user's access is accidentally revoked. +When a user no longer has access to a resource related to a to do item (such as +an issue, merge request, project, or group), for security reasons GitLab +deletes any related to do items within the next hour. Deletion is delayed to +prevent data loss, in the case where a user's access is accidentally revoked. -### Directly addressing a to-do +### Directly addressing a to do > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7926) in GitLab 9.0. -If you're mentioned at the start of a line, the to-do you receive will be listed -as *directly addressed*. For example, in the following comment: +If you're mentioned at the start of a line, the to do item you receive will be +listed as *directly addressed*. For example, in the following comment: ```markdown @alice What do you think? cc: @bob @@ -87,23 +87,27 @@ as *directly addressed*. For example, in the following comment: @erin @frank thank you! ``` -The people receiving directly addressed to-do items are `@alice`, `@erin`, and -`@frank`. Directly addressed to-do items only differ from mentions in their type +The people receiving directly addressed to do items are `@alice`, `@erin`, and +`@frank`. Directly addressed to do items only differ from mentions in their type for filtering purposes; otherwise, they appear as normal. -### Manually creating a to-do +### Manually creating a to do -You can add an issue or merge request (or epic **(ULTIMATE)**) to your -To-Do List by clicking its **Add a To Do** button. +You can also add the following to your To-Do List by clicking the **Add a to do** button on an: + +- [Issue](project/issues/index.md) +- [Merge Request](project/merge_requests/index.md) +- [Epic](group/epics/index.md) **(ULTIMATE)** +- [Design](project/issues/design_management.md)  -## Marking a to-do as done +## Marking a to do as done Any action to an issue or merge request (or epic **(ULTIMATE)**) will mark its -corresponding to-do as done. +corresponding to do item as done. -Actions that dismiss to-do items include: +Actions that dismiss to do items include: - Changing the assignee - Changing the milestone @@ -111,27 +115,28 @@ Actions that dismiss to-do items include: - Commenting on the issue Your To-Do List is personal, and items are only marked as done if you take -action. If you close the issue or merge request, your to-do is marked as done. +action. If you close the issue or merge request, your to do item is marked as +done. To prevent other users from closing issues without you being notified, if someone else closes, merges, or takes action on an issue or merge request (or -epic **(ULTIMATE)**), your to-do will remain pending. +epic **(ULTIMATE)**), your to do item remains pending. -There's just one to-do for each of these, so mentioning a user many times in an -issue will only trigger one to-do item. +There's just one to do item for each of these, so mentioning a user many times +in an issue only triggers one to do item. -If no action is needed, you can manually mark the to-do as done by clicking its -corresponding **Done** button to have GitLab remove the item from your -To-Do List. +If no action is needed, you can manually mark the to do item as done by +clicking its corresponding **Done** button to have GitLab remove the item from +your To-Do List. - + -You can also mark a to-do as done by clicking the **Mark as done** button in the -sidebar of an issue or merge request (or epic **(ULTIMATE)**). +You can also mark a to do item as done by clicking the **Mark as done** button +in the sidebar of an issue or merge request (or epic **(ULTIMATE)**).  -You can mark all your to-do items as done at once by clicking the +You can mark all your to do items as done at once by clicking the **Mark all as done** button. ## Filtering your To-Do List @@ -142,9 +147,9 @@ You can use the following types of filters with your To-Do List: | ------- | ---------------------------------------------------------------- | | Project | Filter by project. | | Group | Filter by group. | -| Author | Filter by the author that triggered the To Do. | +| Author | Filter by the author that triggered the to do. | | Type | Filter by issue, merge request, design, or epic. **(ULTIMATE)** | -| Action | Filter by the action that triggered the To Do. | +| Action | Filter by the action that triggered the to do. | You can also filter by more than one of these at the same time. The previously described [triggering actions](#what-triggers-a-to-do) include: diff --git a/doc/user/upgrade_email_bypass.md b/doc/user/upgrade_email_bypass.md index 027f7337228..35fddc4a1d4 100644 --- a/doc/user/upgrade_email_bypass.md +++ b/doc/user/upgrade_email_bypass.md @@ -82,7 +82,7 @@ As an administrator, you may also confirm a user in the [Admin Area](admin_area/ ## What do I do if I am an administrator and I am locked out? If you are an administrator and cannot otherwise verify your email address, sign in to your GitLab -instance with a [Rails console session](../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session). +instance with a [Rails console session](../administration/operations/rails_console.md#starting-a-rails-console-session). Once connected, run the following commands to confirm your administrator account: ```ruby @@ -94,7 +94,7 @@ admin.save! ## How do I force-confirm all users on my self-managed instance? If you are an administrator and would like to force-confirm all users on your system, sign in to your GitLab -instance with a [Rails console session](../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session). +instance with a [Rails console session](../administration/operations/rails_console.md#starting-a-rails-console-session). Once connected, run the following commands to confirm all user accounts: ```ruby |
