diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
| commit | 9f46488805e86b1bc341ea1620b866016c2ce5ed (patch) | |
| tree | f9748c7e287041e37d6da49e0a29c9511dc34768 /doc/user | |
| parent | dfc92d081ea0332d69c8aca2f0e745cb48ae5e6d (diff) | |
| download | gitlab-ce-9f46488805e86b1bc341ea1620b866016c2ce5ed.tar.gz | |
Add latest changes from gitlab-org/gitlab@13-0-stable-ee
Diffstat (limited to 'doc/user')
272 files changed, 4803 insertions, 2260 deletions
diff --git a/doc/user/admin_area/abuse_reports.md b/doc/user/admin_area/abuse_reports.md index 36de817a29b..77f4a84cf6f 100644 --- a/doc/user/admin_area/abuse_reports.md +++ b/doc/user/admin_area/abuse_reports.md @@ -46,7 +46,7 @@ Blocking a user: The user will be notified with the [following message](https://gitlab.com/gitlab-org/gitlab/blob/master/app/workers/email_receiver_worker.rb#L38): -```text +```plaintext Your account has been blocked. If you believe this is in error, contact a staff member. ``` diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index ed03e8a57f3..5427b0c5200 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -20,10 +20,19 @@ You can style a message's content using the `a` and `br` HTML tags. The `br` tag ## Banners -Banners are shown on the top of a page. +Banners are shown on the top of a page and in Git remote responses.  +```bash +$ git push +... +remote: +remote: **Welcome** to GitLab :wave: +remote: +... +``` + ## Notifications Notifications are shown on the bottom right of a page and can contain placeholders. A placeholder is replaced with an attribute of the active user. Placeholders must be surrounded by curly braces, for example `{{name}}`. diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 204573da02d..e63d0c7c882 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -136,19 +136,22 @@ you must provide the complete email address. #### Users statistics -The **Users statistics** page provides an overview of user accounts by role. Use this information -when validating seat usage of your subscription. +The **Users statistics** page provides an overview of user accounts by role, such as _Users with +highest role Maintainer_. -The page displays subtotals of all users matching criteria such as _Users with highest role -Maintainer_ and _Blocked users_. +The following totals are also included: -The **Total users** is calculated as: **Active users** + **Blocked users**. +- Active users +- Blocked users +- Total users -GitLab billing is based on the number of active users. For details of active users, see +GitLab billing is based on the number of **Active users**, calculated as **Total users** - +**Blocked users**. For details of active users, see [Choosing the number of users](../../subscriptions/index.md#choosing-the-number-of-users). -**Please note** that during the initial stage, the information won't be 100% accurate given that -background processes are still recollecting data. +NOTE: **Note:** +Users statistics are calculated daily, so user changes made since the last update won't be +reflected in the statistics. ### Administering Groups diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index fd74b2f311f..59b71b05b16 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -10,8 +10,8 @@ by **signing into your GitLab instance as an admin** or add it at installation time. The license has the form of a base64 encoded ASCII text with a `.gitlab-license` -extension and can be obtained when you [purchase one][pricing] or when you sign -up for a [free trial]. +extension and can be obtained when you [purchase one](https://about.gitlab.com/pricing/) or when you sign +up for a [free trial](https://about.gitlab.com/free-trial/). NOTE: **Note:** As of GitLab Enterprise Edition 9.4.0, a newly-installed instance without an @@ -104,9 +104,6 @@ expired license(s). It's possible to upload and view more than one license, but only the latest license will be used as the active license. -[free trial]: https://about.gitlab.com/free-trial/ -[pricing]: https://about.gitlab.com/pricing/ - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index f69584fcb94..91e29118e3e 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -4,8 +4,8 @@ type: concepts, howto # Health Check **(CORE ONLY)** -> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. -> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was +> - Liveness and readiness probes were [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10416) in GitLab 9.1. +> - The `health_check` endpoint was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3888) in GitLab 8.8 and was > deprecated in GitLab 9.1. > - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 > in favor of [IP whitelist](#ip-whitelist). @@ -25,15 +25,15 @@ For details, see [how to add IPs to a whitelist for the monitoring endpoints](.. With default whitelist settings, the probes can be accessed from localhost using the following URLs: -```text +```plaintext GET http://localhost/-/health ``` -```text +```plaintext GET http://localhost/-/readiness ``` -```text +```plaintext GET http://localhost/-/liveness ``` @@ -45,7 +45,7 @@ are running. This endpoint circumvents Rails Controllers and is implemented as additional middleware `BasicHealthCheck` very early into the request processing lifecycle. -```text +```plaintext GET /-/health ``` @@ -57,7 +57,7 @@ curl 'https://gitlab.example.com/-/health' Example response: -```text +```plaintext GitLab OK ``` @@ -71,7 +71,7 @@ If the `all=1` parameter is specified, the check will also validate the dependent services (Database, Redis, Gitaly etc.) and gives a status for each. -```text +```plaintext GET /-/readiness GET /-/readiness?all=1 ``` @@ -111,7 +111,7 @@ Checks whether the application server is running. This probe is used to know if Rails Controllers are not deadlocked due to a multi-threading. -```text +```plaintext GET /-/liveness ``` @@ -148,7 +148,7 @@ accepted token can be found under the **Admin Area > Monitoring > Health check** The access token can be passed as a URL parameter: -```text +```plaintext https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` @@ -163,9 +163,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[ce-10416]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10416 -[ce-3888]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3888 -[pingdom]: https://www.pingdom.com -[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html -[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index f05ef18da8e..83b29597bec 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -44,7 +44,7 @@ For instance, consider the following workflow: 1. Your team develops apps which require large files to be stored in the application repository. -1. Although you have enabled [Git LFS](../../../topics/git/lfs/index.md#git-lfs) +1. Although you have enabled [Git LFS](../../../topics/git/lfs/index.md#git-large-file-storage-lfs) to your project, your storage has grown significantly. 1. Before you exceed available storage, you set up a limit of 10 GB per repository. diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 4fab560e081..a99897657ae 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -12,7 +12,7 @@ The logo in the header of some emails can be customized, see the [logo customiza ## Custom additional text **(PREMIUM ONLY)** -> [Introduced][ee-5031] in [GitLab Premium][eep] 10.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5031) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.7. The additional text will appear at the bottom of any email and can be used for legal/auditing/compliance reasons. @@ -22,9 +22,6 @@ legal/auditing/compliance reasons. 1. Enter your text in the **Additional text** field. 1. Click **Save**. -[ee-5031]: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5031 -[eep]: https://about.gitlab.com/pricing/ - ## Custom hostname for private commit emails > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22560) in GitLab 11.5. diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index abf3d79b9ed..3a03e46fa1f 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -33,7 +33,7 @@ authorization service. Whenever access is granted or denied this is logged in a logfile called `external-policy-access-control.log`. -Read more about logs GitLab keeps in the [omnibus documentation][omnibus-log-docs]. +Read more about logs GitLab keeps in the [omnibus documentation](https://docs.gitlab.com/omnibus/settings/logs.html). ## Configuration @@ -62,7 +62,7 @@ The available required properties are: When using TLS Authentication with a self signed certificate, the CA certificate needs to be trusted by the openssl installation. When using GitLab installed using Omnibus, learn to install a custom CA in the -[omnibus documentation][omnibus-ssl-docs]. Alternatively learn where to install +[omnibus documentation](https://docs.gitlab.com/omnibus/settings/ssl.html). Alternatively learn where to install custom certificates using `openssl version -d`. ## How it works @@ -127,6 +127,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[omnibus-ssl-docs]: https://docs.gitlab.com/omnibus/settings/ssl.html -[omnibus-log-docs]: https://docs.gitlab.com/omnibus/settings/logs.html diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index f0e7bf272a7..761d640c535 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -25,7 +25,7 @@ Access the default page for admin area settings by navigating to | [Terms of Service and Privacy Policy](terms.md) | Include a Terms of Service agreement and Privacy Policy that all users must accept. | | [External Authentication](external_authorization.md#configuration) | External Classification Policy Authorization | | [Web terminal](../../../administration/integration/terminal.md#limiting-websocket-connection-time) | Set max session time for web terminal. | -| [Web IDE](../../project/web_ide/index.md#enabling-client-side-evaluation) | Manage Web IDE Features. | +| [Web IDE](../../project/web_ide/index.md#enabling-live-preview) | Manage Web IDE Features. | ## Integrations @@ -35,7 +35,7 @@ Access the default page for admin area settings by navigating to | [PlantUML](../../../administration/integration/plantuml.md#gitlab) | Allow rendering of PlantUML diagrams in Asciidoc documents. | | [Slack application](../../../user/project/integrations/gitlab_slack_application.md#configuration) **(FREE ONLY)** | Slack integration allows you to interact with GitLab via slash commands in a chat window. This option is only available on GitLab.com, though it may be [available for self-managed instances in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/28164). | | [Third party offers](third_party_offers.md) | Control the display of third party offers. | -| [Snowplow](../../../telemetry/index.md#enabling-tracking) | Configure the Snowplow integration. | +| [Snowplow](../../../development/telemetry/snowplow.md) | Configure the Snowplow integration. | | [Google GKE](../../project/clusters/add_gke_clusters.md) | Google GKE integration allows you to provision GKE clusters from GitLab. | | [Amazon EKS](../../project/clusters/add_eks_clusters.md) | Amazon EKS integration allows you to provision EKS clusters from GitLab. | @@ -44,7 +44,7 @@ Access the default page for admin area settings by navigating to | Option | Description | | ------ | ----------- | | [Repository mirror](visibility_and_access_controls.md#allow-mirrors-to-be-set-up-for-projects) | Configure repository mirroring. | -| [Repository storage](../../../administration/repository_storage_types.md#how-to-migrate-to-hashed-storage) | Configure storage path settings. | +| [Repository storage](../../../administration/repository_storage_types.md) | Configure storage path settings. | | Repository maintenance | ([Repository checks](../../../administration/repository_checks.md) and [Housekeeping](../../../administration/housekeeping.md)). Configure automatic Git checks and housekeeping on repositories. | | [Repository static objects](../../../administration/static_objects_external_storage.md) | Serve repository static objects (for example, archives, blobs, ...) from an external storage (for example, a CDN). | @@ -74,7 +74,6 @@ Access the default page for admin area settings by navigating to | Option | Description | | ------ | ----------- | -| [Metrics - Influx](../../../administration/monitoring/performance/gitlab_configuration.md) | Enable and configure InfluxDB metrics. | | [Metrics - Prometheus](../../../administration/monitoring/prometheus/gitlab_metrics.md) | Enable and configure Prometheus metrics. | | [Metrics - Grafana](../../../administration/monitoring/performance/grafana_configuration.md#integration-with-gitlab-ui) | Enable and configure Grafana. | | [Profiling - Performance bar](../../../administration/monitoring/performance/performance_bar.md#enable-the-performance-bar-via-the-admin-panel) | Enable access to the Performance Bar for a given group. | @@ -103,7 +102,7 @@ Access the default page for admin area settings by navigating to | Option | Description | | ------ | ----------- | | [Email](email.md) | Various email settings. | -| [Help page](../../../customization/help_message.md) | Help page text and support page url. | +| [Help page](../../../customization/help_message.md) | Help page text and support page URL. | | [Pages](../../../administration/pages/index.md#custom-domain-verification) | Size and domain settings for static websites | | [Real-time features](../../../administration/polling.md) | Change this value to influence how frequently the GitLab UI polls for updates. | | [Gitaly timeouts](gitaly_timeouts.md) | Configure Gitaly timeouts. | diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index b297d48cb0a..bef4e31259f 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -40,7 +40,7 @@ are supported: Each template must go in its respective subdirectory, have the correct extension and not be empty. So, the hierarchy should look like this: -```text +```plaintext |-- README.md |-- Dockerfile |-- custom_dockerfile.dockerfile diff --git a/doc/user/admin_area/settings/protected_paths.md b/doc/user/admin_area/settings/protected_paths.md index 56f99d3e725..0cfaf5843d0 100644 --- a/doc/user/admin_area/settings/protected_paths.md +++ b/doc/user/admin_area/settings/protected_paths.md @@ -54,24 +54,3 @@ customized on **Admin > Network > Protected Paths**, along with these options:  Requests over the rate limit are logged into `auth.log`. - -## Migrate settings from GitLab 12.3 and earlier - -Omnibus GitLab protected paths throttle is deprecated and is scheduled for removal in -GitLab 13.0. Please see the [GitLab issue](https://gitlab.com/gitlab-org/gitlab/issues/29952) and the [Omnibus GitLab issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4688) for more information. - -NOTE: **Note:** If Omnibus settings are present, applications settings will be automatically ignored to avoid generating multiple requests blocks. - -To migrate from Omnibus GitLab 12.3 and earlier settings: - -1. Customize and enable your protected paths settings by following [Configure using GitLab UI](#configure-using-gitlab-ui) section. - -1. SSH into your frontend nodes and add to `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['rack_attack_admin_area_protected_paths_enabled'] = true - ``` - -1. [Reconfigure GitLab](../../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -That's it. Protected paths throttle are now managed by GitLab admin settings. diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md index 96a20681b2f..dc23865e730 100644 --- a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -4,7 +4,7 @@ type: reference # Rate limits on issue creation -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/55241) in GitLab 12.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28129) in GitLab 12.10. This setting allows you to rate limit the requests to the issue creation endpoint. It defaults to 300 requests per minute. diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index a7a6b0ecefd..8ef5ac8dc8f 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -50,14 +50,14 @@ the minimum number of characters a user must have in their password using the Gi ## Whitelist email domains -> [Introduced][ce-598] in GitLab 7.11.0 +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/598) in GitLab 7.11.0 You can restrict users to only sign up using email addresses matching the given domains list. ## Blacklist email domains -> [Introduced][ce-5259] in GitLab 8.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5259) in GitLab 8.10. With this feature enabled, you can block email addresses of a specific domain from creating an account on your GitLab server. This is particularly useful @@ -94,6 +94,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[ce-5259]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/5259 -[ce-598]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/598 diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md index d89381fd810..77c172aa927 100644 --- a/doc/user/admin_area/settings/terms.md +++ b/doc/user/admin_area/settings/terms.md @@ -8,6 +8,8 @@ type: reference An admin can enforce acceptance of a terms of service and privacy policy. When this option is enabled, new and existing users must accept the terms. +If configured, the Terms of Service page can be viewed via `https://your-instance.com/-/users/terms` at anytime. + ## Configuration To enforce acceptance of a Terms of Service and Privacy Policy: @@ -21,7 +23,7 @@ To enforce acceptance of a Terms of Service and Privacy Policy: 1. Click **Save changes**. 1. When you are presented with the **Terms of Service** statement, click **Accept terms**. -. + For each update to the terms, a new version is stored. When a user accepts or declines the terms, GitLab will record which version they accepted or declined. diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 7869f7de1b6..f3eb094887e 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -58,75 +58,7 @@ sequenceDiagram ## Usage Ping **(CORE ONLY)** -> - [Introduced][ee-557] in GitLab Enterprise Edition 8.10. -> - More statistics [were added][ee-735] in GitLab Enterprise Edition 8.12. -> - [Moved to GitLab Core][ce-23361] in 9.1. -> - More statistics [were added][ee-6602] in GitLab Ultimate 11.2. - -GitLab sends a weekly payload containing usage data to GitLab Inc. The usage -ping uses high-level data to help our product, support, and sales teams. It does -not send any project names, usernames, or any other specific data. The -information from the usage ping is not anonymous, it is linked to the hostname -of the instance. - -You can view the exact JSON payload in the administration panel. To view the payload: - -1. Navigate to the **Admin Area > Settings > Metrics and profiling**. -1. Expand the **Usage statistics** section. -1. Click the **Preview payload** button. - -You can see how [the usage ping data maps to different stages of the product](https://gitlab.com/gitlab-data/analytics/blob/master/transform/snowflake-dbt/data/version_usage_stats_to_stage_mappings.csv). - -Usage ping is important to GitLab as we use it to calculate our [Action Monthly Active Users (AMAU)](https://about.gitlab.com/handbook/product/metrics/#action-monthly-active-users-amau) which helps us measure the success of our features. - -### Request flow example - -The following example shows a basic request/response flow between the self-managed GitLab instance, GitLab Version Application, -GitLab License Application and Salesforce: - -```mermaid -sequenceDiagram - participant GitLab instance - participant Version Application - participant License Application - participant Salesforce - GitLab instance->>Version Application: Usage Ping data - loop Process Usage Data - Version Application->>Version Application: Parse Usage Data - Version Application->>Version Application: Record Usage Data - Version Application->>Version Application: Update license ping time - end - Version Application-xLicense Application: Request Zuora subscription id - License Application-xVersion Application: Zuora subscription id - Version Application-xSalesforce: Request Zuora account id by Zuora subscription id - Salesforce-xVersion Application: Zuora account id - Version Application-xSalesforce: Usage data for the Zuora account - Version Application->>GitLab instance: Conversational Development Index -``` - -### Deactivate the usage ping - -The usage ping is opt-out. If you want to deactivate this feature, go to -the Settings page of your administration panel and uncheck the Usage ping -checkbox. - -To disable the usage ping and prevent it from being configured in future through -the administration panel, Omnibus installs can set the following in -[`gitlab.rb`](https://docs.gitlab.com/omnibus/settings/configuration.html#configuration-options): - -```ruby -gitlab_rails['usage_ping_enabled'] = false -``` - -And source installs can set the following in `gitlab.yml`: - -```yaml -production: &base - # ... - gitlab: - # ... - usage_ping_enabled: false -``` +See [Usage Ping guide](../../../development/telemetry/usage_ping.md). ## Instance statistics visibility **(CORE ONLY)** @@ -148,307 +80,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[ee-557]: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/557 -[ee-735]: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/735 -[ce-23361]: https://gitlab.com/gitlab-org/gitlab-foss/issues/23361 -[ee-6602]: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6602 - -## Usage Statistics Collected - -| Statistic | Section | Stage | Description | -|---|---|---|---| -|uuid||| -|hostname||| -|version||| -|installation_type||| -|active_user_count||| -|recorded_at||| -|edition||| -|license_md5||| -|license_id||| -|historical_max_users||| -|Name|licensee|| -|Email|licensee|| -|Company|licensee|| -|license_user_count||| -|license_starts_at||| -|license_expires_at||| -|license_plan||| -|license_trial||| -|assignee_lists|counts|| -|boards|counts|| -|ci_builds|counts|| -|ci_internal_pipelines|counts|| -|ci_external_pipelines|counts|| -|ci_pipeline_config_auto_devops|counts|| -|ci_pipeline_config_repository|counts|| -|ci_runners|counts|| -|ci_triggers|counts|| -|ci_pipeline_schedules|counts|| -|auto_devops_enabled|counts|| -|auto_devops_disabled|counts|| -|deploy_keys|counts|| -|deployments|counts|| -|successful_deployments|counts|| -|failed_deployments|counts|| -|environments|counts|| -|clusters|counts|| -|clusters_enabled|counts|| -|project_clusters_enabled|counts|| -|group_clusters_enabled|counts|| -|instance_clusters_enabled|counts|| -|clusters_disabled|counts|| -|project_clusters_disabled|counts|| -|group_clusters_disabled|counts|| -|instance_clusters_disabled|counts|| -|clusters_platforms_eks|counts|| -|clusters_platforms_gke|counts|| -|clusters_platforms_user|counts|| -|clusters_applications_helm|counts|| -|clusters_applications_ingress|counts|| -|clusters_applications_cert_managers|counts|| -|clusters_applications_crossplane|counts|| -|clusters_applications_prometheus|counts|| -|clusters_applications_runner|counts|| -|clusters_applications_knative|counts|| -|clusters_applications_elastic_stack|counts|| -|clusters_management_project|counts|| -|in_review_folder|counts|| -|grafana_integrated_projects|counts|| -|groups|counts|| -|issues|counts|| -|issues_created_from_gitlab_error_tracking_ui|counts|| -|issues_with_associated_zoom_link|counts|| -|issues_using_zoom_quick_actions|counts|| -|issues_with_embedded_grafana_charts_approx|counts|| -|issues_with_health_status|counts|| -|keys|counts|| -|label_lists|counts|| -|lfs_objects|counts|| -|milestone_lists|counts|| -|milestones|counts|| -|pages_domains|counts|| -|pool_repositories|counts|| -|projects|counts|| -|projects_imported_from_github|counts|| -|projects_with_repositories_enabled|counts|| -|projects_with_error_tracking_enabled|counts|| -|protected_branches|counts|| -|releases|counts|| -|remote_mirrors|counts|| -|snippets|counts|| -|suggestions|counts|| -|todos|counts|| -|uploads|counts|| -|web_hooks|counts|| -|projects_alerts_active|counts|| -|projects_asana_active|counts|| -|projects_assembla_active|counts|| -|projects_bamboo_active|counts|| -|projects_bugzilla_active|counts|| -|projects_buildkite_active|counts|| -|projects_campfire_active|counts|| -|projects_custom_issue_tracker_active|counts|| -|projects_discord_active|counts|| -|projects_drone_ci_active|counts|| -|projects_emails_on_push_active|counts|| -|projects_external_wiki_active|counts|| -|projects_flowdock_active|counts|| -|projects_github_active|counts|| -|projects_hangouts_chat_active|counts|| -|projects_hipchat_active|counts|| -|projects_irker_active|counts|| -|projects_jenkins_active|counts|| -|projects_jenkins_deprecated_active|counts|| -|projects_jira_active -|counts|| -|projects_mattermost_active|counts|| -|projects_mattermost_slash_commands_active|counts|| -|projects_microsoft_teams_active|counts|| -|projects_packagist_active|counts|| -|projects_pipelines_email_active|counts|| -|projects_pivotaltracker_active|counts|| -|projects_prometheus_active|counts|| -|projects_pushover_active|counts|| -|projects_redmine_active|counts|| -|projects_slack_active|counts|| -|projects_slack_slash_commands_active|counts|| -|projects_teamcity_active|counts|| -|projects_unify_circuit_active|counts|| -|projects_youtrack_active|counts|| -|projects_slack_notifications_active|counts|| -|projects_slack_slash_active|counts|| -|projects_jira_server_active|counts|| -|projects_jira_cloud_active|counts|| -|projects_jira_dvcs_cloud_active|counts|| -|projects_jira_dvcs_server_active|counts|| -|labels|counts|| -|merge_requests|counts|| -|notes|counts|| -|wiki_pages_create|counts|| -|wiki_pages_update|counts|| -|wiki_pages_delete|counts|| -|web_ide_commits|counts|| -|web_ide_views|counts|| -|web_ide_merge_requests|counts|| -|web_ide_previews|counts|| -|snippet_comment|counts|| -|commit_comment|counts|| -|merge_request_comment|counts|| -|snippet_create|counts|| -|snippet_update|counts|| -|navbar_searches|counts|| -|cycle_analytics_views|counts|| -|productivity_analytics_views|counts|| -|source_code_pushes|counts|| -|merge_request_create|counts|| -|design_management_designs_create|counts|| -|design_management_designs_update|counts|| -|design_management_designs_delete|counts|| -|licenses_list_views|counts|| -|user_preferences_group_overview_details|counts|| -|user_preferences_group_overview_security_dashboard|counts|| -|ingress_modsecurity_blocking|counts|| -|ingress_modsecurity_disabled|counts|| -|dependency_list_usages_total|counts|| -|epics|counts|| -|feature_flags|counts|| -|geo_nodes|counts|| -|incident_issues|counts|| -|ldap_group_links|counts|| -|ldap_keys|counts|| -|ldap_users|counts|| -|pod_logs_usages_total|counts|| -|projects_enforcing_code_owner_approval|counts|| -|projects_mirrored_with_pipelines_enabled|counts|| -|projects_reporting_ci_cd_back_to_github|counts|| -|projects_with_packages|counts|| -|projects_with_prometheus_alerts|counts|| -|projects_with_tracing_enabled|counts|| -|projects_with_alerts_service_enabled|counts|| -|template_repositories|counts|| -|container_scanning_jobs|counts|| -|dependency_scanning_jobs|counts|| -|license_management_jobs|counts|| -|sast_jobs|counts|| -|status_page_projects|counts|monitor| -|status_page_issues|counts|monitor| -|epics_deepest_relationship_level|counts|| -|operations_dashboard_default_dashboard|counts|| -|operations_dashboard_users_with_projects_added|counts|| -|container_registry_enabled||| -|dependency_proxy_enabled||| -|gitlab_shared_runners_enabled||| -|gravatar_enabled||| -|influxdb_metrics_enabled||| -|ldap_enabled||| -|mattermost_enabled||| -|omniauth_enabled||| -|prometheus_metrics_enabled||| -|reply_by_email_enabled||| -|signup_enabled||| -|web_ide_clientside_preview_enabled||| -|ingress_modsecurity_enabled||| -|elasticsearch_enabled||| -|license_trial_ends_on||| -|geo_enabled||| -|version|Git|| -|version|Gitaly|| -|servers|Gitaly|| -|filesystems|Gitaly|| -|enabled|gitlab_pages|| -|version|gitlab_pages|| -|adapter|database|| -|version|database|| -|average|avg_cycle_analytics - issue|| -|sd|avg_cycle_analytics - issue|| -|missing|avg_cycle_analytics - issue|| -|average|avg_cycle_analytics - plan|| -|sd|avg_cycle_analytics - plan|| -|missing|avg_cycle_analytics - plan|| -|average|avg_cycle_analytics - code|| -|sd|avg_cycle_analytics - code|| -|missing|avg_cycle_analytics - code|| -|average|avg_cycle_analytics - test|| -|sd|avg_cycle_analytics - test|| -|missing|avg_cycle_analytics - test|| -|average|avg_cycle_analytics - review|| -|sd|avg_cycle_analytics - review|| -|missing|avg_cycle_analytics - review|| -|average|avg_cycle_analytics - staging|| -|sd|avg_cycle_analytics - staging|| -|missing|avg_cycle_analytics - staging|| -|average|avg_cycle_analytics - production|| -|sd|avg_cycle_analytics - production|| -|missing|avg_cycle_analytics - production|| -|total|avg_cycle_analytics|| -|clusters_applications_cert_managers|usage_activity_by_stage|configure| -|clusters_applications_helm|usage_activity_by_stage|configure| -|clusters_applications_ingress|usage_activity_by_stage|configure| -|clusters_applications_knative|usage_activity_by_stage|configure| -|clusters_management_project|usage_activity_by_stage|configure| -|clusters_disabled|usage_activity_by_stage|configure| -|clusters_enabled|usage_activity_by_stage|configure| -|clusters_platforms_gke|usage_activity_by_stage|configure| -|clusters_platforms_eks|usage_activity_by_stage|configure| -|clusters_platforms_user|usage_activity_by_stage|configure| -|instance_clusters_disabled|usage_activity_by_stage|configure| -|instance_clusters_enabled|usage_activity_by_stage|configure| -|group_clusters_disabled|usage_activity_by_stage|configure| -|group_clusters_enabled|usage_activity_by_stage|configure| -|project_clusters_disabled|usage_activity_by_stage|configure| -|project_clusters_enabled|usage_activity_by_stage|configure| -|projects_slack_notifications_active|usage_activity_by_stage|configure| -|projects_slack_slash_active|usage_activity_by_stage|configure| -|projects_with_prometheus_alerts: 0|usage_activity_by_stage|configure| -|deploy_keys|usage_activity_by_stage|create| -|keys|usage_activity_by_stage|create| -|merge_requests|usage_activity_by_stage|create| -|projects_enforcing_code_owner_approval|usage_activity_by_stage|create| -|projects_imported_from_github|usage_activity_by_stage|create| -|projects_with_repositories_enabled|usage_activity_by_stage|create| -|protected_branches|usage_activity_by_stage|create| -|remote_mirrors|usage_activity_by_stage|create| -|snippets|usage_activity_by_stage|create| -|suggestions:|usage_activity_by_stage|create| -|groups|usage_activity_by_stage|manage| -|ldap_keys|usage_activity_by_stage|manage| -|ldap_users: 0|usage_activity_by_stage|manage| -|users_created|usage_activity_by_stage|manage| -|clusters|usage_activity_by_stage|monitor| -|clusters_applications_prometheus|usage_activity_by_stage|monitor| -|operations_dashboard_default_dashboard|usage_activity_by_stage|monitor| -|operations_dashboard_users_with_projects_added|usage_activity_by_stage|monitor| -|projects_prometheus_active|usage_activity_by_stage|monitor| -|projects_with_error_tracking_enabled|usage_activity_by_stage|monitor| -|projects_with_tracing_enabled: 0|usage_activity_by_stage|monitor| -|projects_with_packages: 0|usage_activity_by_stage|package| -|assignee_lists|usage_activity_by_stage|plan| -|epics|usage_activity_by_stage|plan| -|issues|usage_activity_by_stage|plan| -|label_lists|usage_activity_by_stage|plan| -|milestone_lists|usage_activity_by_stage|plan| -|notes|usage_activity_by_stage|plan| -|projects|usage_activity_by_stage|plan| -|projects_jira_active|usage_activity_by_stage|plan| -|projects_jira_dvcs_cloud_active|usage_activity_by_stage|plan| -|projects_jira_dvcs_server_active|usage_activity_by_stage|plan| -|service_desk_enabled_projects|usage_activity_by_stage|plan| -|service_desk_issues|usage_activity_by_stage|plan| -|todos: 0|usage_activity_by_stage|plan| -|deployments|usage_activity_by_stage|release| -|failed_deployments|usage_activity_by_stage|release| -|projects_mirrored_with_pipelines_enabled|usage_activity_by_stage|release| -|releases|usage_activity_by_stage|release| -|successful_deployments: 0|usage_activity_by_stage|release| -|user_preferences_group_overview_security_dashboard: 0|usage_activity_by_stage|secure| -|ci_builds|usage_activity_by_stage|verify| -|ci_external_pipelines|usage_activity_by_stage|verify| -|ci_internal_pipelines|usage_activity_by_stage|verify| -|ci_pipeline_config_auto_devops|usage_activity_by_stage|verify| -|ci_pipeline_config_repository|usage_activity_by_stage|verify| -|ci_pipeline_schedules|usage_activity_by_stage|verify| -|ci_pipelines|usage_activity_by_stage|verify| -|ci_triggers|usage_activity_by_stage|verify| -|clusters_applications_runner|usage_activity_by_stage|verify| -|projects_reporting_ci_cd_back_to_github: 0|usage_activity_by_stage|verify| diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index f827fed833b..95748f68a55 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -28,6 +28,21 @@ For more details, see [Protected branches](../../project/protected_branches.md). To change this setting for a specific group, see [Default branch protection for groups](../../group/index.md#changing-the-default-branch-protection-of-a-group) +### Disable group owners from updating default branch protection **(PREMIUM ONLY)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211944) in GitLab 13.0. + +By default, group owners are allowed to override the branch protection set at the global level. + +In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can disable this privilege of group owners. + +To do this: + +1. Uncheck the **Allow owners to manage default branch protection per group** checkbox. + +NOTE: **Note:** +GitLab administrators can still update the default branch protection of a group. + ## Default project creation protection Project creation protection specifies which roles can create projects. @@ -62,6 +77,13 @@ To change this period: 1. Select the desired option. 1. Click **Save changes**. +### Override default deletion adjourned period + +Alternatively, projects that are marked for removal can be deleted immediately. To do so: + +1. [Restore the project](../../project/settings/#restore-a-project-premium). +1. Delete the project as described in the [Administering Projects page](../../admin_area/#administering-projects). + ## Default project visibility To set the default visibility levels for new projects: diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index bb74e673b56..e0aa01c29b2 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -1,7 +1,12 @@ --- description: "Learn how long your open merge requests have spent in code review, and what distinguishes the longest-running." # Up to ~200 chars long. They will be displayed in Google Search snippets. It may help to write the page intro first, and then reuse it here. +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- + # Code Review Analytics **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/38062) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.7. diff --git a/doc/user/analytics/img/repository_analytics_v13_0.png b/doc/user/analytics/img/repository_analytics_v13_0.png Binary files differnew file mode 100644 index 00000000000..b70b63a6239 --- /dev/null +++ b/doc/user/analytics/img/repository_analytics_v13_0.png diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index b2f7da234ad..48df7bc340a 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -1,3 +1,10 @@ +--- +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Analytics ## Analytics workspace @@ -32,6 +39,6 @@ The following analytics features are available at the project level: - [Code Review](code_review_analytics.md). **(STARTER)** - [Insights](../group/insights/index.md). **(ULTIMATE)** - [Issues](../group/issues_analytics/index.md). **(PREMIUM)** -- Repository. **(STARTER)** +- [Repository](repository_analytics.md). - [Value Stream](value_stream_analytics.md), enabled with the `cycle_analytics` [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-in-development). **(STARTER)** diff --git a/doc/user/analytics/productivity_analytics.md b/doc/user/analytics/productivity_analytics.md index 0fa990150d7..d0fda61d6a5 100644 --- a/doc/user/analytics/productivity_analytics.md +++ b/doc/user/analytics/productivity_analytics.md @@ -1,3 +1,10 @@ +--- +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Productivity Analytics **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12079) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3. diff --git a/doc/user/analytics/repository_analytics.md b/doc/user/analytics/repository_analytics.md new file mode 100644 index 00000000000..17032990b09 --- /dev/null +++ b/doc/user/analytics/repository_analytics.md @@ -0,0 +1,40 @@ +--- +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# Repository Analytics + +Get high-level overview of the project's Git repository. + + + +## Availability + +Repository Analytics is part of [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-foss). It's available to anyone who has permission to clone the repository. + +The feature requires: + +- An initialized Git repository. +- At least one commit in the default branch (`master` by default). + +## Overview + +You can find Repository Analytics in the project's sidebar. To access the page, go to **{chart}** **Analytics > Repository**. + +NOTE: **Note:** +Without a Git commit in the default branch, the menu item won't be visible. + +### Charts + +The data in the charts are updated soon after each commit in the default branch. + +Available charts: + +- Programming languages used in the repository +- Commit statistics (last month) +- Commits per day of month +- Commits per weekday +- Commits per day hour (UTC) diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 32f393d342b..90a4f96f00b 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -1,3 +1,10 @@ +--- +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Value Stream Analytics > - Introduced as Cycle Analytics prior to GitLab 12.3 at the project level. @@ -11,9 +18,6 @@ spent in each stage defined in the process. For information on how to contribute to the development of Value Stream Analytics, see our [contributor documentation](../../development/value_stream_analytics.md). -NOTE: **Note:** -Use the `cycle_analytics` feature flag to enable at the group level. - Value Stream Analytics is useful in order to quickly determine the velocity of a given project. It points to bottlenecks in the development process, enabling management to uncover, triage, and identify the root cause of slowdowns in the software development life cycle. @@ -26,7 +30,7 @@ calculates a separate median for each stage. Value Stream Analytics is available: - From GitLab 12.9, at the group level via **Group > Analytics > Value Stream**. **(PREMIUM)** -- At the project level via **Project > Value Stream Analytics**. +- At the project level via **Project > Analytics > Value Stream**. There are seven stages that are tracked as part of the Value Stream Analytics calculations. @@ -69,7 +73,7 @@ Each stage of Value Stream Analytics is further described in the table below. | **Stage** | **Description** | | --------- | --------------- | -| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list](../project/issue_board.md#creating-a-new-list) created for it. | +| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list](../project/issue_board.md) created for it. | | Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. | | Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. | | Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. | @@ -165,6 +169,21 @@ development workflow. NOTE: **Note:** Customizability is [only available for group-level](https://gitlab.com/gitlab-org/gitlab/-/issues/35823#note_272558950) Value Stream Analytics. +### Stage path + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. + +Stages are visually depicted as a horizontal process flow. Selecting a stage will update the +the content below the value stream. + +This is disabled by default. If you have a self-managed instance, an +administrator can [open a Rails console](../../administration/troubleshooting/navigating_gitlab_via_rails_console.md) +and enable it with the following command: + +```ruby +Feature.enable(:value_stream_analytics_path_navigation) +``` + ### Adding a stage In the following example we're creating a new stage that measures and tracks issues from creation @@ -293,15 +312,6 @@ toggled to show data for merge requests and further refined for specific group-l By default the top group-level labels (max. 10) are pre-selected, with the ability to select up to a total of 15 labels. -### Disabling chart - -This chart is enabled by default. If you have a self-managed instance, an -administrator can open a Rails console and disable it with the following command: - -```ruby -Feature.disable(:tasks_by_type_chart) -``` - ## Permissions The current permissions on the Project Value Stream Analytics dashboard are: @@ -324,14 +334,6 @@ For Value Stream Analytics functionality introduced in GitLab 12.3 and later: - Features are available only on [Premium or Silver tiers](https://about.gitlab.com/pricing/) and above. -## Troubleshooting - -If you see an error as listed in the following table, try the noted solution: - -| Error | Solution | -|---------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| There was an error fetching the top labels. | Manually enable tasks by type feature in the [rails console](../../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session), specifically `Feature.enable(:tasks_by_type_chart)`. | - ## More resources Learn more about Value Stream Analytics in the following resources: diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 131247910ab..dfddbde379d 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -11,7 +11,7 @@ type: reference, howto The security configuration page displays the configuration state of each of the security features and can be accessed through a project's sidebar nav. - + The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines/index.md) to determine the configuration state of each feature. If a job with the expected security report artifact exists in the pipeline, diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png b/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png Binary files differdeleted file mode 100644 index 13cacc6a489..00000000000 --- a/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png b/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png Binary files differnew file mode 100644 index 00000000000..7a079a65072 --- /dev/null +++ b/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 68ad2d427dd..6e52d7dbdcf 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -22,7 +22,7 @@ GitLab checks the Container Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. - + ## Contribute your scanner @@ -59,7 +59,7 @@ To enable Container Scanning in your pipeline, you need: [predefined environment variables](../../../ci/variables/predefined_variables.md) as defined below: - ```text + ```plaintext $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA ``` @@ -103,7 +103,7 @@ The included template will: and scan it for possible vulnerabilities. The results will be saved as a -[Container Scanning report artifact](../../../ci/yaml/README.md#artifactsreportscontainer_scanning-ultimate) +[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest Container Scanning artifact available. Behind the scenes, the @@ -169,6 +169,7 @@ using environment variables. | Environment Variable | Description | Default | | ------ | ------ | ------ | +| `SECURE_ANALYZERS_PREFIX` | Set the Docker registry base address from which to download the analyzer. | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | | `KLAR_TRACE` | Set to true to enable more verbose output from klar. | `"false"` | | `CLAIR_TRACE` | Set to true to enable more verbose output from the clair server process. | `"false"` | | `DOCKER_USER` | Username for accessing a Docker registry requiring authentication. | `$CI_REGISTRY_USER` | @@ -179,11 +180,11 @@ using environment variables. | `CLAIR_VULNERABILITIES_DB_URL` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | `clair-vulnerabilities-db` | | `CLAIR_DB_CONNECTION_STRING` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | | `CI_APPLICATION_REPOSITORY` | Docker repository URL for the image to be scanned. | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | -| `CI_APPLICATION_TAG` | Docker respository tag for the image to be scanned. | `$CI_COMMIT_SHA` | +| `CI_APPLICATION_TAG` | Docker repository tag for the image to be scanned. | `$CI_COMMIT_SHA` | | `CLAIR_DB_IMAGE` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | `arminc/clair-db:latest` | | `CLAIR_DB_IMAGE_TAG` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | `latest` | | `DOCKERFILE_PATH` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner will look for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | `Dockerfile` | -| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" | +| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" | ### Overriding the Container Scanning template @@ -229,25 +230,29 @@ To use Container Scanning in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner may try to pull remote images even if a local copy is available. Set GitLab -Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) -in an offline environment if you prefer using only locally available Docker images. +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. #### Make GitLab Container Scanning analyzer images available inside your Docker registry -For Container Scanning, import and host the following images from `registry.gitlab.com` to your -offline [local Docker container registry](../../packages/container_registry/index.md): +For Container Scanning, import the following default images from `registry.gitlab.com` into your +[local Docker container registry](../../packages/container_registry/index.md): -- [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db) -- GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar` +```plaintext +registry.gitlab.com/gitlab-org/security-products/analyzers/klar +https://hub.docker.com/r/arminc/clair-db +``` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. - -Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +process by which you can import or temporarily access external resources. Note that these scanners +are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) with new definitions, so consider if you are able to make periodic updates yourself. -You can read more specific steps on how to do this [below](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). + +For more information, see [the specific steps on how to update an image with a pipeline](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -255,8 +260,6 @@ For details on saving and transporting Docker images as a file, see Docker's doc #### Set Container Scanning CI job variables to use local Container Scanner analyzers -Container Scanning can be executed on an offline GitLab Ultimate installation using the following process: - 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: ```yaml @@ -412,11 +415,11 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | | `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | @@ -435,7 +438,7 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | | `remediations` | An array of objects containing information on cured vulnerabilities along with patch diffs to apply. Empty if no remediations provided by an underlying analyzer. | | `remediations[].fixes` | An array of strings that represent references to vulnerabilities fixed by this particular remediation. | -| `remediations[].fixes[].id` | The id of a fixed vulnerability. | +| `remediations[].fixes[].id` | The ID of a fixed vulnerability. | | `remediations[].fixes[].cve` | (**DEPRECATED - use `remediations[].fixes[].id` instead**) A string value that describes a fixed vulnerability in the same format as `vulnerabilities[].cve`. | | `remediations[].summary` | Overview of how the vulnerabilities have been fixed. | | `remediations[].diff` | base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | @@ -476,7 +479,7 @@ When the GitLab Runner uses the Docker executor and NFS is used (for example, `/var/lib/docker` is on an NFS mount), Container Scanning might fail with an error like the following: -```text +```plaintext docker: Error response from daemon: failed to copy xattrs: failed to set xattr "security.selinux" on /path/to/file: operation not supported. ``` diff --git a/doc/user/application_security/dast/img/dast_all_v12_9.png b/doc/user/application_security/dast/img/dast_all_v12_9.png Binary files differdeleted file mode 100644 index 548cea3f7f9..00000000000 --- a/doc/user/application_security/dast/img/dast_all_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_all_v13_0.png b/doc/user/application_security/dast/img/dast_all_v13_0.png Binary files differnew file mode 100644 index 00000000000..7b67fc44fae --- /dev/null +++ b/doc/user/application_security/dast/img/dast_all_v13_0.png diff --git a/doc/user/application_security/dast/img/dast_single_v12_9.png b/doc/user/application_security/dast/img/dast_single_v12_9.png Binary files differdeleted file mode 100644 index a8a4b1c1d4f..00000000000 --- a/doc/user/application_security/dast/img/dast_single_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_single_v13_0.png b/doc/user/application_security/dast/img/dast_single_v13_0.png Binary files differnew file mode 100644 index 00000000000..91ed4d916ab --- /dev/null +++ b/doc/user/application_security/dast/img/dast_single_v13_0.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index abf194aae48..1d5f96d96bb 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -19,42 +19,48 @@ Dynamic Application Security Testing (DAST) comes into place. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web application(s) +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web applications for known vulnerabilities using Dynamic Application Security Testing (DAST). - You can take advantage of DAST by either [including the CI job](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). +[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate), +provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the DAST report, compares the found vulnerabilities between the source and target -branches, and shows the information right on the merge request. +branches, and shows the information on the merge request. NOTE: **Note:** This comparison logic uses only the latest pipeline executed for the target branch's base commit. Running the pipeline on any other commit has no effect on the merge request. - + -By clicking on one of the detected linked vulnerabilities, you will be able to +By clicking on one of the detected linked vulnerabilities, you can see the details and the URL(s) affected. - + [Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_Application_Security_Testing) -is using the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) +uses the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) to perform an analysis on your running web application. -By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) and will perform passive scanning only. It will not actively attack your application. - +By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) +and performs passive scanning only. It won't actively attack your application. However, DAST can be [configured](#full-scan) -to also perform a so-called "active scan". That is, attack your application and produce a more extensive security report. +to also perform an *active scan*: attack your application and produce a more extensive security report. It can be very useful combined with [Review Apps](../../../ci/review_apps/index.md). +NOTE: **Note:** +A pipeline may consist of multiple jobs, including SAST and DAST scanning. If any +job fails to finish for any reason, the security dashboard won't show DAST scanner +output. For example, if the DAST job finishes but the SAST job fails, the security +dashboard won't show DAST results. The analyzer will output an +[exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Use cases It helps you automatically find security vulnerabilities in your running web -applications while you are developing and testing your applications. +applications while you're developing and testing your applications. ## Requirements @@ -66,9 +72,8 @@ To run a DAST job, you need GitLab Runner with the For GitLab 11.9 and later, to enable DAST, you must [include](../../../ci/yaml/README.md#includetemplate) the [`DAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) -that's provided as a part of your GitLab installation. -For GitLab versions earlier than 11.9, you can copy and use the job as defined -in that template. +that's provided as a part of your GitLab installation. For GitLab versions earlier +than 11.9, you can copy and use the job as defined in that template. Add the following to your `.gitlab-ci.yml` file: @@ -85,32 +90,39 @@ There are two ways to define the URL to be scanned by DAST: 1. Set the `DAST_WEBSITE` [variable](../../../ci/yaml/README.md#variables). 1. Add it in an `environment_url.txt` file at the root of your project. - This is great for testing in dynamic environments. In order to run DAST against - an app that is dynamically created during a GitLab CI/CD pipeline, have the app - persist its domain in an `environment_url.txt` file, and DAST will - automatically parse that file to find its scan target. - You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) - of this in our Auto DevOps CI YML. + This is great for testing in dynamic environments. In order to run DAST against + an app dynamically created during a GitLab CI/CD pipeline, have the app + persist its domain in an `environment_url.txt` file, and DAST + automatically parses that file to find its scan target. + You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + of this in our Auto DevOps CI YAML. -If both values are set, the `DAST_WEBSITE` value will take precedence. +If both values are set, the `DAST_WEBSITE` value takes precedence. -The included template will create a `dast` job in your CI/CD pipeline and scan +The included template creates a `dast` job in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. -The results will be saved as a -[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate) +The results are saved as a +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) that you can later download and analyze. Due to implementation limitations we always take the latest DAST artifact available. Behind the scenes, the [GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast) is used to run the tests on the specified URL and scan it for possible vulnerabilities. -By default, the DAST template will use the latest major version of the DAST Docker image. Using the `DAST_VERSION` variable, -you can choose to automatically update DAST with new features and fixes by pinning to a major version (e.g. 1), only update fixes by pinning to a minor version (e.g. 1.6) or prevent all updates by pinning to a specific version (e.g. 1.6.4). +By default, the DAST template will use the latest major version of the DAST Docker +image. Using the `DAST_VERSION` variable, you can choose how DAST updates: + +- Automatically update DAST with new features and fixes by pinning to a major version (such as `1`). +- Only update fixes by pinning to a minor version (such as `1.6`). +- Prevent all updates by pinning to a specific version (such as `1.6.4`). + Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases) page. ### When DAST scans run -When using `DAST.gitlab-ci.yml` template, the `dast` job is run last as shown in the example below. To ensure DAST is scanning the latest code, your CI pipeline should deploy changes to the web server in one of the jobs preceeding the `dast` job. +When using `DAST.gitlab-ci.yml` template, the `dast` job is run last as shown in +the example below. To ensure DAST is scanning the latest code, your CI pipeline +should deploy changes to the web server in one of the jobs preceding the `dast` job. ```yaml stages: @@ -120,12 +132,15 @@ stages: - dast ``` -Be aware that if your pipeline is configured to deploy to the same webserver in each run, running a pipeline while another is still running, could cause a race condition -where one pipeline overwrites the code from another pipeline. The site to be scanned should be excluded from changes for the duration of a DAST scan. -The only changes to the site should be from the DAST scanner. Be aware that any changes that users, scheduled tasks, database or code changes, other pipelines, or other scanners make to +Be aware that if your pipeline is configured to deploy to the same webserver in +each run, running a pipeline while another is still running could cause a race condition +where one pipeline overwrites the code from another pipeline. The site to be scanned +should be excluded from changes for the duration of a DAST scan. +The only changes to the site should be from the DAST scanner. Be aware that any +changes that users, scheduled tasks, database changes, code changes, other pipelines, or other scanners make to the site during a scan could lead to inaccurate results. -### Authenticated scan +### Authentication It's also possible to authenticate the user before performing the DAST checks: @@ -144,12 +159,15 @@ variables: ``` The results will be saved as a -[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest DAST artifact available. DANGER: **Danger:** -**DO NOT** run an authenticated scan against a production server. When an authenticated scan is run, it may perform *any* function that the authenticated user can. This includes modifying and deleting data, submitting forms, following links, and so on. Only run an authenticated scan against a test server. +**NEVER** run an authenticated scan against a production server. When an authenticated +scan is run, it may perform *any* function that the authenticated user can. This +includes actions like modifying and deleting data, submitting forms, and following links. +Only run an authenticated scan against a test server. ### Full scan @@ -170,7 +188,8 @@ The DAST job can be run anywhere, which means you can accidentally hit live web and potentially damage them. You could even take down your production environment. For that reason, you should use domain validation. -Domain validation is not required by default. It can be required by setting the [environment variable](#available-variables) `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` to true. +Domain validation is not required by default. It can be required by setting the +[environment variable](#available-variables) `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` to `"true"`. ```yaml include: @@ -181,19 +200,23 @@ variables: DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED: "true" ``` -Since ZAP full scan actively attacks the target application, DAST sends a ping to the target (normally defined in `DAST_WEBSITE` or `environment_url.txt`) beforehand. +Since ZAP full scan actively attacks the target application, DAST sends a ping +to the target (normally defined in `DAST_WEBSITE` or `environment_url.txt`) beforehand. -If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is false or unset, the scan will _proceed_ unless the response to the ping -includes a `Gitlab-DAST-Permission` header with a value of `deny`. +- If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is `false` or unset, the scan will + proceed unless the response to the ping includes a `Gitlab-DAST-Permission` + header with a value of `deny`. +- If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is `true`, the scan will exit + unless the response to the ping includes a `Gitlab-DAST-Permission` header with + a value of `allow`. -If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is true, the scan will _exit_ unless the response to the ping -includes a `Gitlab-DAST-Permission` header with a value of `allow`. - -Here are some examples of adding the `Gitlab-DAST-Permission` header to a response in Rails, Django, and Node (with Express). +Here are some examples of adding the `Gitlab-DAST-Permission` header to a response +in Rails, Django, and Node (with Express). ##### Ruby on Rails -Here's how you would add a [custom header in Ruby on Rails](https://guides.rubyonrails.org/action_controller_overview.html#setting-custom-headers): +Here's how you would add a +[custom header in Ruby on Rails](https://guides.rubyonrails.org/action_controller_overview.html#setting-custom-headers): ```ruby class DastWebsiteTargetController < ActionController::Base @@ -207,7 +230,8 @@ end ##### Django -Here's how you would add a [custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): +Here's how you would add a +[custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): ```python class DastWebsiteTargetView(View): @@ -220,7 +244,8 @@ class DastWebsiteTargetView(View): ##### Node (with Express) -Here's how you would add a [custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append): +Here's how you would add a +[custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append): ```javascript app.get('/dast-website-target', function(req, res) { @@ -235,7 +260,8 @@ It's also possible to add the `Gitlab-DAST-Permission` header via a proxy. ###### NGINX -The following config allows NGINX to act as a reverse proxy and add the `Gitlab-DAST-Permission` [header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header): +The following configuration allows NGINX to act as a reverse proxy and add the +`Gitlab-DAST-Permission` [header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header): ```nginx # default.conf @@ -287,9 +313,9 @@ API scans support OpenAPI V2 and OpenAPI V3 specifications. You can define these #### Import API specification from a URL If your API specification is accessible at a URL, you can pass that URL in directly as the target. -The specification doesn't have to be hosted on the same host as the API being tested. +The specification does not have to be hosted on the same host as the API being tested. -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -299,9 +325,11 @@ variables: #### Import API specification from a file -If your API specification is in your repository, you can provide the specification's filename directly as the target. The specification file is expected to be in the `/zap/wrk` directory. +If your API specification is in your repository, you can provide the specification's +filename directly as the target. The specification file is expected to be in the +`/zap/wrk` directory. -```yml +```yaml dast: script: - mkdir -p /zap/wrk @@ -314,23 +342,27 @@ dast: #### Full scan -API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` environment variable. Domain validation isn't supported for full API scans. +API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` +environment variable. Domain validation is not supported for full API scans. #### Host override -Specifications often define a host, which contains a domain name and a port. The host referenced may be different than the host of the API's review instance. -This can cause incorrect URLs to be imported, or a scan on an incorrect host. Use the `DAST_API_HOST_OVERRIDE` environment variable to override these values. +Specifications often define a host, which contains a domain name and a port. The +host referenced may be different than the host of the API's review instance. +This can cause incorrect URLs to be imported, or a scan on an incorrect host. +Use the `DAST_API_HOST_OVERRIDE` environment variable to override these values. For example, with a OpenAPI V3 specification containing: -```yml +```yaml servers: - url: https://api.host.com ``` -If the test version of the API is running at `https://api-test.host.com`, then the following DAST configuration can be used: +If the test version of the API is running at `https://api-test.host.com`, then +the following DAST configuration can be used: -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -343,9 +375,11 @@ Note that `DAST_API_HOST_OVERRIDE` is only applied to specifications imported by #### Authentication using headers -Tokens in request headers are often used as a way to authenticate API requests. You can achieve this by using the `DAST_REQUEST_HEADERS` environment variable. Headers are applied to every request DAST makes. +Tokens in request headers are often used as a way to authenticate API requests. +You can achieve this by using the `DAST_REQUEST_HEADERS` environment variable. +Headers are applied to every request DAST makes. -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -404,7 +438,8 @@ don't forget to add `stage: dast` when you override the template job definition. DAST can be [configured](#customizing-the-dast-settings) using environment variables. | Environment variable | Required | Description | -|-----------------------------| ----------|--------------------------------------------------------------------------------| +|-----------------------------| -----------|--------------------------------------------------------------------------------| +| `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address from which to download the analyzer. | | `DAST_WEBSITE` | no| The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. | | `DAST_API_SPECIFICATION` | no | The API specification to import. `DAST_WEBSITE` must be specified if this is omitted. | | `DAST_AUTH_URL` | no | The authentication URL of the website to scan. Not supported for API scans. | @@ -416,14 +451,16 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_TARGET_AVAILABILITY_TIMEOUT` | no | Time limit in seconds to wait for target availability. Scan is attempted nevertheless if it runs out. Integer. Defaults to `60`. | | `DAST_FULL_SCAN_ENABLED` | no | Switches the tool to execute [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | | `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | no | Requires [domain validation](#domain-validation) when running DAST full scans. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. Not supported for API scans. | -| `DAST_AUTO_UPDATE_ADDONS` | no | Set to `false` to pin the versions of ZAProxy add-ons to those provided with the DAST image. Defaults to `true`. | +| `DAST_AUTO_UPDATE_ADDONS` | no | By default the versions of ZAP add-ons are pinned to those provided with the DAST image. Set to `true` to allow ZAP to download the latest versions. | | `DAST_API_HOST_OVERRIDE` | no | Used to override domains defined in API specification files. | | `DAST_EXCLUDE_RULES` | no | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from scans. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/master/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. | | `DAST_REQUEST_HEADERS` | no | Set to a comma-separated list of request header names and values. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | +| `DAST_ZAP_USE_AJAX_SPIDER` | no | Use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | ### DAST command-line options -Not all DAST configuration is available via environment variables. To find out all possible options, run the following configuration. +Not all DAST configuration is available via environment variables. To find out all +possible options, run the following configuration. Available command-line options will be printed to the job log: ```yaml @@ -435,7 +472,8 @@ dast: - /analyze --help ``` -You must then overwrite the `script` command to pass in the appropriate argument. For example, AJAX spidering can be enabled by using `-j`, as shown in the following configuration: +You must then overwrite the `script` command to pass in the appropriate argument. +For example, debug messages can be enabled by using `-d`, as shown in the following configuration: ```yaml include: @@ -444,14 +482,16 @@ include: dast: script: - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -j -t $DAST_WEBSITE + - /analyze -d -t $DAST_WEBSITE ``` ### Custom ZAProxy configuration The ZAProxy server contains many [useful configurable values](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_245801885). -Many key/values for `-config` remain undocumented, but there is an untested list of [possible keys](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_244981023). -Note that these options are not supported by DAST, and may break the DAST scan when used. An example of how to rewrite the Authorization header value with `TOKEN` follows: +Many key/values for `-config` remain undocumented, but there is an untested list of +[possible keys](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_244981023). +Note that these options are not supported by DAST, and may break the DAST scan +when used. An example of how to rewrite the Authorization header value with `TOKEN` follows: ```yaml include: @@ -479,50 +519,52 @@ successfully run. For more information, see [Offline environments](../offline_de To use DAST in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- Docker Container Registry with a locally available copy of the DAST [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). +- Docker Container Registry with a locally available copy of the DAST + [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the + [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner may try to pull remote images even if a local copy is available. Set GitLab -Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) -in an offline environment if you prefer using only locally available Docker images. +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab DAST analyzer images available inside your Docker registry -For DAST, import the following default DAST analyzer image from `registry.gitlab.com` to your local "offline" -registry: +For DAST, import the following default DAST analyzer image from `registry.gitlab.com` to your [local Docker container registry](../../packages/container_registry/index.md): - `registry.gitlab.com/gitlab-org/security-products/dast:latest` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. Note +that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you're able to make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on -[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), -[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), +[`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and +[`docker import`](https://docs.docker.com/engine/reference/commandline/import/). ### Set DAST CI job variables to use local DAST analyzers -1. Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer - to the DAST Docker image hosted on your local Docker container registry: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer to +the DAST Docker image hosted on your local Docker container registry: - ```yaml - include: - - template: DAST.gitlab-ci.yml - - dast: - image: registry.example.com/namespace/dast:latest - script: - - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -t $DAST_WEBSITE --auto-update-addons false -z"-silent" - ``` +```yaml +include: + - template: DAST.gitlab-ci.yml +dast: + image: registry.example.com/namespace/dast:latest +``` -The option `--auto-update-addons false` instructs ZAP not to update add-ons. +The DAST job should now use local copies of the DAST analyzers to scan your code and generate +security reports without requiring internet access. -The option `-z` passes the quoted `-silent` parameter to ZAP. The `-silent` parameter ensures ZAP -does not make any unsolicited requests including checking for updates. +Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. ## Reports @@ -530,7 +572,8 @@ The DAST job can emit various reports. ### List of URLs scanned -When DAST completes scanning, the merge request page states the number of URLs that were scanned. Click **View details** to view the web console output which includes the list of scanned URLs. +When DAST completes scanning, the merge request page states the number of URLs scanned. +Click **View details** to view the web console output which includes the list of scanned URLs.  @@ -539,9 +582,14 @@ When DAST completes scanning, the merge request page states the number of URLs t CAUTION: **Caution:** The JSON report artifacts are not a public API of DAST and their format is expected to change in the future. -The DAST tool always emits a JSON report file called `gl-dast-report.json` and sample reports can be found in the [DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect). +The DAST tool always emits a JSON report file called `gl-dast-report.json` and +sample reports can be found in the +[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect). -There are two formats of data in the JSON report that are used side by side: the proprietary ZAP format which will be eventually deprecated, and a "common" format which will be the default in the future. +There are two formats of data in the JSON report that are used side by side: + +- The proprietary ZAP format that will be eventually deprecated. +- A common format that will be the default in the future. ### Other formats @@ -574,7 +622,9 @@ vulnerabilities in your groups, projects and pipelines. Read more about the ## Bleeding-edge vulnerability definitions -ZAProxy first creates rules in the `alpha` class. After a testing period with the community, they are promoted to `beta`. DAST uses `beta` definitions by default. To request `alpha` definitions, use `-a` as shown in the following configuration: +ZAProxy first creates rules in the `alpha` class. After a testing period with the +community, they are promoted to `beta`. DAST uses `beta` definitions by default. +To request `alpha` definitions, use `-a` as shown in the following configuration: ```yaml include: @@ -608,6 +658,18 @@ Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> +## Optimizing DAST + +By default, DAST will download all artifacts defined by previous jobs in the pipeline. If +your DAST job does not rely on `environment_url.txt` to define the URL under test or any other files created +in previous jobs, we recommend you don't download artifacts. To avoid downloading +artifacts, add the following to your `gitlab-ci.yml` file: + +```json +dast: + dependencies: [] +``` + ## Troubleshooting ### Running out of memory @@ -615,14 +677,14 @@ but commented out to help encourage others to add to it in the future. --> By default, ZAProxy, which DAST relies on, is allocated memory that sums to 25% of the total memory on the host. Since it keeps most of its information in memory during a scan, -it is possible for DAST to run out of memory while scanning large applications. +it's possible for DAST to run out of memory while scanning large applications. This results in the following error: ```plaintext [zap.out] java.lang.OutOfMemoryError: Java heap space ``` -Fortunately, it is straightforward to increase the amount of memory available +Fortunately, it's straightforward to increase the amount of memory available for DAST by overwriting the `script` key in the DAST template: ```yaml diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 26352f21cfb..474f9339d0b 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -43,7 +43,7 @@ include: template: Dependency-Scanning.gitlab-ci.yml variables: - DS_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images ``` This configuration requires that your custom registry provides images for all diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning.png Binary files differdeleted file mode 100644 index 18df356f846..00000000000 --- a/doc/user/application_security/dependency_scanning/img/dependency_scanning.png +++ /dev/null diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png Binary files differnew file mode 100644 index 00000000000..9f3990df957 --- /dev/null +++ b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index cda621e61a6..53462cf232e 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -7,25 +7,24 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5105) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.7. Dependency Scanning helps to automatically find security vulnerabilities in your dependencies -while you are developing and testing your applications, for example when your +while you're developing and testing your applications, such as when your application is using an external (open source) library which is known to be vulnerable. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your dependencies for known +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your dependencies for known vulnerabilities using Dependency Scanning. All dependencies are scanned, including the transitive dependencies (also known as nested dependencies). - You can take advantage of Dependency Scanning by either [including the Dependency Scanning template](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). +the [Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) +provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the Dependency Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. - + The results are sorted by the severity of the vulnerability: @@ -38,17 +37,16 @@ The results are sorted by the severity of the vulnerability: ## Requirements -To run a Dependency Scanning job, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +To run Dependency Scanning jobs, by default, you need GitLab Runner with the +[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +If you're using the shared Runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** -If you use your own Runners, make sure that the Docker version you have installed +If you use your own Runners, make sure your installed version of Docker is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. -Privileged mode is not necessary if you've [disabled Docker in Docker for Dependency Scanning](#disabling-docker-in-docker-for-dependency-scanning) +Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for Dependency Scanning](#enabling-docker-in-docker). ## Supported languages and package managers @@ -56,16 +54,16 @@ The following languages and dependency managers are supported. | Language (package managers) | Supported | Scan tool(s) | |----------------------------- | --------- | ------------ | -| Java ([Gradle](https://gradle.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | -| PHP ([Composer](https://getcomposer.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Python ([pip](https://pip.pypa.io/en/stable/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Java ([Gradle](https://gradle.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Java ([Maven](https://maven.apache.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | +| PHP ([Composer](https://getcomposer.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Python ([pip](https://pip.pypa.io/en/stable/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Python ([Pipfile](https://pipenv.kennethreitz.org/en/latest/basics/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | | Python ([poetry](https://python-poetry.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/7006 "Support Poetry in Dependency Scanning")) | not available | -| Ruby ([gem](https://rubygems.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | -| Scala ([sbt](https://www.scala-sbt.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Go ([Go Modules](https://github.com/golang/go/wiki/Modules)) | yes ([alpha](https://gitlab.com/gitlab-org/gitlab/issues/7132)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Ruby ([gem](https://rubygems.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | +| Scala ([sbt](https://www.scala-sbt.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Go ([Go Modules](https://github.com/golang/go/wiki/Modules)) | yes ([alpha](https://gitlab.com/gitlab-org/gitlab/issues/7132)) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | ## Contribute your scanner @@ -73,7 +71,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Configuration -For GitLab 11.9 and later, to enable Dependency Scanning, you must +To enable Dependency Scanning for GitLab 11.9 and later, you must [include](../../../ci/yaml/README.md#includetemplate) the [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) that's provided as a part of your GitLab installation. @@ -87,11 +85,10 @@ include: - template: Dependency-Scanning.gitlab-ci.yml ``` -The included template will create a `dependency_scanning` job in your CI/CD +The included template will create Dependency Scanning jobs in your CI/CD pipeline and scan your project's source code for possible vulnerabilities. - The results will be saved as a -[Dependency Scanning report artifact](../../../ci/yaml/README.md#artifactsreportsdependency_scanning-ultimate) +[Dependency Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest Dependency Scanning artifact available. @@ -99,7 +96,6 @@ always take the latest Dependency Scanning artifact available. The Dependency Scanning settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. - For example: ```yaml @@ -113,23 +109,24 @@ variables: Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable will take precedence. -### Overriding the Dependency Scanning template +### Overriding Dependency Scanning jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `dependency_scanning` job -after the template inclusion and specify any additional keys under it. For example: +To override a job definition (for example, to change properties like `variables` or `dependencies`), +declare a new job with the same name as the one to override. Place this new job after the template +inclusion and specify any additional keys under it. For example, this disables `DS_REMEDIATE` for +the `gemnasium` analyzer: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml -dependency_scanning: +gemnasium-dependency_scanning: variables: - CI_DEBUG_TRACE: "true" + DS_REMEDIATE: "false" ``` ### Available variables @@ -143,19 +140,20 @@ The following variables allow configuration of global dependency scanning settin | Environment variable | Description | | --------------------------------------- |------------ | -| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_PREFIX` | **DEPRECATED:** Use `SECURE_ANALYZERS_PREFIX` instead. | | `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | -| `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#disabling-docker-in-docker-for-dependency-scanning).| +| `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. | #### Configuring Docker-in-Docker orchestrator -The following variables configure the Docker-in-Docker orchestrator. +The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). | Environment variable | Default | Description | | --------------------------------------- | ----------- | ----------- | -| `DS_ANALYZER_IMAGES` | | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGES` | | Comma-separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | | `DS_ANALYZER_IMAGE_TAG` | | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | | `DS_PULL_ANALYZER_IMAGES` | | Pull the images from the Docker registry (set to `0` to disable). | | `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, or `h`. For example, `300ms`, `1.5h`, or `2h45m`. | @@ -168,20 +166,20 @@ The following variables are used for configuring specific analyzers (used for a | Environment variable | Analyzer | Default | Description | | --------------------------------------- | ------------------ | ---------------------------- |------------ | -| `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local gemnasium database. | -| `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the gemnasium database. | +| `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local Gemnasium database. | +| `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | -| `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | +| `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | | `PIP_INDEX_URL` | `gemnasium-python` | `https://pypi.org/simple` | Base URL of Python Package Index. | -| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma separated. | +| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma-separated. | | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) | | `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)| -| `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repos](../index.md#using-private-maven-repos). | -| `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | -| `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | -| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| +| `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repos). | +| `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | +| `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | +| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| | `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | | `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` environment variable. | @@ -190,39 +188,31 @@ The following variables are used for configuring specific analyzers (used for a ### Using private Maven repos -If you have a private Maven repository which requires login credentials, +If your private Maven repository requires login credentials, you can use the `MAVEN_CLI_OPTS` environment variable. -Read more on [how to use private Maven repos](../index.md#using-private-maven-repos). +Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Disabling Docker in Docker for Dependency Scanning +### Enabling Docker-in-Docker > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12487) in GitLab Ultimate 12.5. -You can avoid the need for Docker in Docker by running the individual analyzers. -This does not require running the executor in privileged mode. For example: +If needed, you can enable Docker-in-Docker to restore the Dependency Scanning behavior that existed +prior to GitLab 13.0. Follow these steps to do so: -```yaml -include: - - template: Dependency-Scanning.gitlab-ci.yml +1. Configure GitLab Runner with Docker-in-Docker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). +1. Set the `DS_DISABLE_DIND` variable to `false`: -variables: - DS_DISABLE_DIND: "true" -``` + ```yaml + include: + - template: Dependency-Scanning.gitlab-ci.yml -This will create individual `<analyzer-name>-dependency_scanning` jobs for each analyzer that runs in your CI/CD pipeline. + variables: + DS_DISABLE_DIND: "false" + ``` -By removing Docker-in-Docker (DIND), GitLab relies on [Linguist](https://github.com/github/linguist) -to start relevant analyzers depending on the detected repository language(s) instead of the -[orchestrator](https://gitlab.com/gitlab-org/security-products/dependency-scanning/). However, there -are some differences in the way repository languages are detected between DIND and non-DIND. You can -observe these differences by checking both Linguist and the common library. For instance, Linguist -looks for `*.java` files to spin up the [gemnasium-maven](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) -image, while orchestrator only looks for the existence of `pom.xml` or `build.gradle`. GitLab uses -Linguist to detect new file types in the default branch. This means that when introducing files or -dependencies for a new language or package manager, the corresponding scans won't be triggered in -the MR and will only run on the default branch once the MR is merged. This will be addressed by -[#211702](https://gitlab.com/gitlab-org/gitlab/-/issues/211702). +This creates a single `dependency_scanning` job in your CI/CD pipeline instead of multiple +`<analyzer-name>-dependency_scanning` jobs. ## Interacting with the vulnerabilities @@ -232,9 +222,8 @@ Once a vulnerability is found, you can interact with it. Read more on how to ## Solutions for vulnerabilities (auto-remediation) Some vulnerabilities can be fixed by applying the solution that GitLab -automatically generates. - -Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). +automatically generates. Read more about the +[solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). ## Security Dashboard @@ -371,33 +360,33 @@ it highlighted: CAUTION: **Deprecation:** Beginning with GitLab 12.9, dependency scanning no longer reports `undefined` severity and confidence levels. -Here is the description of the report file structure nodes and their meaning. All fields are mandatory to be present in -the report JSON unless stated otherwise. Presence of optional fields depends on the underlying analyzers being used. +This table describes the report file structure nodes and their meaning. All fields are mandatory to be present in +the report JSON, unless stated otherwise. The presence of optional fields depends on the underlying analyzers used. | Report JSON node | Description | |------------------------------------------------------|-------------| | `version` | Report syntax version used to generate this JSON. | | `vulnerabilities` | Array of vulnerability objects. | | `vulnerabilities[].id` | Unique identifier of the vulnerability. | -| `vulnerabilities[].category` | Where this vulnerability belongs (SAST, Dependency Scanning etc.). For Dependency Scanning, it will always be `dependency_scanning`. | -| `vulnerabilities[].name` | Name of the vulnerability, this must not include the occurrence's specific information. Optional. | -| `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | +| `vulnerabilities[].category` | Where this vulnerability belongs, such as SAST or Dependency Scanning. For Dependency Scanning, it will always be `dependency_scanning`. | +| `vulnerabilities[].name` | Name of the vulnerability. Must not include the occurrence's specific information. Optional. | +| `vulnerabilities[].message` | A short text that describes the vulnerability. May include occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | -| `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | +| `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. Used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a `snake_case` string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | -| `vulnerabilities[].location.file` | Path to the dependencies file (e.g., `yarn.lock`). Optional. | +| `vulnerabilities[].location.file` | Path to the dependencies file (such as `yarn.lock`). Optional. | | `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | | `vulnerabilities[].location.dependency.package` | A node that provides the information on the package where the vulnerability is located. | | `vulnerabilities[].location.dependency.package.name` | Name of the package where the vulnerability is located. Optional. | | `vulnerabilities[].location.dependency.version` | Version of the vulnerable package. Optional. | | `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | -| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (e.g. `gemnasium` for [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/)). | +| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (such as `gemnasium` for [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/)). | | `vulnerabilities[].identifiers[].name` | Name of the identifier for display purpose. | | `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purpose. | | `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | @@ -406,10 +395,10 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | | `remediations` | An array of objects containing information on cured vulnerabilities along with patch diffs to apply. Empty if no remediations provided by an underlying analyzer. | | `remediations[].fixes` | An array of strings that represent references to vulnerabilities fixed by this particular remediation. | -| `remediations[].fixes[].id` | The id of a fixed vulnerability. | +| `remediations[].fixes[].id` | The ID of a fixed vulnerability. | | `remediations[].fixes[].cve` | (**DEPRECATED - use `remediations[].fixes[].id` instead**) A string value that describes a fixed vulnerability in the same format as `vulnerabilities[].cve`. | | `remediations[].summary` | Overview of how the vulnerabilities have been fixed. | -| `remediations[].diff` | base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | +| `remediations[].diff` | Base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | ## Versioning and release process @@ -424,32 +413,33 @@ You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security ## Running Dependency Scanning in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for dependency scannings jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). +to external resources through the internet, some adjustments are required for Dependency Scanning +jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). ### Requirements for offline Dependency Scanning Here are the requirements for using Dependency Scanning in an offline environment: -- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-dependency-scanning) +- Keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. +- Docker Container Registry with locally available copies of Dependency Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/) - _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). - _Only if scanning npm/yarn projects_: Host an offline copy of the [retire.js](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [js](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner will try to pull Docker images from the GitLab container registry even if a local +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we -recommend keeping the pull policy setting to `always` as it will better enable updated scanners to -be utilized within your CI/CD pipelines. +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab Dependency Scanning analyzer images available inside your Docker registry -For Dependency Scanning, import docker images ([supported languages and frameworks](#supported-languages-and-package-managers)) -from `registry.gitlab.com` to your offline docker registry. The Dependency Scanning analyzer -docker images are: +For Dependency Scanning with all [supported languages and frameworks](#supported-languages-and-package-managers), +import the following default Dependency Scanning analyzer images from `registry.gitlab.com` into +your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 @@ -461,39 +451,34 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. +Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you can make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). -### Set Dependency Scanning CI config for "offline" use +### Set Dependency Scanning CI job variables to use local Dependency Scanning analyzers -Below is a general `.gitlab-ci.yml` template to configure your environment for running Dependency -Scanning offline: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace +`DS_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml variables: - DS_DISABLE_DIND: "true" DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers" + GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" + GIT_SSL_NO_VERIFY: "true" ``` See explanations of the variables above in the [configuration section](#configuration). ### Specific settings for languages and package managers -For every language and package manager, add the following to the variables section of -`.gitlab-ci.yml`: - -```yaml -GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" -``` - -See the following sections for additional instructions on specific languages and package managers. +See the following sections for configuring specific languages and package managers. #### JavaScript (npm and yarn) projects @@ -515,10 +500,12 @@ BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git" #### Java (Maven) projects -When using a self-signed certificates, add the following to the variables section of`.gitlab-ci.yml`: +When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`: ```yaml -MAVEN_CLI_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"` +gemnasium-maven-dependency_scanning: + variables: + MAVEN_CLI_OPTS: "-s settings.xml -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true" ``` #### Java (Gradle) projects @@ -527,13 +514,12 @@ When using self-signed certificates, add the following job section to the `.gitl ```yaml gemnasium-maven-dependency_scanning: - variables: - before_script: - - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt - - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt + before_script: + - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt ``` -This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. +This adds the self-signed certificates of your Maven repository to the Java KeyStore of the analyzer's Docker image. #### Scala (sbt) projects @@ -541,41 +527,20 @@ When using self-signed certificates, add the following job section to the `.gitl ```yaml gemnasium-maven-dependency_scanning: - variables: - before_script: - - echo -n | openssl s_client -connect gitlab-airgap-test.us-west1-b.c.group-secure-a89fe7.internal:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt - - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt -``` - -This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. - -#### Python (pip) and Python (Pipfile) projects - -Add the following `pip.conf` to your repository to define your index URL and trust its self-signed -certificate: - -```toml -[global] -index-url = https://pypi.example.com -trusted-host = pypi.example.com -``` - -Add the following job section to `.gitlab-ci.yml`: - -```yaml -gemnasium-python-dependency_scanning: before_script: - - mkdir ~/.config/pip - - cp pip.conf ~/.config/pip/pip.conf + - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt ``` +This adds the self-signed certificates of your Maven repository to the Java KeyStore of the analyzer's Docker image. + #### Python (setuptools) -When using self-signed certificates for your private PyPi repo no extra job configuration (aside +When using self-signed certificates for your private PyPi repository, no extra job configuration (aside from the template `.gitlab-ci.yml` above) is needed. However, you must update your `setup.py` to -ensure that it can reach your private repo. Here is an example configuration: +ensure that it can reach your private repository. Here is an example configuration: -1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repo for each +1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repository for each dependency in the `install_requires` list: ```python @@ -596,11 +561,27 @@ ensure that it can reach your private repo. Here is an example configuration: setuptools.ssl_support.cert_paths = ['internal.crt'] ``` +## Limitations + +### Referencing local dependencies using a path in JavaScript projects + +The [Retire.js](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) analyzer +doesn't support dependency references made with [local paths](https://docs.npmjs.com/files/package.json#local-paths) +in the `package.json` of JavaScript projects. The dependency scan outputs the following error for +such references: + +```plaintext +ERROR: Could not find dependencies: <dependency-name>. You may need to run npm install +``` + +As a workaround, remove the [`retire.js`](analyzers.md#selecting-specific-analyzers) analyzer from +[DS_DEFAULT_ANALYZERS](#configuring-dependency-scanning). + ## Troubleshooting ### Error response from daemon: error processing tar file: docker-tar: relocation error -This error occurs when the Docker version used to run the SAST job is `19.03.0`. -You are advised to update to Docker `19.03.1` or greater. Older versions are not +This error occurs when the Docker version that runs the Dependency Scanning job is `19.03.00`. +Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/issues/13830#note_211354992 "Current SAST container fails"). diff --git a/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png b/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png Binary files differnew file mode 100644 index 00000000000..385236d08f2 --- /dev/null +++ b/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png diff --git a/doc/user/application_security/img/dismissed_info_v12_3.png b/doc/user/application_security/img/dismissed_info_v12_3.png Binary files differdeleted file mode 100644 index 92037493eaa..00000000000 --- a/doc/user/application_security/img/dismissed_info_v12_3.png +++ /dev/null diff --git a/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png b/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png Binary files differnew file mode 100644 index 00000000000..866ad74d42c --- /dev/null +++ b/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png diff --git a/doc/user/application_security/img/interactive_reports.png b/doc/user/application_security/img/interactive_reports.png Binary files differdeleted file mode 100644 index 1b2ef0d3da9..00000000000 --- a/doc/user/application_security/img/interactive_reports.png +++ /dev/null diff --git a/doc/user/application_security/img/security_configuration_page_v12_9.png b/doc/user/application_security/img/security_configuration_page_v12_9.png Binary files differdeleted file mode 100644 index a81d82e03c3..00000000000 --- a/doc/user/application_security/img/security_configuration_page_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/img/security_configuration_page_v13_1.png b/doc/user/application_security/img/security_configuration_page_v13_1.png Binary files differnew file mode 100644 index 00000000000..0147084f705 --- /dev/null +++ b/doc/user/application_security/img/security_configuration_page_v13_1.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index dadff8583db..eefa52eb5c3 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -44,6 +44,12 @@ To add Container Scanning, follow the steps listed in the [Container Scanning do To further configure any of the other scanners, refer to each scanner's documentation. +### Override the default registry base address + +By default, GitLab security scanners use `registry.gitlab.com/gitlab-org/security-products/analyzers` as the +base address for Docker images. You can override this globally by setting the variable +`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once. + ## Security scanning tools GitLab uses the following tools to scan and report known vulnerabilities found in your project. @@ -54,6 +60,7 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | +| [Secret Detection](secret_detection/index.md) **(ULTIMATE)** | Analyze Git history for leaked secrets. | | [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) **(ULTIMATE)** | Analyze source code for known vulnerabilities. | @@ -69,9 +76,10 @@ The scanning tools and vulnerabilities database are updated regularly. | [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | Currently, you do not have to update GitLab to benefit from the latest vulnerabilities definitions. -The security tools are released as Docker images. The vendored job definitions to enable them use -the `x-y-stable` image tags that get overridden each time a new release of the tools is pushed. The -Docker images are updated to match the previous GitLab releases, so users automatically get the +The security tools are released as Docker images. The vendored job definitions that enable them use +major release tags according to [Semantic Versioning](https://semver.org/). Each new release of the +tools overrides these tags. +The Docker images are updated to match the previous GitLab releases, so users automatically get the latest versions of the scanning tools without having to do anything. There are some known issues with this approach, however, and there is a [plan to resolve them](https://gitlab.com/gitlab-org/gitlab/issues/9725). @@ -80,9 +88,6 @@ with this approach, however, and there is a > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.8. -CAUTION: **Warning:** -This feature is currently [Alpha](https://about.gitlab.com/handbook/product/#alpha-beta-ga) and while you can start using it, it may receive important changes in the future. - Each security vulnerability in the merge request report or the [Security Dashboard](security_dashboard/index.md) is actionable. Click an entry to view detailed information with several options: @@ -95,25 +100,27 @@ information with several options: - [Solution](#solutions-for-vulnerabilities-auto-remediation): For some vulnerabilities, a solution is provided for how to fix the vulnerability. - + ### Dismissing a vulnerability -You can dismiss vulnerabilities by clicking the **Dismiss vulnerability** button. -This will dismiss the vulnerability and re-render it to reflect its dismissed state. -If you wish to undo this dismissal, you can click the **Undo dismiss** button. +To dismiss a vulnerability, you must set its status to Dismissed. Follow these steps to do so: + +1. Select the vulnerability in the Security Dashboard. +1. Select **Dismissed** from the **Status** selector menu at the top-right. + +You can undo this action by selecting a different status from the same menu. #### Adding a dismissal reason > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. -When dismissing a vulnerability, it's often helpful to provide a reason for doing so. -If you press the comment button next to **Dismiss vulnerability** in the modal, -a text box appears for you to add a comment with your dismissal. -Once added, you can edit or delete it. This allows you to add and update -context for a vulnerability as you learn more over time. +When dismissing a vulnerability, it's often helpful to provide a reason for doing so. Upon setting a +vulnerability's status to Dismissed, a text box appears for you to add a comment with your +dismissal. Once added, you can edit or delete it. This allows you to add and update context for a +vulnerability as you learn more over time. - + #### Dismissing multiple vulnerabilities @@ -193,9 +200,19 @@ security team when a merge request would introduce one of the following security - A security vulnerability - A software license compliance violation -This threshold is defined as `high`, `critical`, or `unknown` severity. When any vulnerabilities are -present within a merge request, an approval is required from the `Vulnerability-Check` approver -group. +The security vulnerability threshold is defined as `high`, `critical`, or `unknown` severity. The +`Vulnerability-Check` approver group must approve merge requests that contain vulnerabilities. + +When GitLab can assess vulnerability severity, the rating can be one of the following: + +- `unknown` +- `low` +- `medium` +- `high` +- `critical` + +The rating `unknown` indicates that the underlying scanner doesn't contain or provide a severity +rating. ### Enabling Security Approvals within a project @@ -209,7 +226,7 @@ Any code changes cause the approvals required to reset. An approval is required when a security report: -- Contains a new vulnerability of `high`, `critical`, or `unknown` severity. +- Contains a new vulnerability of `high`, `critical`, or `unknown` severity, regardless of dismissal. - Is not generated during pipeline execution. An approval is optional when a security report: @@ -259,7 +276,7 @@ to pass a username and password. You can set it under your project's settings so that your credentials aren't exposed in `.gitlab-ci.yml`. If the username is `myuser` and the password is `verysecret` then you would -[set the following variable](../../ci/variables/README.md#via-the-ui) +[set the following variable](../../ci/variables/README.md#create-a-custom-variable-in-the-ui) under your project's settings: | Type | Key | Value | @@ -313,7 +330,8 @@ You can do it quickly by following the hyperlink given to run a new pipeline. ### Getting error message `sast job: stage parameter should be [some stage name here]` -When including a security job template like [`SAST`](sast/index.md#configuration), +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: ```plaintext @@ -326,15 +344,115 @@ This error appears when the included job's stage (named `test`) isn't declared i To fix this issue, you can either: - Add a `test` stage in your `.gitlab-ci.yml`. -- Change the default stage of the included security jobs. For example, with `SAST`: +- Change the default stage of the included security jobs. For example, with SpotBugs (SAST): ```yaml include: template: SAST.gitlab-ci.yml - sast: + spotbugs-sast: stage: unit-tests ``` -[Learn more on overriding the SAST template](sast/index.md#overriding-the-sast-template). +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). All the security scanning tools define their stage, so this error can occur with all of them. + +### Getting error message `sast job: config key may not be used with 'rules': only/except` + +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), +the following error may occur, depending on your GitLab CI/CD configuration: + +```plaintext +Found errors in your .gitlab-ci.yml: + + jobs:sast config key may not be used with `rules`: only/except +``` + +This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) +with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#onlyexcept-basic) +To fix this issue, you must either: + +- [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). +- (Temporarily) [Pin your templates to the deprecated versions](#pin-your-templates-to-the-deprecated-versions) + +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). + +#### Transitioning your `only/except` syntax to `rules` + +When overriding the template to control job execution, previous instances of +[`only` or `except`](../../ci/yaml/README.md#onlyexcept-basic) are no longer compatible +and must be transitioned to [the `rules` syntax](../../ci/yaml/README.md#rules). + +If your override is aimed at limiting jobs to only run on `master`, the previous syntax +would look similar to: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on master or merge requests +spotbugs-sast: + only: + refs: + - master + - merge_requests +``` + +To transition the above configuration to the new `rules` syntax, the override +would be written as follows: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on master or merge requests +spotbugs-sast: + rules: + - if: $CI_COMMIT_BRANCH == "master" + - if: $CI_MERGE_REQUEST_ID +``` + +If your override is aimed at limiting jobs to only run on branches, not tags, +it would look similar to: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + except: + - tags +``` + +To transition to the new `rules` syntax, the override would be rewritten as: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + rules: + - if: $CI_COMMIT_TAG == null +``` + +[Learn more on the usage of `rules`](../../ci/yaml/README.md#rules). + +#### Pin your templates to the deprecated versions + +To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/README.md#rules). + +If you're unable to immediately update your CI configuration, there are several workarounds that +involve pinning to the previous template versions, for example: + + ```yaml + include: + remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/12-10-stable-ee/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml' + ``` + +Additionally, we provide a dedicated project containing the versioned legacy templates. +This can be useful for offline setups or anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). + +Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 61b34901849..3deb38902f2 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -61,6 +61,14 @@ Once a vulnerability is found, you can interact with it. Read more on how to Please note that in some cases the reported vulnerabilities provide metadata that can contain external links exposed in the UI. These links might not be accessible within an offline environment. +### Suggested Solutions for vulnerabilities + +The [suggested solutions](../index.md#solutions-for-vulnerabilities-auto-remediation) feature +(auto-remediation) is available for Dependency Scanning and Container Scanning, but may not work +depending on your instance's configuration. We can only suggest solutions, which are generally more +current versions that have been patched, when we are able to access up-to-date registry services +hosting the latest versions of that dependency or image. + ### Scanner signature and rule updates When connected to the internet, some scanners will reference public databases @@ -79,3 +87,4 @@ above. You can find more information at each of the pages below: - [SAST offline directions](../sast/index.md#running-sast-in-an-offline-environment) - [DAST offline directions](../dast/index.md#running-dast-in-an-offline-environment) - [License Compliance offline directions](../../compliance/license_compliance/index.md#running-license-compliance-in-an-offline-environment) +- [Dependency Scanning offline directions](../dependency_scanning/index.md#running-dependency-scanning-in-an-offline-environment) diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 9010a7cae0b..08078a66719 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -52,7 +52,7 @@ include: - template: SAST.gitlab-ci.yml variables: - SAST_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images ``` This configuration requires that your custom registry provides images for all @@ -149,7 +149,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) | End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | | Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | | End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| External id (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | | Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | diff --git a/doc/user/application_security/sast/img/sast_v12_9.png b/doc/user/application_security/sast/img/sast_v12_9.png Binary files differdeleted file mode 100644 index 3c6ee7a276b..00000000000 --- a/doc/user/application_security/sast/img/sast_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/sast/img/sast_v13_0.png b/doc/user/application_security/sast/img/sast_v13_0.png Binary files differnew file mode 100644 index 00000000000..b4aea6ea466 --- /dev/null +++ b/doc/user/application_security/sast/img/sast_v13_0.png diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a6457d58fe2..370c6d0e8e7 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -13,7 +13,7 @@ to learn how to protect your organization. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known vulnerabilities using Static Application Security Testing (SAST). You can take advantage of SAST by doing one of the following: @@ -25,7 +25,7 @@ You can take advantage of SAST by doing one of the following: GitLab checks the SAST report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. - + The results are sorted by the priority of the vulnerability: @@ -36,6 +36,9 @@ The results are sorted by the priority of the vulnerability: 1. Unknown 1. Everything else +NOTE: **Note:** +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard won't show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard won't show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Use cases - Your code has a potentially dangerous attribute in a class, or unsafe code @@ -45,19 +48,17 @@ The results are sorted by the priority of the vulnerability: ## Requirements -To run a SAST job, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +To run SAST jobs, by default, you need GitLab Runner with the +[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +If you're using the shared Runners on GitLab.com, this is enabled by default. -Privileged mode is not necessary if you've [disabled Docker in Docker -for SAST](#disabling-docker-in-docker-for-sast) +Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker). CAUTION: **Caution:** Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** -If you use your own Runners, make sure that the Docker version you have installed +If you use your own Runners, make sure the Docker version installed is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. ## Supported languages and frameworks @@ -66,9 +67,10 @@ The following table shows which languages, package managers and frameworks are s | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| -| .NET | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | | Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | -| Apex (Salesforce) | [pmd](https://pmd.github.io/pmd/index.html) | 12.1 | +| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | | C/C++ | [Flawfinder](https://dwheeler.com/flawfinder/) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | | Go | [Gosec](https://github.com/securego/gosec) | 10.7 | @@ -82,7 +84,7 @@ The following table shows which languages, package managers and frameworks are s | React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | -| TypeScript | [TSLint config security](https://github.com/webschik/tslint-config-security/) | 11.9 | +| TypeScript | [`tslint-config-security`](https://github.com/webschik/tslint-config-security/) | 11.9 | NOTE: **Note:** The Java analyzers can also be used for variants like the @@ -101,7 +103,7 @@ provided by [Auto DevOps](../../../topics/autodevops/index.md). For GitLab 11.9 and later, to enable SAST you must [include](../../../ci/yaml/README.md#includetemplate) the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) -that is provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you +provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined that template. Add the following to your `.gitlab-ci.yml` file: @@ -111,22 +113,19 @@ include: - template: SAST.gitlab-ci.yml ``` -The included template will create a `sast` job in your CI/CD pipeline and scan +The included template will create SAST jobs in your CI/CD pipeline and scan your project's source code for possible vulnerabilities. The results will be saved as a -[SAST report artifact](../../../ci/yaml/README.md#artifactsreportssast-ultimate) +[SAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssast-ultimate) that you can later download and analyze. Due to implementation limitations, we -always take the latest SAST artifact available. Behind the scenes, the -[GitLab SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) -is used to detect the languages/frameworks and in turn runs the matching scan tools. +always take the latest SAST artifact available. ### Customizing the SAST settings The SAST settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. - In the following example, we include the SAST template and at the same time we set the `SAST_GOSEC_LEVEL` variable to `2`: @@ -139,25 +138,26 @@ variables: ``` Because the template is [evaluated before](../../../ci/yaml/README.md#include) -the pipeline configuration, the last mention of the variable will take precedence. +the pipeline configuration, the last mention of the variable takes precedence. -### Overriding the SAST template +### Overriding SAST jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `sast` job after the -template inclusion and specify any additional keys under it. For example: +To override a job definition, (for example, change properties like `variables` or `dependencies`), +declare a job with the same name as the SAST job to override. Place this new job after the template +inclusion and specify any additional keys under it. For example, this enables `FAIL_NEVER` for the +`spotbugs` analyzer: ```yaml include: - template: SAST.gitlab-ci.yml -sast: +spotbugs-sast: variables: - CI_DEBUG_TRACE: "true" + FAIL_NEVER: 1 ``` ### Using environment variables to pass credentials for private repositories @@ -170,57 +170,42 @@ it via [custom environment variables](#custom-environment-variables). #### Using a variable to pass username and password to a private Maven repository -If you have a private Maven repository which requires login credentials, +If your private Maven repository requires login credentials, you can use the `MAVEN_CLI_OPTS` environment variable. -Read more on [how to use private Maven repos](../index.md#using-private-maven-repos). +Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Disabling Docker in Docker for SAST +### Enabling Docker-in-Docker -You can avoid the need for Docker in Docker by running the individual analyzers. -This does not require running the executor in privileged mode. For example: +If needed, you can enable Docker-in-Docker to restore the SAST behavior that existed prior to GitLab +13.0. Follow these steps to do so: -```yaml -include: - - template: SAST.gitlab-ci.yml +1. Configure GitLab Runner with Docker-inDocker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). +1. Set the variable `SAST_DISABLE_DIND` set to `false`: -variables: - SAST_DISABLE_DIND: "true" -``` + ```yaml + include: + - template: SAST.gitlab-ci.yml -This will create individual `<analyzer-name>-sast` jobs for each analyzer that runs in your CI/CD pipeline. + variables: + SAST_DISABLE_DIND: "false" + ``` -By removing Docker-in-Docker (DIND), GitLab relies on [Linguist](https://github.com/github/linguist) -to start relevant analyzers depending on the detected repository language(s) instead of the -[orchestrator](https://gitlab.com/gitlab-org/security-products/dependency-scanning/). However, there -are some differences in the way repository languages are detected between DIND and non-DIND. You can -observe these differences by checking both Linguist and the common library. For instance, Linguist -looks for `*.java` files to spin up the [spotbugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) -image, while orchestrator only looks for the existence of `pom.xml`, `build.xml`, `gradlew`, -`grailsw`, or `mvnw`. GitLab uses Linguist to detect new file types in the default branch. This -means that when introducing files or dependencies for a new language or package manager, the -corresponding scans won't be triggered in the MR and will only run on the default branch once the -MR is merged. This will be addressed by [#211702](https://gitlab.com/gitlab-org/gitlab/-/issues/211702). - -NOTE: **Note:** -With the current language detection logic, any new languages or frameworks introduced within the -context of a merge request don't trigger a corresponding scan. These scans only occur once the code -is committed to the default branch. +This creates a single `sast` job in your CI/CD pipeline instead of multiple `<analyzer-name>-sast` +jobs. -#### Enabling kubesec analyzer +#### Enabling Kubesec analyzer > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12752) in GitLab Ultimate 12.6. -When [Docker in Docker is disabled](#disabling-docker-in-docker-for-sast), -you will need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the -kubesec analyzer. In `.gitlab-ci.yml`, define: +You need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the +Kubesec analyzer. In `.gitlab-ci.yml`, define: ```yaml include: - template: SAST.gitlab-ci.yml variables: - SAST_DISABLE_DIND: "true" SCAN_KUBERNETES_MANIFESTS: "true" ``` @@ -246,9 +231,6 @@ stages: include: - template: SAST.gitlab-ci.yml -variables: - SAST_DISABLE_DIND: "true" - build: stage: build script: @@ -291,10 +273,11 @@ The following are Docker image-related variables. | Environment variable | Description | |------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `SAST_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | +| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `SAST_ANALYZER_IMAGE_PREFIX` | **DEPRECATED**: Use `SECURE_ANALYZERS_PREFIX` instead. | +| `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | -| `SAST_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-sast). | +| `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | #### Vulnerability filters @@ -307,25 +290,22 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre | `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | | `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | | `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | -| `SAST_GOSEC_LEVEL` | 0 | Ignore gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | -| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a gitleaks scan starts at. | -| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a gitleaks scan ends at. | -| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic gitleaks scan. | +| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | +| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | +| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | +| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | #### Docker-in-Docker orchestrator -The following variables configure the Docker-in-Docker orchestrator. +The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). | Environment variable | Default value | Description | |------------------------------------------|---------------|-------------| -| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | -| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | -| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". | -| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". | -| `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m".| - -NOTE: **Note:** -Timeout variables are not applicable for setups with [disabled Docker In Docker](index.md#disabling-docker-in-docker-for-sast). +| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). | +| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | +| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | +| `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`.| #### Analyzer settings @@ -333,25 +313,26 @@ Some analyzers can be customized with environment variables. | Environment variable | Analyzer | Description | |-----------------------------|----------|-------------| -| `SCAN_KUBERNETES_MANIFESTS` | kubesec | Set to `"true"` to scan Kubernetes manifests when [Docker in Docker](#disabling-docker-in-docker-for-sast) is disabled. | -| `ANT_HOME` | spotbugs | The `ANT_HOME` environment variable. | -| `ANT_PATH` | spotbugs | Path to the `ant` executable. | -| `GRADLE_PATH` | spotbugs | Path to the `gradle` executable. | -| `JAVA_OPTS` | spotbugs | Additional arguments for the `java` executable. | -| `JAVA_PATH` | spotbugs | Path to the `java` executable. | -| `SAST_JAVA_VERSION` | spotbugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | -| `MAVEN_CLI_OPTS` | spotbugs | Additional arguments for the `mvn` or `mvnw` executable. | -| `MAVEN_PATH` | spotbugs | Path to the `mvn` executable. | -| `MAVEN_REPO_PATH` | spotbugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | -| `SBT_PATH` | spotbugs | Path to the `sbt` executable. | -| `FAIL_NEVER` | spotbugs | Set to `1` to ignore compilation failure. | +| `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | +| `ANT_HOME` | SpotBugs | The `ANT_HOME` environment variable. | +| `ANT_PATH` | SpotBugs | Path to the `ant` executable. | +| `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | +| `JAVA_OPTS` | SpotBugs | Additional arguments for the `java` executable. | +| `JAVA_PATH` | SpotBugs | Path to the `java` executable. | +| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | +| `MAVEN_CLI_OPTS` | SpotBugs | Additional arguments for the `mvn` or `mvnw` executable. | +| `MAVEN_PATH` | SpotBugs | Path to the `mvn` executable. | +| `MAVEN_REPO_PATH` | SpotBugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | +| `SBT_PATH` | SpotBugs | Path to the `sbt` executable. | +| `FAIL_NEVER` | SpotBugs | Set to `1` to ignore compilation failure. | +| `SAST_GOSEC_CONFIG` | Gosec | Path to configuration for Gosec (optional). | #### Custom environment variables > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18193) in GitLab Ultimate 12.5. In addition to the aforementioned SAST configuration variables, -all [custom environment variables](../../../ci/variables/README.md#creating-a-custom-environment-variable) are propagated +all [custom environment variables](../../../ci/variables/README.md#custom-environment-variables) are propagated to the underlying SAST analyzer images if [the SAST vendored template](#configuration) is used. @@ -451,16 +432,16 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `version` | Report syntax version used to generate this JSON. | | `vulnerabilities` | Array of vulnerability objects. | | `vulnerabilities[].id` | Unique identifier of the vulnerability. | -| `vulnerabilities[].category` | Where this vulnerability belongs (SAST, Dependency Scanning etc.). For SAST, it will always be `sast`. | -| `vulnerabilities[].name` | Name of the vulnerability, this must not include the occurrence's specific information. Optional. | +| `vulnerabilities[].category` | Where this vulnerability belongs (such as SAST, Dependency Scanning). For SAST, it will always be `sast`. | +| `vulnerabilities[].name` | Name of the vulnerability. Must not include the occurrence's specific information. Optional. | | `vulnerabilities[].message` | A short text that describes the vulnerability, it may include the occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | | `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.file` | Path to the file where the vulnerability is located. Optional. | @@ -468,29 +449,15 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].location.end_line` | The last line of the code affected by the vulnerability. Optional. | | `vulnerabilities[].location.class` | If specified, provides the name of the class where the vulnerability is located. Optional. | | `vulnerabilities[].location.method` | If specified, provides the name of the method where the vulnerability is located. Optional. | -| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | -| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (e.g., `bandit_test_id` for [Bandit analyzer](https://wiki.openstack.org/wiki/Security/Projects/Bandit)). | +| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external databases. | +| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (like `bandit_test_id` for [Bandit analyzer](https://wiki.openstack.org/wiki/Security/Projects/Bandit)). | | `vulnerabilities[].identifiers[].name` | Name of the identifier for display purposes. | | `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purposes. | | `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | ## Secret detection -GitLab is also able to detect secrets and credentials that have been unintentionally pushed to the -repository (for example, an API key that allows write access to third-party deployment -environments). - -This check is performed by a specific analyzer during the `sast` job. It runs regardless of the programming -language of your app, and you don't need to change anything to your -CI/CD configuration file to turn it on. Results are available in the SAST report. - -GitLab currently includes [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) checks. - -NOTE: **Note:** -The secrets analyzer will ignore "Password in URL" vulnerabilities if the password begins -with a dollar sign (`$`) as this likely indicates the password being used is an environment -variable. For example, `https://username:$password@example.com/path/to/repo` will not be -detected, whereas `https://username:password@example.com/path/to/repo` would be detected. +Learn more about [Secret Detection](../secret_detection). ## Security Dashboard @@ -503,7 +470,12 @@ vulnerabilities in your groups, projects and pipelines. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). -## Vulnerabilities database update +## Vulnerabilities database + +Vulnerabilities contained within the vulnerability database can be searched +and viewed at the [GitLab vulnerability advisory database](https://advisories.gitlab.com). + +### Vulnerabilities database update For more information about the vulnerabilities database update, check the [maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). @@ -512,29 +484,29 @@ For more information about the vulnerabilities database update, check the For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required for the SAST job to -successfully run. For more information, see [Offline environments](../offline_deployments/index.md). +run successfully. For more information, see [Offline environments](../offline_deployments/index.md). ### Requirements for offline SAST To use SAST in an offline environment, you need: -- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-sast) +- To keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - Docker Container Registry with locally available copies of SAST [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner will try to pull Docker images from the GitLab container registry even if a local +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we -recommend keeping the pull policy setting to `always` as it will better enable updated scanners to -be utilized within your CI/CD pipelines. +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab SAST analyzer images available inside your Docker registry For SAST with all [supported languages and frameworks](#supported-languages-and-frameworks), -import the following default SAST analyzer images from `registry.gitlab.com` to your local "offline" -registry: +import the following default SAST analyzer images from `registry.gitlab.com` into your +[local Docker container registry](../../packages/container_registry/index.md): ```plaintext registry.gitlab.com/gitlab-org/security-products/analyzers/bandit:2 @@ -557,7 +529,7 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/tslint:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +with new definitions, so consider if you're able to make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -565,18 +537,15 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set SAST CI job variables to use local SAST analyzers -[Override SAST environment variables](#customizing-the-sast-settings) to use to your [local container registry](./analyzers.md#using-a-custom-docker-mirror) -as the source for SAST analyzer images. - -For example, assuming a local Docker registry repository of `localhost:5000/analyzers`: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace +`SAST_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: SAST.gitlab-ci.yml variables: - SAST_ANALYZER_IMAGE_PREFIX: "localhost:5000/analyzers" - SAST_DISABLE_DIND: "true" + SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" ``` The SAST job should now use local copies of the SAST analyzers to scan your code and generate @@ -586,7 +555,7 @@ security reports without requiring internet access. ### Error response from daemon: error processing tar file: docker-tar: relocation error -This error occurs when the Docker version used to run the SAST job is `19.03.0`. -You are advised to update to Docker `19.03.1` or greater. Older versions are not +This error occurs when the Docker version that runs the SAST job is `19.03.0`. +Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/issues/13830#note_211354992 "Current SAST container fails"). diff --git a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png Binary files differnew file mode 100644 index 00000000000..17893610f10 --- /dev/null +++ b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md new file mode 100644 index 00000000000..74dd3c89984 --- /dev/null +++ b/doc/user/application_security/secret_detection/index.md @@ -0,0 +1,67 @@ +--- +type: reference, howto +--- + +# Secret Detection **(ULTIMATE)** + +> [Introduced](https://about.gitlab.com/releases/2019/03/22/gitlab-11-9-released/#detect-secrets-and-credentials-in-the-repository) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9. + +## Overview + +A recurring problem when developing applications is that developers may unintentionally commit +secrets and credentials to their remote repositories. If other people have access to the source, +or if the project is public, the sensitive information is then exposed and can be leveraged by +malicious users to gain access to resources like deployment environments. + +GitLab 11.9 includes a new check called Secret Detection. It scans the content of the repository +to find API keys and other information that should not be there. + +GitLab displays identified secrets as part of the SAST reports visibly in a few places: + +- [Security Dashboard](../security_dashboard/) +- Pipelines' **Security** tab +- Report in the merge request widget + + + +## Use cases + +- Detecting accidental commit of secrets like keys, passwords, and API tokens. +- Performing a single or recurring scan of the full history of your repository for secrets. + +## Configuration + +If you already have SAST enabled for your app, you don’t need to take any action to benefit from this +new feature. It is also included in the Auto DevOps default configuration. + +Secret Detection is performed by a [specific analyzer](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L180) +during the `sast` job. It runs regardless of the programming +language of your app, and you don't need to change your +CI/CD configuration file to enable it. Results are available in the SAST report. + +The Secret Detection analyzer includes [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) checks. + +NOTE: **Note:** +The Secret Detection analyzer will ignore "Password in URL" vulnerabilities if the password begins +with a dollar sign (`$`) as this likely indicates the password being used is an environment +variable. For example, `https://username:$password@example.com/path/to/repo` won't be +detected, whereas `https://username:password@example.com/path/to/repo` would be detected. + +## Full History Secret Scan + +GitLab 12.11 introduced support for scanning the full history of a reposity. This new functionality +is particularly useful when you are enabling Secret Detection in a repository for the first time and you +want to perform a full secret scan. Running a secret scan on the full history can take a long time, +especially for larger repositories with lengthy Git histories. We recommend not setting this variable +as part of your normal job defintion. + +A new configuration variable ([`SAST_GITLEAKS_HISTORIC_SCAN`](../sast/#vulnerability-filters)) +can be set to change the behavior of the GitLab Secret Detection scan to run on the entire Git history of a repository. + +We have created a [short video walkthrough](https://youtu.be/wDtc_K00Y0A) showcasing how you can perform a full history secret scan. +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=wDtc_K00Y0A">Walkthrough of historical secret scan</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/wDtc_K00Y0A" frameborder="0" allowfullscreen="true"> </iframe> +</figure> diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png Binary files differdeleted file mode 100644 index c93a3ce8c35..00000000000 --- a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png Binary files differnew file mode 100644 index 00000000000..c788e2165ad --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png Binary files differnew file mode 100644 index 00000000000..77e75551bd9 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png Binary files differdeleted file mode 100644 index fd0548d0b34..00000000000 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png Binary files differnew file mode 100644 index 00000000000..a500f186c2b --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12_10.png Binary files differindex 07b41b471d4..07b41b471d4 100644 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12_10.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png Binary files differdeleted file mode 100644 index 51e80bdb50d..00000000000 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png Binary files differnew file mode 100644 index 00000000000..bb88b7f371c --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 42b28b7b9f2..2988b3642ef 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -30,7 +30,7 @@ To use the instance, group, project, or pipeline security dashboard: 1. At least one project inside a group must be configured with at least one of the [supported reports](#supported-reports). -1. The configured jobs must use the [new `reports` syntax](../../../ci/yaml/README.md#artifactsreports). +1. The configured jobs must use the [new `reports` syntax](../../../ci/pipelines/job_artifacts.md#artifactsreports). 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used. If you're using the shared Runners on GitLab.com, this is already the case. @@ -44,15 +44,17 @@ Visit the page for any pipeline which has run any of the [supported reports](#su  +NOTE: **Note:** +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Project Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6165) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.1. -At the project level, the Security Dashboard displays the latest security reports -for your project from the last successful pipeline. Use it to find and fix vulnerabilities affecting the -[default branch](../../project/repository/branches/index.md#default-branch). +At the project level, the Security Dashboard displays the latest security reports for your project. +Use it to find and fix vulnerabilities. - + ### Export vulnerabilities @@ -64,7 +66,7 @@ NOTE: **Note:** It may take several minutes for the download to start if your project consists of thousands of vulnerabilities. Do not close the page until the download finishes. - + ## Group Security Dashboard @@ -78,32 +80,27 @@ First, navigate to the Security Dashboard found under your group's Once you're on the dashboard, at the top you should see a series of filters for: +- Status - Severity -- Confidence - Report type -- Project - -To the right of the filters, you should see a **Hide dismissed** toggle button. NOTE: **Note:** -The dashboard only shows projects with [security reports](#supported-reports) enabled in a group -according to the last successful projects' pipelines. +The dashboard only shows projects with [security reports](#supported-reports) enabled in a group. - + -Selecting one or more filters will filter the results in this page. Disabling the **Hide dismissed** -toggle button will let you also see vulnerabilities that have been dismissed. +Selecting one or more filters will filter the results in this page. The main section is a list of all the vulnerabilities in the group, sorted by severity. In that list, you can see the severity of the vulnerability, its name, its confidence (likelihood of the vulnerability to be a positive one), and the project it's from. -If you hover over a row, there will appear some actions you can take: +If you hover over a row, the following actions appear: -- "More info" -- "Create issue" -- "Dismiss vulnerability" +- More info +- Create issue +- Dismiss vulnerability Next to the list is a timeline chart that shows how many open vulnerabilities your projects had at various points in time. You can filter among 30, 60, and @@ -147,7 +144,23 @@ To add projects to the dashboard: Once added, the dashboard will display the vulnerabilities found in your chosen projects. - + + +### Export vulnerabilities + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. + +You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** +button located at top right of the **Instance Security Dashboard**. After the report +is built, the CSV report downloads to your local machine. The report contains all +vulnerabilities for the projects defined in the **Instance Security Dashboard**, +as filters don't apply to the export function. + +NOTE: **Note:** +It may take several minutes for the download to start if your project contains +thousands of vulnerabilities. Do not close the page until the download finishes. + + ## Keeping the dashboards up to date diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index 482fceea680..7bd148edd15 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -21,7 +21,7 @@ The Web Application Firewall section provides metrics for the NGINX Ingress controller and ModSecurity firewall. This section has the following prerequisites: -- Project has to have at least one [environment](../../../ci/environments.md). +- Project has to have at least one [environment](../../../ci/environments/index.md). - [Web Application Firewall](../../clusters/applications.md#web-application-firewall-modsecurity) has to be enabled. - [Elastic Stack](../../clusters/applications.md#web-application-firewall-modsecurity) has to be installed. @@ -38,7 +38,7 @@ about your Ingress traffic: If a significant percentage of traffic is anomalous, you should investigate it for potential threats by -[examining the application logs](../../clusters/applications.md#web-application-firewall-modsecurity). +[examining the Web Application Firewall logs](../../clusters/applications.md#web-application-firewall-modsecurity). ## Container Network Policy @@ -48,7 +48,7 @@ The **Container Network Policy** section provides packet flow metrics for your application's Kubernetes namespace. This section has the following prerequisites: -- Your project contains at least one [environment](../../../ci/environments.md) +- Your project contains at least one [environment](../../../ci/environments/index.md) - You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd) - You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration) diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 5cb4f16e0d8..b691a97fc32 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -4,11 +4,7 @@ type: reference, howto # Standalone Vulnerability pages -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. - -CAUTION: **Warning:** -This feature is currently [Alpha](https://about.gitlab.com/handbook/product/#alpha-beta-ga). -You can begin using it, but it may receive important changes in the future. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. Each security vulnerability in the [Vulnerability List](../dependency_list/index.md) has its own standalone page. diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index 47cbc0d4a1e..9ede9d9fdef 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -1,10 +1,16 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # GitLab Managed Apps GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for [Review Apps](../../ci/review_apps/index.md) -and [deployments](../../ci/environments.md) when using [Auto DevOps](../../topics/autodevops/index.md). +and [deployments](../../ci/environments/index.md) when using [Auto DevOps](../../topics/autodevops/index.md). You can install them after you [create a cluster](../project/clusters/add_remove_clusters.md). @@ -128,9 +134,9 @@ before deploying one. NOTE: **Note:** The [`runner/gitlab-runner`](https://gitlab.com/gitlab-org/charts/gitlab-runner) -chart is used to install this application with a -[`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/runner/values.yaml) -file. Customizing installation by modifying this file is not supported. +chart is used to install this application, using +[a preconfigured `values.yaml`](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/blob/master/values.yaml) +file. Customizing the installation by modifying this file is not supported. ### Ingress @@ -314,6 +320,16 @@ To change your WAF's mode: 1. Under **Global default**, select your desired mode. 1. Click **Save changes**. +##### WAF version updates + +Enabling, disabling, or changing the logging mode for **ModSecurity** is only allowed within same version of [Ingress](#ingress) due to limitations in [Helm](https://helm.sh/) which might be overcome in future releases. + +**ModSecurity** UI controls are disabled if the version deployed differs from the one available in GitLab, while actions at the [Ingress](#ingress) level, such as uninstalling, can still be performed: + + + +Updating [Ingress](#ingress) to the most recent version enables you to take advantage of bug fixes, security fixes, and performance improvements. To update [Ingress application](#ingress), you must first uninstall it, and then re-install it as described in [Install ModSecurity](../../topics/web_application_firewall/quick_start_guide.md). + ##### Viewing Web Application Firewall traffic > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/14707) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. @@ -356,7 +372,7 @@ will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](h More information on creating executable runbooks can be found in [our Runbooks -documentation](../project/clusters/runbooks/index.md#executable-runbooks). Note that +documentation](../project/clusters/runbooks/index.md#configure-an-executable-runbook-with-gitlab). Note that Ingress must be installed and have an IP address assigned before JupyterHub can be installed. @@ -487,18 +503,25 @@ and you will have access to more advanced querying capabilities. Log data is automatically deleted after 30 days using [Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/about.html). -To enable log shipping, install Elastic Stack into the cluster with the **Install** button. +To enable log shipping: + +1. Ensure your cluster contains at least 3 nodes of instance types larger than + `f1-micro`, `g1-small`, or `n1-standard-1`. +1. Navigate to **{cloud-gear}** **Operations > Kubernetes**. +1. In **Kubernetes Cluster**, select a cluster. +1. In the **Applications** section, find **Elastic Stack** and click **Install**. NOTE: **Note:** -The [`stable/elastic-stack`](https://github.com/helm/charts/tree/master/stable/elastic-stack) +The [`gitlab/elastic-stack`](https://gitlab.com/gitlab-org/charts/elastic-stack) chart is used to install this application with a [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/blob/master/vendor/elastic_stack/values.yaml) file. NOTE: **Note:** -The chart will deploy 5 Elasticsearch nodes: 2 masters, 2 data and 1 client node, -with resource requests totalling 0.125 CPU and 4.5GB RAM. Each data node requests 1.5GB of memory, -which makes it incompatible with clusters of `f1-micro` and `g1-small` instance types. +The chart deploys 3 identical Elasticsearch pods which can't be colocated, and each +require 1 CPU and 2 GB of RAM, making them incompatible with clusters containing +fewer than 3 nodes or consisting of `f1-micro`, `g1-small`, `n1-standard-1`, or +`*-highcpu-2` instance types. NOTE: **Note:** The Elastic Stack cluster application is intended as a log aggregation solution and is not related to our @@ -517,25 +540,25 @@ Save the following to `kibana.yml`: elasticsearch: enabled: false -logstash: +filebeat: enabled: false kibana: enabled: true - env: - ELASTICSEARCH_HOSTS: http://elastic-stack-elasticsearch-client.gitlab-managed-apps.svc.cluster.local:9200 + elasticsearchHosts: http://elastic-stack-elasticsearch-master.gitlab-managed-apps.svc.cluster.local:9200 ``` Then install it on your cluster: ```shell -helm install --name kibana stable/elastic-stack --values kibana.yml +helm repo add gitlab https://charts.gitlab.io +helm install --name kibana gitlab/elastic-stack --values kibana.yml ``` -To access kibana, forward the port to your local machine: +To access Kibana, forward the port to your local machine: ```shell -kubectl port-forward svc/kibana 5601:443 +kubectl port-forward svc/kibana-kibana 5601:5601 ``` Then, you can visit Kibana at `http://localhost:5601`. @@ -556,11 +579,10 @@ To enable Fluentd: 1. Provide the host domain name or URL in **SIEM Hostname**. 1. Provide the host port number in **SIEM Port**. 1. Select a **SIEM Protocol**. -1. Check **Send ModSecurity Logs**. If you do not select this checkbox, the **Install** - button is disabled. +1. Select at least one of the available logs (such as WAF or Cilium). 1. Click **Save changes**. - + ### Future apps @@ -777,7 +799,7 @@ In order for GitLab Runner to function, you **must** specify the following: - `runnerRegistrationToken` - The registration token for adding new Runners to GitLab. This must be [retrieved from your GitLab instance](../../ci/runners/README.md). -These values can be specifed using [CI variables](../../ci/variables/README.md): +These values can be specified using [CI variables](../../ci/variables/README.md): - `GITLAB_RUNNER_GITLAB_URL` will be used for `gitlabUrl`. - `GITLAB_RUNNER_REGISTRATION_TOKEN` will be used for `runnerRegistrationToken` @@ -792,10 +814,12 @@ available configuration options. > [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/22) in GitLab 12.8. -[Cilium](https://cilium.io/) is a networking plugin for Kubernetes -that you can use to implement support for -[NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) -resources. For more information on [Network Policies](../../topics/autodevops/stages.md#network-policy), see the documentation. +[Cilium](https://cilium.io/) is a networking plugin for Kubernetes that you can use to implement +support for [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) +resources. For more information, see [Network Policies](../../topics/autodevops/stages.md#network-policy). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see the [Container Network Security Demo for GitLab 12.8](https://www.youtube.com/watch?v=pgUEdhdhoUI). Enable Cilium in the `.gitlab/managed-apps/config.yaml` file to install it: @@ -822,7 +846,8 @@ management project. Refer to the for the available configuration options. CAUTION: **Caution:** -Installation and removal of the Cilium [requires restart](https://cilium.readthedocs.io/en/stable/gettingstarted/k8s-install-gke/#restart-remaining-pods) +Installation and removal of the Cilium requires a **manual** +[restart](https://cilium.readthedocs.io/en/stable/gettingstarted/k8s-install-gke/#restart-remaining-pods) of all affected pods in all namespaces to ensure that they are [managed](https://cilium.readthedocs.io/en/stable/troubleshooting/#ensure-pod-is-managed-by-cilium) by the correct networking plugin. @@ -908,15 +933,15 @@ vault: installed: true ``` -By default you will get a basic Vault setup with no high availability nor any scalable -storage backend. This is enough for simple testing and small scale deployments, though has limits +By default you will get a basic Vault setup with no scalable +storage backend. This is enough for simple testing and small-scale deployments, though has limits to how much it can scale, and as it is a single instance deployment, you will experience downtime when upgrading the Vault application. To optimally use Vault in a production environment, it's ideal to have a good understanding of the internals of Vault and how to configure it. This can be done by reading the [the Vault documentation](https://www.vaultproject.io/docs/internals/) as well as -the Vault Helm chart [values.yaml file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). +the Vault Helm chart [`values.yaml` file](https://github.com/hashicorp/vault-helm/blob/v0.3.3/values.yaml). At a minimum you will likely set up: @@ -1009,11 +1034,11 @@ In addition, the following variables must be specified using [CI variables](../. | CI Variable | Description | |:---------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `JUPYTERHUB_PROXY_SECRET_TOKEN` | Sets [`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference.html#proxy-secrettoken). Generate using `openssl rand -hex 32`. | -| `JUPYTERHUB_COOKIE_SECRET` | Sets [`hub.cookieSecret`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference.html#hub-cookiesecret). Generate using `openssl rand -hex 32`. | +| `JUPYTERHUB_PROXY_SECRET_TOKEN` | Secure string used for signing communications from the hub. See[`proxy.secretToken`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#proxy-secrettoken). | +| `JUPYTERHUB_COOKIE_SECRET` | Secure string used for signing secure cookies. See [`hub.cookieSecret`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#hub-cookiesecret). | | `JUPYTERHUB_HOST` | Hostname used for the installation. For example, `jupyter.gitlab.example.com`. | | `JUPYTERHUB_GITLAB_HOST` | Hostname of the GitLab instance used for authentication. For example, `gitlab.example.com`. | -| `JUPYTERHUB_AUTH_CRYPTO_KEY` | Sets [`auth.state.cryptoKey`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference.html#auth-state-cryptokey). Generate using `openssl rand -hex 32`. | +| `JUPYTERHUB_AUTH_CRYPTO_KEY` | A 32-byte encryption key used to set [`auth.state.cryptoKey`](https://zero-to-jupyterhub.readthedocs.io/en/stable/reference/reference.html#auth-state-cryptokey). | | `JUPYTERHUB_AUTH_GITLAB_CLIENT_ID` | "Application ID" for the OAuth Application. | | `JUPYTERHUB_AUTH_GITLAB_CLIENT_SECRET` | "Secret" for the OAuth Application. | @@ -1042,12 +1067,12 @@ elasticStack: Elastic Stack is installed into the `gitlab-managed-apps` namespace of your cluster. -You can check the default [values.yaml](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) we set for this chart. +You can check the default [`values.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/elastic_stack/values.yaml) we set for this chart. You can customize the installation of Elastic Stack by defining `.gitlab/managed-apps/elastic-stack/values.yaml` file in your cluster management project. Refer to the -[chart](https://github.com/helm/charts/blob/master/stable/elastic-stack/values.yaml) for the +[chart](https://gitlab.com/gitlab-org/charts/elastic-stack) for the available configuration options. NOTE: **Note:** @@ -1070,7 +1095,7 @@ Crossplane: Crossplane is installed into the `gitlab-managed-apps` namespace of your cluster. You can check the default -[values.yaml](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) +[`values.yaml`](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) we set for this chart. You can customize the installation of Crossplane by defining @@ -1090,7 +1115,7 @@ Fluentd: installed: true ``` -You can also review the default values set for this chart in the [values.yaml](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. +You can also review the default values set for this chart in the [`values.yaml`](https://github.com/helm/charts/blob/master/stable/fluentd/values.yaml) file. You can customize the installation of Fluentd by defining `.gitlab/managed-apps/fluentd/values.yaml` file in your cluster management @@ -1207,7 +1232,7 @@ epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). Applications can fail with the following error: -```text +```plaintext Error: remote error: tls: bad certificate ``` diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index 4e2ae87ecb9..a9a5f768ec8 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Crossplane configuration Once Crossplane [is installed](applications.md#crossplane), it must be configured for @@ -161,7 +167,7 @@ metadata: specTemplate: writeConnectionSecretsToNamespace: gitlab-managed-apps forProvider: - databaseVersion: POSTGRES_9_6 + databaseVersion: POSTGRES_11_7 region: $REGION settings: tier: db-custom-1-3840 @@ -183,7 +189,7 @@ metadata: specTemplate: writeConnectionSecretsToNamespace: gitlab-managed-apps forProvider: - databaseVersion: POSTGRES_9_6 + databaseVersion: POSTGRES_11_7 region: $REGION settings: tier: db-custom-1-3840 diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index f83be85726a..a2adf238dda 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -1,9 +1,15 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Cluster Environments **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13392) for group-level clusters in [GitLab Premium](https://about.gitlab.com/pricing/) 12.3. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/14809) for instance-level clusters in [GitLab Premium](https://about.gitlab.com/pricing/) 12.4. -Cluster environments provide a consolidated view of which CI [environments](../../ci/environments.md) are +Cluster environments provide a consolidated view of which CI [environments](../../ci/environments/index.md) are deployed to the Kubernetes cluster and it: - Shows the project and the relevant environment related to the deployment. diff --git a/doc/user/clusters/img/fluentd_v12_10.png b/doc/user/clusters/img/fluentd_v12_10.png Binary files differdeleted file mode 100644 index e8c5c832020..00000000000 --- a/doc/user/clusters/img/fluentd_v12_10.png +++ /dev/null diff --git a/doc/user/clusters/img/fluentd_v13_0.png b/doc/user/clusters/img/fluentd_v13_0.png Binary files differnew file mode 100644 index 00000000000..edc73285238 --- /dev/null +++ b/doc/user/clusters/img/fluentd_v13_0.png diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index 2b8ed83bdb2..03b4dc45015 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Cluster management project (alpha) CAUTION: **Warning:** diff --git a/doc/user/compliance/license_compliance/img/license_compliance.png b/doc/user/compliance/license_compliance/img/license_compliance.png Binary files differdeleted file mode 100644 index cdce6b5fe38..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_add_license_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_add_license_v12_3.png Binary files differdeleted file mode 100644 index ea4db16284c..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance_add_license_v12_3.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_add_license_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_add_license_v13_0.png Binary files differnew file mode 100644 index 00000000000..992c08edcd3 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_add_license_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_compliance_decision.png b/doc/user/compliance/license_compliance/img/license_compliance_decision.png Binary files differdeleted file mode 100644 index fbf90bec7fd..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance_decision.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_decision_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_decision_v13_0.png Binary files differnew file mode 100644 index 00000000000..d6c6142c0e7 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_decision_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v12_3.png Binary files differdeleted file mode 100644 index fd519d63b3e..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v12_3.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v13_0.png Binary files differnew file mode 100644 index 00000000000..9ae59e2b96b --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_compliance_search_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_search_v12_3.png Binary files differdeleted file mode 100644 index 4a7cff2e85c..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance_search_v12_3.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_search_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_search_v13_0.png Binary files differnew file mode 100644 index 00000000000..8ee55003768 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_search_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_compliance_settings_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_settings_v12_3.png Binary files differdeleted file mode 100644 index 72d0888a9dc..00000000000 --- a/doc/user/compliance/license_compliance/img/license_compliance_settings_v12_3.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_compliance_settings_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_settings_v13_0.png Binary files differnew file mode 100644 index 00000000000..52b26abd9c5 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_settings_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_compliance_v13_0.png b/doc/user/compliance/license_compliance/img/license_compliance_v13_0.png Binary files differnew file mode 100644 index 00000000000..dc227bf05ef --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_compliance_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/license_list_v12_6.png b/doc/user/compliance/license_compliance/img/license_list_v12_6.png Binary files differdeleted file mode 100644 index 8f2b510be0d..00000000000 --- a/doc/user/compliance/license_compliance/img/license_list_v12_6.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/license_list_v13_0.png b/doc/user/compliance/license_compliance/img/license_list_v13_0.png Binary files differnew file mode 100644 index 00000000000..3964c837c6a --- /dev/null +++ b/doc/user/compliance/license_compliance/img/license_list_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v12_9.png b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v12_9.png Binary files differdeleted file mode 100644 index ad5a49eebe5..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v12_9.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png Binary files differnew file mode 100644 index 00000000000..8070e2cb1a5 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_maintainer_add_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v12_9.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v12_9.png Binary files differdeleted file mode 100644 index 4f2380a0bf6..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v12_9.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png Binary files differnew file mode 100644 index 00000000000..741d1237751 --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_maintainer_edit_v13_0.png diff --git a/doc/user/compliance/license_compliance/img/policies_v12_9.png b/doc/user/compliance/license_compliance/img/policies_v12_9.png Binary files differdeleted file mode 100644 index b3bca716ae5..00000000000 --- a/doc/user/compliance/license_compliance/img/policies_v12_9.png +++ /dev/null diff --git a/doc/user/compliance/license_compliance/img/policies_v13_0.png b/doc/user/compliance/license_compliance/img/policies_v13_0.png Binary files differnew file mode 100644 index 00000000000..4712d2b7aba --- /dev/null +++ b/doc/user/compliance/license_compliance/img/policies_v13_0.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 2e771a17163..cbabed00283 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -30,12 +30,20 @@ will be displayed in the merge request area. That is the case when you add the Consecutive merge requests will have something to compare to and the license compliance report will be shown properly. - + If you are a project or group Maintainer, you can click on a license to be given the choice to allow it or deny it. - + + +When GitLab detects a **Denied** license, you can view it in the [license list](#license-list). + + + +You can view and modify existing policies from the [policies](#policies) tab. + + ## Use cases @@ -49,19 +57,30 @@ The following languages and package managers are supported. | Language | Package managers | Scan Tool | |------------|-------------------------------------------------------------------|----------------------------------------------------------| -| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Go | [Godep](https://github.com/tools/godep), go get ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), gvt ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), glide ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), dep ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), trash ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) and govendor ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), [go mod](https://github.com/golang/go/wiki/Modules) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Go | [Godep](https://github.com/tools/godep), [go mod](https://github.com/golang/go/wiki/Modules) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Java | [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| .NET | [Nuget](https://www.nuget.org/) (.NET Framework is supported via the [mono project](https://www.mono-project.com/). Windows specific dependencies are not supported at this time.) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| .NET | [Nuget](https://www.nuget.org/) (.NET Framework is supported via the [mono project](https://www.mono-project.com/). Windows specific dependencies are not supported at this time.) |[License Finder](https://github.com/pivotal/LicenseFinder)| | Python | [pip](https://pip.pypa.io/en/stable/) (Python is supported through [requirements.txt](https://pip.readthedocs.io/en/1.1/requirements.html) and [Pipfile.lock](https://github.com/pypa/pipfile#pipfilelock).) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Ruby | [gem](https://rubygems.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Erlang | [rebar](https://www.rebar3.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage) , [CocoaPods v0.39 and below](https://cocoapods.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Elixir | [mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| C++/C | [conan](https://conan.io/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Scala | [sbt](https://www.scala-sbt.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Rust | [cargo](https://crates.io/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| PHP | [composer](https://getcomposer.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| +| Ruby | [gem](https://rubygems.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage) |[License Finder](https://github.com/pivotal/LicenseFinder)| + +### Experimental support + +The following languages and package managers are [supported experimentally](https://github.com/pivotal/LicenseFinder#experimental-project-types), +which means that the reported licenses might be incomplete or inaccurate. + +| Language | Package managers | Scan Tool | +|------------|-------------------------------------------------------------------|----------------------------------------------------------| +| JavaScript | [yarn](https://yarnpkg.com/)|[License Finder](https://github.com/pivotal/LicenseFinder)| +| Go | go get, gvt, glide, dep, trash, govendor |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Erlang | [rebar](https://www.rebar3.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Objective-C, Swift | [CocoaPods](https://cocoapods.org/) v0.39 and below |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Elixir | [mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| C++/C | [conan](https://conan.io/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Scala | [sbt](https://www.scala-sbt.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Rust | [cargo](https://crates.io/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| PHP | [composer](https://getcomposer.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| ## Requirements @@ -81,7 +100,7 @@ For GitLab versions earlier than 11.9, you can copy and use the job as defined that template. NOTE: **Note:** -In GitLab 13.0, the `License-Management.gitlab-ci.yml` template is scheduled to be removed. +GitLab 13.0 removes the `License-Management.gitlab-ci.yml` template. Use `License-Scanning.gitlab-ci.yml` instead. Add the following to your `.gitlab-ci.yml` file: @@ -96,12 +115,12 @@ and scan your dependencies to find their licenses. NOTE: **Note:** Before GitLab 12.8, the `license_scanning` job was named `license_management`. -In GitLab 13.0, the `license_management` job is scheduled to be removed completely, +GitLab 13.0 removes the `license_management` job, so you're advised to migrate to the `license_scanning` job and used the new `License-Scanning.gitlab-ci.yml` template. The results will be saved as a -[License Compliance report artifact](../../../ci/yaml/README.md#artifactsreportslicense_scanning-ultimate) +[License Compliance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportslicense_scanning-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest License Compliance artifact available. Behind the scenes, the [GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) @@ -114,15 +133,17 @@ The License Compliance settings can be changed through [environment variables](# License Compliance can be configured using environment variables. -| Environment variable | Required | Description | -|-----------------------|----------|-------------| -| `MAVEN_CLI_OPTS` | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. | -| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if your project has both Golang and Ruby code stored in different directories and you want to only scan the Ruby code, you can update your `.gitlab-ci-yml` template to specify which project directories to scan, like `LICENSE_FINDER_CLI_OPTS: '--debug --aggregate-paths=. ruby'`. | -| `LM_JAVA_VERSION` | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. | -| `LM_PYTHON_VERSION` | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. | -| `SETUP_CMD` | no | Custom setup for the dependency installation. (experimental) | -| `PIP_INDEX_URL` | no | Base URL of Python Package Index (default: `https://pypi.org/simple/`). | -| `ADDITIONAL_CA_CERT_BUNDLE` | no | Bundle of trusted CA certificates (currently supported in Python projects). | +| Environment variable | Required | Description | +|-----------------------------|----------|-------------| +| `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address to download the analyzer from. | +| `ADDITIONAL_CA_CERT_BUNDLE` | no | Bundle of trusted CA certificates (currently supported in Pip, Pipenv, Maven, Gradle, Yarn, and NPM projects). | +| `GRADLE_CLI_OPTS` | no | Additional arguments for the gradle executable. If not supplied, defaults to `--exclude-task=test`. | +| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if your project has both Golang and Ruby code stored in different directories and you want to only scan the Ruby code, you can update your `.gitlab-ci-yml` template to specify which project directories to scan, like `LICENSE_FINDER_CLI_OPTS: '--debug --aggregate-paths=. ruby'`. | +| `LM_JAVA_VERSION` | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. | +| `LM_PYTHON_VERSION` | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. | +| `MAVEN_CLI_OPTS` | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. | +| `PIP_INDEX_URL` | no | Base URL of Python Package Index (default: `https://pypi.org/simple/`). | +| `SETUP_CMD` | no | Custom setup for the dependency installation (experimental). | ### Installing custom dependencies @@ -263,21 +284,80 @@ license_scanning: The [`pip.conf`](https://pip.pypa.io/en/stable/reference/pip/) allows you to specify a list of [trusted hosts](https://pip.pypa.io/en/stable/reference/pip/#cmdoption-trusted-host): -```text +```plaintext [global] trusted-host = pypi.example.com ``` +#### Using private Python repos + +If you have a private Python repository you can use the `PIP_INDEX_URL` [environment variable](#available-variables) +to specify its location. It's also possible to provide a custom `pip.conf` for +[additional configuration](#custom-root-certificates-for-python). + +### Configuring NPM projects + +You can configure NPM projects by using an [`.npmrc`](https://docs.npmjs.com/configuring-npm/npmrc.html) +file. + +#### Using private NPM registries + +If you have a private NPM registry you can use the +[`registry`](https://docs.npmjs.com/using-npm/config#registry) +setting to specify its location. + +For example: + +```plaintext +registry = https://npm.example.com +``` + +#### Custom root certificates for NPM + +You can supply a custom root certificate to complete TLS verification by using the +`ADDITIONAL_CA_CERT_BUNDLE` [environment variable](#available-variables). + +To disable TLS verification you can provide the [`strict-ssl`](https://docs.npmjs.com/using-npm/config#strict-ssl) +setting. + +For example: + +```plaintext +strict-ssl = false +``` + +### Configuring Yarn projects + +You can configure Yarn projects by using a [`.yarnrc.yml`](https://yarnpkg.com/configuration/yarnrc) +file. + +#### Using private Yarn registries + +If you have a private Yarn registry you can use the +[`npmRegistryServer`](https://yarnpkg.com/configuration/yarnrc#npmRegistryServer) +setting to specify its location. + +For example: + +```text +npmRegistryServer: "https://npm.example.com" +``` + +#### Custom root certificates for Yarn + +You can supply a custom root certificate to complete TLS verification by using the +`ADDITIONAL_CA_CERT_BUNDLE` [environment variable](#available-variables). + ### Migration from `license_management` to `license_scanning` In GitLab 12.8 a new name for `license_management` job was introduced. This change was made to improve clarity around the purpose of the scan, which is to scan and collect the types of licenses present in a projects dependencies. -The support of `license_management` is scheduled to be dropped in GitLab 13.0. +GitLab 13.0 drops support for `license_management`. If you're using a custom setup for License Compliance, you're required to update your CI config accordingly: 1. Change the CI template to `License-Scanning.gitlab-ci.yml`. 1. Change the job name to `license_scanning` (if you mention it in `.gitlab-ci.yml`). -1. Change the artifact name to `gl-license-scanning-report.json` (if you mention it in `.gitlab-ci.yml`). +1. Change the artifact name to `license_scanning`, and the file name to `gl-license-scanning-report.json` (if you mention it in `.gitlab-ci.yml`). For example, the following `.gitlab-ci.yml`: @@ -303,11 +383,21 @@ license_scanning: license_scanning: gl-license-scanning-report.json ``` +If you use the `license_management` artifact in GitLab 13.0 or later, the License Compliance job generates this error: + +```plaintext +WARNING: Uploading artifacts to coordinator... failed id=:id responseStatus=400 Bad Request status=400 Bad Request token=:sha + +FATAL: invalid_argument +``` + +If you encounter this error, follow the instructions described in this section. + ## Running License Compliance in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required for the License Compliance job to -successfully run. +successfully run. For more information, see [Offline environments](../../application_security/offline_deployments/index.md). ### Requirements for offline License Compliance @@ -318,11 +408,11 @@ To use License Compliance in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner will try to pull Docker images from the GitLab container registry even if a local +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we -recommend leaving the pull policy set to `always`, as it better enables updated scanners to be used -within your CI/CD pipelines. +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab License Compliance analyzer images available inside your Docker registry @@ -345,10 +435,8 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set License Compliance CI job variables to use local License Compliance analyzers -Override License Compliance environment variables to use to your local container registry -as the source for License Compliance analyzer images. - -For example, this assumes a local Docker registry repository of `localhost:5000/analyzers`: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer to +the License Compliance Docker image hosted on your local Docker container registry: ```yaml include: @@ -362,8 +450,8 @@ license_scanning: The License Compliance job should now use local copies of the License Compliance analyzers to scan your code and generate security reports, without requiring internet access. -Additional [configuration](#using-private-maven-repos) may be needed for connecting to private Maven -repositories. +Additional configuration may be needed for connecting to [private Maven repositories](#using-private-maven-repos), +[private NPM registries](#using-private-npm-registries), [private Yarn registries](#using-private-yarn-registries), and [private Python repositories](#using-private-python-repos). Exact name matches are required for [project policies](#project-policies-for-license-compliance) when running in an offline environment ([see related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/212388)). @@ -384,7 +472,7 @@ To allow or deny a license: **License Compliance** section. 1. Click the **Add a license** button. -  +  1. In the **License name** dropdown, either: - Select one of the available licenses. You can search for licenses in the field @@ -398,23 +486,23 @@ To modify an existing license: 1. In the **License Compliance** list, click the **Allow/Deny** dropdown to change it to the desired status. -  +  Searching for Licenses: 1. Use the **Search** box to search for a specific license. -  +  ## License Compliance report under pipelines > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5491) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2. From your project's left sidebar, navigate to **CI/CD > Pipelines** and click on the -pipeline ID that has a `license_management` job to see the Licenses tab with the listed +pipeline ID that has a `license_scanning` job to see the Licenses tab with the listed licenses (if any). - + <!-- ## Troubleshooting @@ -447,8 +535,9 @@ in your project's sidebar, and you'll see the licenses displayed, where: - **Name:** The name of the license. - **Component:** The components which have this license. +- **Policy Violation:** The license has a [license policy](#policies) marked as **Deny**. - + ## Policies @@ -459,9 +548,9 @@ and the associated classifications for each. Policies can be configured by maintainers of the project. - - + + Developers of the project can view the policies configured in a project. - + diff --git a/doc/user/feature_highlight.md b/doc/user/feature_highlight.md index df85a129ce1..47f8671afae 100644 --- a/doc/user/feature_highlight.md +++ b/doc/user/feature_highlight.md @@ -1,6 +1,6 @@ # Feature highlight -> [Introduced][ce-16379] in GitLab 10.5 +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/16379) in GitLab 10.5 Feature highlights are represented by a pulsing blue dot. Hovering over the dot will open up callout with more information. @@ -11,5 +11,3 @@ at the bottom of the callout. There isn't a way to restore the feature highlight after it has been dismissed.  - -[ce-16379]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/16379 diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 7c005a60bea..d615b67f3d2 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -28,12 +28,12 @@ gitlab.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAA ## Mail configuration -GitLab.com sends emails from the `mg.gitlab.com` domain via [Mailgun] and has +GitLab.com sends emails from the `mg.gitlab.com` domain via [Mailgun](https://www.mailgun.com/) and has its own dedicated IP address (`198.61.254.240`). ## Alternative SSH port -GitLab.com can be reached via a [different SSH port][altssh] for `git+ssh`. +GitLab.com can be reached via a [different SSH port](https://about.gitlab.com/blog/2016/02/18/gitlab-dot-com-now-supports-an-alternate-git-plus-ssh-port/) for `git+ssh`. | Setting | Value | | --------- | ------------------- | @@ -89,7 +89,7 @@ or over the size limit, you can [reduce your repository size with Git](../projec | Repository size including LFS | 10G | Unlimited | NOTE: **Note:** -A single `git push` is limited to 5GB. LFS is not affected by this limit. +`git push` and GitLab project imports are limited to 5GB per request. Git LFS and imports other than a file upload are not affected by this limit. ## IP range @@ -115,9 +115,12 @@ A limit of: GitLab offers Linux and Windows shared runners hosted on GitLab.com for executing your pipelines. +NOTE: **Note:** +Shared Runners provided by GitLab are **not** configurable. Consider [installing your own Runner](https://docs.gitlab.com/runner/install/) if you have specific configuration needs. + ### Linux Shared Runners -Linux Shared Runners on GitLab.com run in [autoscale mode] and are powered by Google Cloud Platform. +Linux Shared Runners on GitLab.com run in [autoscale mode](https://docs.gitlab.com/runner/configuration/autoscale.html) and are powered by Google Cloud Platform. Autoscaling means reduced waiting times to spin up CI/CD jobs, and isolated VMs for each project, thus maximizing security. They're free to use for public open source projects and limited to 2000 CI minutes per month per group for private projects. More minutes @@ -133,16 +136,36 @@ The `gitlab-shared-runners-manager-X.gitlab.com` fleet of Runners are dedicated Jobs handled by the shared Runners on GitLab.com (`shared-runners-manager-X.gitlab.com`), **will be timed out after 3 hours**, regardless of the timeout configured in a -project. Check the issues [4010] and [4070] for the reference. +project. Check the issues [4010](https://gitlab.com/gitlab-com/infrastructure/issues/4010) and [4070](https://gitlab.com/gitlab-com/infrastructure/issues/4070) for the reference. Below are the shared Runners settings. | Setting | GitLab.com | Default | | ----------- | ----------------- | ---------- | -| [GitLab Runner] | [Runner versions dashboard](https://dashboards.gitlab.com/d/000000159/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light) | - | +| [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) | [Runner versions dashboard](https://dashboards.gitlab.com/d/000000159/ci?from=now-1h&to=now&refresh=5m&orgId=1&panelId=12&fullscreen&theme=light) | - | | Executor | `docker+machine` | - | | Default Docker image | `ruby:2.5` | - | -| `privileged` (run [Docker in Docker]) | `true` | `false` | +| `privileged` (run [Docker in Docker](https://hub.docker.com/_/docker/)) | `true` | `false` | + +#### Pre-clone script + +Linux Shared Runners on GitLab.com provide a way to run commands in a CI +job before the Runner attempts to run `git init` and `git fetch` to +download a GitLab repository. The +[pre_clone_script](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) +can be used for: + +- Seeding the build directory with repository data +- Sending a request to a server +- Downloading assets from a CDN +- Any other commands that must run before the `git init` + +To use this feature, define a [CI/CD variable](../../ci/variables/README.md#create-a-custom-variable-in-the-ui) called +`CI_PRE_CLONE_SCRIPT` that contains a bash script. + +[This example](../../development/pipelines.md#pre-clone-step) +demonstrates how you might use a pre-clone step to seed the build +directory. #### `config.toml` @@ -164,6 +187,7 @@ sentry_dsn = "X" request_concurrency = X url = "https://gitlab.com/" token = "SHARED_RUNNER_TOKEN" + pre_clone_script = "eval \"$CI_PRE_CLONE_SCRIPT\"" executor = "docker+machine" environment = [ "DOCKER_DRIVER=overlay2", @@ -432,7 +456,7 @@ of proposed changes can be found at ## Unicorn -GitLab.com adjusts the memory limits for the [unicorn-worker-killer][unicorn-worker-killer] gem. +GitLab.com adjusts the memory limits for the [unicorn-worker-killer](https://rubygems.org/gems/unicorn-worker-killer) gem. Base default: @@ -600,13 +624,3 @@ Service discovery: High Performance TCP/HTTP Load Balancer: - [`gitlab-cookbooks` / `gitlab-haproxy` · GitLab](https://gitlab.com/gitlab-cookbooks/gitlab-haproxy) - -[autoscale mode]: https://docs.gitlab.com/runner/configuration/autoscale.html "How Autoscale works" -[runners-post]: https://about.gitlab.com/blog/2016/04/05/shared-runners/ "Shared Runners on GitLab.com" -[GitLab Runner]: https://gitlab.com/gitlab-org/gitlab-runner -[altssh]: https://about.gitlab.com/blog/2016/02/18/gitlab-dot-com-now-supports-an-alternate-git-plus-ssh-port/ "GitLab.com now supports an alternate git+ssh port" -[docker in docker]: https://hub.docker.com/_/docker/ "Docker in Docker at DockerHub" -[mailgun]: https://www.mailgun.com/ "Mailgun website" -[unicorn-worker-killer]: https://rubygems.org/gems/unicorn-worker-killer "unicorn-worker-killer" -[4010]: https://gitlab.com/gitlab-com/infrastructure/issues/4010 "Find a good value for maximum timeout for Shared Runners" -[4070]: https://gitlab.com/gitlab-com/infrastructure/issues/4070 "Configure per-runner timeout for shared-runners-manager-X on GitLab.com" diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 224836b20b5..5e6ff980c8b 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -1,13 +1,14 @@ --- type: reference +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Group-level Kubernetes clusters > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/34758) in GitLab 11.6. -## Overview - Similar to [project-level](../../project/clusters/index.md) and [instance-level](../../instance/clusters/index.md) Kubernetes clusters, group-level Kubernetes clusters allow you to connect a Kubernetes cluster to @@ -22,47 +23,43 @@ and troubleshooting applications for your group cluster, see ## RBAC compatibility -For each project under a group with a Kubernetes cluster, GitLab will -create a restricted service account with [`edit` -privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) -in the project namespace. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/29398) in GitLab 11.4. +> - [Project namespace restriction](https://gitlab.com/gitlab-org/gitlab-foss/issues/51716) was introduced in GitLab 11.5. -NOTE: **Note:** -RBAC support was introduced in -[GitLab 11.4](https://gitlab.com/gitlab-org/gitlab-foss/issues/29398), and -Project namespace restriction was introduced in -[GitLab 11.5](https://gitlab.com/gitlab-org/gitlab-foss/issues/51716). +For each project under a group with a Kubernetes cluster, GitLab creates a restricted +service account with [`edit` privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) +in the project namespace. ## Cluster precedence -GitLab will use the project's cluster before using any cluster belonging -to the group containing the project if the project's cluster is available and not disabled. - -In the case of sub-groups, GitLab will use the cluster of the closest ancestor group +If the project's cluster is available and not disabled, GitLab uses the +project's cluster before using any cluster belonging to the group containing +the project. +In the case of sub-groups, GitLab uses the cluster of the closest ancestor group to the project, provided the cluster is not disabled. ## Multiple Kubernetes clusters **(PREMIUM)** -With GitLab Premium, you can associate more than one Kubernetes clusters to your -group. That way you can have different clusters for different environments, -like dev, staging, production, etc. +With [GitLab Premium](https://about.gitlab.com/pricing/premium/), you can associate +more than one Kubernetes cluster to your group, and maintain different clusters +for different environments, such as development, staging, and production. -Add another cluster similar to the first one and make sure to -[set an environment scope](#environment-scopes-premium) that will -differentiate the new cluster from the rest. +When adding another cluster, +[set an environment scope](#environment-scopes-premium) to help +differentiate the new cluster from your other clusters. ## GitLab-managed clusters > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 11.5. > - Became [optional](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26565) in GitLab 11.11. -You can choose to allow GitLab to manage your cluster for you. If your cluster is -managed by GitLab, resources for your projects will be automatically created. See the -[Access controls](../../project/clusters/add_remove_clusters.md#access-controls) section for details on which resources will -be created. +You can choose to allow GitLab to manage your cluster for you. If GitLab manages +your cluster, resources for your projects will be automatically created. See the +[Access controls](../../project/clusters/add_remove_clusters.md#access-controls) +section for details on which resources GitLab creates for you. -For clusters not managed by GitLab, project-specific resources will not be created -automatically. If you are using [Auto DevOps](../../../topics/autodevops/index.md) +For clusters not managed by GitLab, project-specific resources won't be created +automatically. If you're using [Auto DevOps](../../../topics/autodevops/index.md) for deployments with a cluster not managed by GitLab, you must ensure: - The project's deployment service account has permissions to deploy to @@ -72,8 +69,8 @@ for deployments with a cluster not managed by GitLab, you must ensure: `KUBE_NAMESPACE` directly is discouraged. NOTE: **Note:** -If you [install applications](#installing-applications) on your cluster, GitLab will create -the resources required to run these even if you have chosen to manage your own cluster. +If you [install applications](#installing-applications) on your cluster, GitLab creates +the resources required to run them even if you choose to manage your own cluster. ### Clearing the cluster cache @@ -86,7 +83,8 @@ your cluster, which can cause deployment jobs to fail. To clear the cache: -1. Navigate to your group’s **Kubernetes** page, and select your cluster. +1. Navigate to your group’s **{cloud-gear}** **Kubernetes** page, + and select your cluster. 1. Expand the **Advanced settings** section. 1. Click **Clear cluster cache**. @@ -105,17 +103,17 @@ The domain should have a wildcard DNS configured to the Ingress IP address. When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with -[environments](../../../ci/environments.md) similar to how the -[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables) +[environments](../../../ci/environments/index.md) similar to how the +[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. While evaluating which environment matches the environment scope of a -cluster, [cluster precedence](#cluster-precedence) will take -effect. The cluster at the project level will take precedence, followed +cluster, [cluster precedence](#cluster-precedence) takes +effect. The cluster at the project level takes precedence, followed by the closest ancestor group, followed by that groups' parent and so on. -For example, let's say we have the following Kubernetes clusters: +For example, if your project has the following Kubernetes clusters: | Cluster | Environment scope | Where | | ---------- | ------------------- | ----------| @@ -151,23 +149,22 @@ deploy to production: url: https://example.com/ ``` -The result will then be: +The result is: -- The Project cluster will be used for the `test` job. -- The Staging cluster will be used for the `deploy to staging` job. -- The Production cluster will be used for the `deploy to production` job. +- The Project cluster is used for the `test` job. +- The Staging cluster is used for the `deploy to staging` job. +- The Production cluster is used for the `deploy to production` job. ## Cluster environments **(PREMIUM)** -For a consolidated view of which CI [environments](../../../ci/environments.md) +For a consolidated view of which CI [environments](../../../ci/environments/index.md) are deployed to the Kubernetes cluster, see the documentation for [cluster environments](../../clusters/environments.md). ## Security of Runners For important information about securely configuring GitLab Runners, see -[Security of -Runners](../../project/clusters/add_remove_clusters.md#security-of-gitlab-runners) +[Security of Runners](../../project/clusters/add_remove_clusters.md#security-of-gitlab-runners) documentation for project-level clusters. ## More information diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index 1bbc40a14a4..03f0ad6ad1c 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -1,7 +1,10 @@ --- type: reference +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- - # Contribution Analytics **(STARTER)** > - Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. diff --git a/doc/user/group/epics/img/epic_view_v12.3.png b/doc/user/group/epics/img/epic_view_v12.3.png Binary files differdeleted file mode 100644 index 79758cf3d52..00000000000 --- a/doc/user/group/epics/img/epic_view_v12.3.png +++ /dev/null diff --git a/doc/user/group/epics/img/epic_view_v13.0.png b/doc/user/group/epics/img/epic_view_v13.0.png Binary files differnew file mode 100644 index 00000000000..b25a91d318a --- /dev/null +++ b/doc/user/group/epics/img/epic_view_v13.0.png diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 024f346ad47..d53acaf502a 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -5,25 +5,14 @@ type: reference, howto # Epics **(PREMIUM)** > - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.2. -> - In [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/issues/37081), single-level Epics were moved to the Premium tier. +> - Single-level Epics [were moved](https://gitlab.com/gitlab-org/gitlab/issues/37081) to [GitLab Premium](https://about.gitlab.com/pricing/) in 12.8. Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. -## Relationships between epics and issues - -The possible relationships between epics and issues are: - -- An epic is the parent of one or more issues. -- An epic is the parent of one or more child epics. For details see [Multi-level child epics](#multi-level-child-epics-ultimate). **(ULTIMATE)** - -```mermaid -graph TD - Parent_epic --> Issue1 - Parent_epic --> Child_epic - Child_epic --> Issue2 -``` +<!-- Possibly swap this file with one of a single epic --> + ## Use cases @@ -31,75 +20,37 @@ graph TD - Track when the work for the group of issues is targeted to begin, and when it's targeted to end. - Discuss and collaborate on feature ideas and scope at a high level. - - -## Creating an epic - -A paginated list of epics is available in each group from where you can create -a new epic. The list of epics includes also epics from all subgroups of the -selected group. From your group page: - -1. Go to **Epics**. -1. Click **New epic**. -1. Enter a descriptive title and click **Create epic**. - -You will be taken to the new epic where can edit the following details: - -- Title -- Description -- Start date -- Due date -- Labels - -An epic's page contains the following tabs: - -- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are shown in a tree view. - - Click on the <kbd>></kbd> beside a parent epic to reveal the child epics and issues. - - Hover over the total counts to see a breakdown of open and closed items. -- **Roadmap**: a roadmap view of child epics which have start and due dates. - - - -## Adding an issue to an epic - -You can add an existing issue to an epic, or, from an epic's page, create a new issue that's automatically added to the epic. +## Manage epics -### Adding an existing issue to an epic +To learn what you can do with an epic, see [Manage epics](manage_epics.md). Possible actions include: -Existing issues that belong to a project in an epic's group, or any of the epic's -subgroups, are eligible to be added to the epic. Newly added issues appear at the top of the list of issues in the **Epics and Issues** tab. +- [Create an epic](manage_epics.md#create-an-epic) +- [Bulk-edit epics](manage_epics.md#bulk-edit-epics) +- [Delete an epic](manage_epics.md#delete-an-epic) +- [Close an epic](manage_epics.md#close-an-epic) +- [Reopen a closed epic](manage_epics.md#reopen-a-closed-epic) +- [Go to an epic from an issue](manage_epics.md#go-to-an-epic-from-an-issue) +- [Search for an epic from epics list page](manage_epics.md#search-for-an-epic-from-epics-list-page) +- [Make an epic confidential](manage_epics.md#make-an-epic-confidential) +- [Manage issues assigned to an epic](manage_epics.md#manage-issues-assigned-to-an-epic) +- [Manage multi-level child epics **(ULTIMATE)**](manage_epics.md#manage-multi-level-child-epics-ultimate) -An epic contains a list of issues and an issue can be associated with at most -one epic. When you add an issue that's already linked to an epic, -the issue is automatically unlinked from its current parent. - -To add an issue to an epic: - -1. Click **Add an issue**. -1. Identify the issue to be added, using either of the following methods: - - Paste the link of the issue. - - Search for the desired issue by entering part of the issue's title, then selecting the desired match. ([From GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/issues/9126)) - - If there are multiple issues to be added, press <kbd>Spacebar</kbd> and repeat this step. -1. Click **Add**. - -### Creating an issue from an epic - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5419) in GitLab 12.7. - -Creating an issue from an epic enables you to maintain focus on the broader context of the epic while dividing work into smaller parts. +## Relationships between epics and issues -To create an issue from an epic: +The possible relationships between epics and issues are: -1. On the epic's page, under **Epics and Issues**, click the arrow next to **Add an issue** and select **Create new issue**. -1. Under **Title**, enter the title for the new issue. -1. From the **Project** dropdown, select the project in which the issue should be created. -1. Click **Create issue**. +- An epic is the parent of one or more issues. +- An epic is the parent of one or more child epics. For details see [Multi-level child epics](#multi-level-child-epics-ultimate). **(ULTIMATE)** -To remove an issue from an epic: +```mermaid +graph TD + Parent_epic --> Issue1 + Parent_epic --> Child_epic + Child_epic --> Issue2 +``` -1. Click on the <kbd>x</kbd> button in the epic's issue list. -1. Click **Remove** in the **Remove issue** warning message. +See [Manage issues assigned to an epic](manage_epics.md#manage-issues-assigned-to-an-epic) for steps +to add an issue to an epic, reorder issues, move issues between epics, or promote an issue to an epic. ## Issue health status in Epic tree **(ULTIMATE)** @@ -118,28 +69,15 @@ This feature comes with a feature flag enabled by default. For steps to disable > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/8333) in GitLab Ultimate 11.7. -Any epic that belongs to a group, or subgroup of the parent epic's group, is -eligible to be added. New child epics appear at the top of the list of epics in the **Epics and Issues** tab. +Any epic that belongs to a group, or subgroup of the parent epic's group, is eligible to be added. +New child epics appear at the top of the list of epics in the **Epics and Issues** tab. When you add an epic that's already linked to a parent epic, the link to its current parent is removed. -An epic can have multiple child epics with -the maximum depth being 5. - -To add a child epic to an epic: - -1. Click **Add an epic**. -1. Identify the epic to be added, using either of the following methods: - - Paste the link of the epic. - - Search for the desired issue by entering part of the epic's title, then selecting the desired match. ([From GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/issues/9126)) +An epic can have multiple child epics up to the maximum depth of five. - If there are multiple epics to be added, press <kbd>Spacebar</kbd> and repeat this step. -1. Click **Add**. - -To remove a child epic from a parent epic: - -1. Click on the <kbd>x</kbd> button in the parent epic's list of epics. -1. Click **Remove** in the **Remove epic** warning message. +See [Manage multi-level child epics](manage_epics.md#manage-multi-level-child-epics-ultimate) for +steps to create, move, reorder, or delete child epics. ## Start date and due date @@ -197,137 +135,6 @@ have a [start or due date](#start-date-and-due-date), a  ---- - -## Reordering issues and child epics - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9367) in GitLab 12.5. - -New issues and child epics are added to the top of their respective lists in the **Epics and Issues** tab. You can reorder the list of issues and the list of child epics. Issues and child epics cannot be intermingled. - -To reorder issues assigned to an epic: - -1. Go to the **Epics and Issues** tab. -1. Drag and drop issues into the desired order. - -To reorder child epics assigned to an epic: - -1. Go to the **Epics and Issues** tab. -1. Drag and drop epics into the desired order. - -## Updating epics - -### Using bulk editing - -To apply labels across multiple epics: - -1. Go to the Epics list. -1. Click **Edit epics**. - - Checkboxes will appear beside each epic. - - A sidebar on the right-hand side will appear, with an editable field for labels. -1. Check the checkbox beside each epic to be edited. -1. Select the desired labels. -1. Click **Update all**. - - - -## Deleting an epic - -NOTE: **Note:** -To delete an epic, you need to be an [Owner](../../permissions.md#group-members-permissions) of a group/subgroup. - -When inside a single epic view, click the **Delete** button to delete the epic. -A modal will pop-up to confirm your action. - -Deleting an epic releases all existing issues from their associated epic in the -system. - -## Closing and reopening epics - -### Using buttons - -Whenever you decide that there is no longer need for that epic, -close the epic using the close button: - - - -You can always reopen it using the reopen button. - - - ---- - -### Using quick actions - -You can close or reopen an epic using [Quick actions](../../project/quick_actions.md) - -## Navigating to an epic from an issue - -If an issue belongs to an epic, you can navigate to the containing epic with the -link in the issue sidebar. - - - ---- - -## Promoting an issue to an epic - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3777) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.6. -> - In [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/issues/37081), it was moved to the Premium tier. - -If you have [permissions](../../permissions.md) to close an issue and create an -epic in the parent group, you can promote an issue to an epic with the `/promote` -[quick action](../../project/quick_actions.md#quick-actions-for-issues-merge-requests-and-epics). -Only issues from projects that are in groups can be promoted. When attempting to promote a confidential -issue, a warning will display. Promoting a confidential issue to an epic will make all information -related to the issue public as epics are public to group members. - -When the quick action is executed: - -- An epic is created in the same group as the project of the issue. -- Subscribers of the issue are notified that the epic was created. - -The following issue metadata will be copied to the epic: - -- Title, description, activity/comment thread. -- Upvotes/downvotes. -- Participants. -- Group labels that the issue already has. - -## Searching for an epic from epics list page - -> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.5. -> - In [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/issues/37081), it was moved to the Premium tier. - -You can search for an epic from the list of epics using filtered search bar (similar to -that of Issues and Merge Requests) based on following parameters: - -- Title or description -- Author name / username -- Labels - - - -To search, go to the list of epics and click on the field **Search or filter results**. -It will display a dropdown menu, from which you can add an author. You can also enter plain -text to search by epic title or description. When done, press <kbd>Enter</kbd> on your -keyboard to filter the list. - -You can also sort epics list by: - -- Created date -- Last updated -- Start date -- Due date - -Each option contains a button that can toggle the order between **Ascending** and **Descending**. -The sort option and order is saved and used wherever you browse epics, including the -[Roadmap](../roadmap/index.md). - - - ---- - ## Permissions If you have access to view an epic and have access to view an issue already diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md new file mode 100644 index 00000000000..50eb0c64f4b --- /dev/null +++ b/doc/user/group/epics/manage_epics.md @@ -0,0 +1,302 @@ +--- +type: howto +--- + +<!-- When adding a new section here, remember to mention it in index.md#manage-epics --> + +# Manage epics **(PREMIUM)** + +This page collects instructions for all the things you can do with [epics](index.md) or in relation +to them. + +## Create an epic + +A paginated list of epics is available in each group from where you can create +a new epic. The list of epics includes also epics from all subgroups of the +selected group. From your group page: + +1. Go to **Epics**. +1. Click **New epic**. +1. Enter a descriptive title. +1. Click **Create epic**. + +You will be taken to the new epic where can edit the following details: + +- Title +- Description +- Start date +- Due date +- Labels + +An epic's page contains the following tabs: + +- **Epics and Issues**: epics and issues added to this epic. Child epics, and their issues, are shown in a tree view. + - Click the <kbd>></kbd> beside a parent epic to reveal the child epics and issues. + - Hover over the total counts to see a breakdown of open and closed items. +- **Roadmap**: a roadmap view of child epics which have start and due dates. + + + +## Bulk-edit epics + +You can edit multiple epics at once. For example, to apply labels to multiple epics: + +1. Go to the Epics list. +1. Click **Edit epics**. + - Checkboxes appear next to each epic. + - A sidebar on the right-hand side appears with an editable field for labels. +1. Select the checkbox next to each epic to be edited. +1. Select the labels you want. +1. Click **Update all**. + + + +## Delete an epic + +NOTE: **Note:** +To delete an epic, you need to be an [Owner](../../permissions.md#group-members-permissions) of a group/subgroup. + +When editing the description of an epic, click the **Delete** button to delete the epic. +A modal appears to confirm your action. + +Deleting an epic releases all existing issues from their associated epic in the system. + +## Close an epic + +Whenever you decide that there is no longer need for that epic, +close the epic by: + +- Clicking the **Close epic** button. + +  + +- Using a [quick action](../../project/quick_actions.md). + +## Reopen a closed epic + +You can reopen an epic that was closed by: + +- Clicking the **Reopen epic** button. + +  + +- Using a [quick action](../../project/quick_actions.md). + +## Go to an epic from an issue + +If an issue belongs to an epic, you can navigate to the containing epic with the +link in the issue sidebar. + + + +## Search for an epic from epics list page + +> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.5. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/37081) to the [Premium](https://about.gitlab.com/pricing/) tier in GitLab 12.8. + +You can search for an epic from the list of epics using filtered search bar (similar to +that of Issues and Merge Requests) based on following parameters: + +- Title or description +- Author name / username +- Labels + + + +To search, go to the list of epics and click the field **Search or filter results**. +It will display a dropdown menu, from which you can add an author. You can also enter plain +text to search by epic title or description. When done, press <kbd>Enter</kbd> on your +keyboard to filter the list. + +You can also sort epics list by: + +- Created date +- Last updated +- Start date +- Due date + +Each option contains a button that can toggle the order between **Ascending** and **Descending**. +The sort option and order is saved and used wherever you browse epics, including the +[Roadmap](../roadmap/index.md). + + + +## Make an epic confidential + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213068) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. +> - It's deployed behind a feature flag, disabled by default. +> - It's disabled on GitLab.com. +> - It's not recommended for production use. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-confidential-epics-premium-only). **(PREMIUM ONLY)** + +When you're creating an epic, you can make it confidential by selecting the **Make this epic +confidential** checkbox. + +### Enable Confidential Epics **(PREMIUM ONLY)** + +The Confidential Epics feature is under development and not ready for production use. It's deployed behind a +feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. + +To enable it: + +```ruby +Feature.enable(:confidential_epics) +``` + +To disable it: + +```ruby +Feature.disable(:confidential_epics) +``` + +## Manage issues assigned to an epic + +### Add an issue to an epic + +You can add an existing issue to an epic, or, create a new issue that's +automatically added to the epic. + +#### Add an existing issue to an epic + +Existing issues that belong to a project in an epic's group, or any of the epic's +subgroups, are eligible to be added to the epic. Newly added issues appear at the top of the list of +issues in the **Epics and Issues** tab. + +An epic contains a list of issues and an issue can be associated with at most one epic. +When you add an issue that's already linked to an epic, the issue is automatically unlinked from its +current parent. + +To add an issue to an epic: + +1. Click the **Add** dropdown button. +1. Click **Add an issue**. +1. Identify the issue to be added, using either of the following methods: + - Paste the link of the issue. + - Search for the desired issue by entering part of the issue's title, then selecting the desired + match (introduced in [GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/issues/9126)). + + If there are multiple issues to be added, press <kbd>Spacebar</kbd> and repeat this step. +1. Click **Add**. + +#### Create an issue from an epic + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5419) in GitLab 12.7. + +Creating an issue from an epic enables you to maintain focus on the broader context of the epic +while dividing work into smaller parts. + +To create an issue from an epic: + +1. On the epic's page, under **Epics and Issues**, click the **Add** dropdown button and select + **Create new issue**. +1. Under **Title**, enter the title for the new issue. +1. From the **Project** dropdown, select the project in which the issue should be created. +1. Click **Create issue**. + +To remove an issue from an epic: + +1. Click the <kbd>x</kbd> button in the epic's issue list. +1. Click **Remove** in the **Remove issue** warning message. + +### Reorder issues assigned to an epic + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9367) in GitLab 12.5. + +New issues are added to the top of their list in the **Epics and Issues** tab. +You can reorder the list of issues. Issues and child epics cannot be intermingled. + +To reorder issues assigned to an epic: + +1. Go to the **Epics and Issues** tab. +1. Drag and drop issues into the desired order. + +To reorder child epics assigned to an epic: + +1. Go to the **Epics and Issues** tab. +1. Drag and drop epics into the desired order. + +### Move issues between epics **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33039) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. + +New issues are added to the top of their list in the **Epics and Issues** +tab. You can move issues from one epic to another. Issues and child epics cannot be intermingled. + +To move an issue to another epic: + +1. Go to the **Epics and Issues** tab. +1. Drag and drop issues into the desired parent epic. + +### Promote an issue to an epic + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/3777) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.6. +> - In [GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/issues/37081), it was moved to the Premium tier. + +If you have [permissions](../../permissions.md) to close an issue and create an +epic in the parent group, you can promote an issue to an epic with the `/promote` +[quick action](../../project/quick_actions.md#quick-actions-for-issues-merge-requests-and-epics). +Only issues from projects that are in groups can be promoted. When attempting to promote a confidential +issue, a warning will display. Promoting a confidential issue to an epic will make all information +related to the issue public as epics are public to group members. + +When the quick action is executed: + +- An epic is created in the same group as the project of the issue. +- Subscribers of the issue are notified that the epic was created. + +The following issue metadata will be copied to the epic: + +- Title, description, activity/comment thread. +- Upvotes/downvotes. +- Participants. +- Group labels that the issue already has. + +## Manage multi-level child epics **(ULTIMATE)** + +### Add a child epic to an epic + +To add a child epic to an epic: + +1. Click the **Add** dropdown button. +1. Click **Add an epic**. +1. Identify the epic to be added, using either of the following methods: + - Paste the link of the epic. + - Search for the desired issue by entering part of the epic's title, then selecting the desired match (introduced in [GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/issues/9126)). + + If there are multiple epics to be added, press <kbd>Spacebar</kbd> and repeat this step. +1. Click **Add**. + +### Move child epics between epics + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33039) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. + +New child epics are added to the top of their list in the **Epics and Issues** tab. +You can move child epics from one epic to another. +When you add an epic that's already linked to a parent epic, the link to its current parent is removed. +Issues and child epics cannot be intermingled. + +To move child epics to another epic: + +1. Go to the **Epics and Issues** tab. +1. Drag and drop epics into the desired parent epic. + +### Reorder child epics assigned to an epic + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9367) in GitLab 12.5. + +New child epics are added to the top of their list in the **Epics and Issues** tab. +You can reorder the list of child epics. Issues and child epics cannot be intermingled. + +To reorder child epics assigned to an epic: + +1. Go to the **Epics and Issues** tab. +1. Drag and drop epics into the desired order. + +### Remove a child epic from a parent epic + +To remove a child epic from a parent epic: + +1. Click on the <kbd>x</kbd> button in the parent epic's list of epics. +1. Click **Remove** in the **Remove epic** warning message. diff --git a/doc/user/group/index.md b/doc/user/group/index.md index b819e3e971e..f36f3b3fd4f 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -196,6 +196,9 @@ To change this setting for a specific group: To change this setting globally, see [Default branch protection](../admin_area/settings/visibility_and_access_controls.md#default-branch-protection). +NOTE: **Note:** +In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../admin_area/settings/visibility_and_access_controls.md#disable-group-owners-from-updating-default-branch-protection-premium-only). + ## Add projects to a group There are two different ways to add a new project to a group: @@ -211,8 +214,8 @@ There are two different ways to add a new project to a group: ### Default project-creation level -> - [Introduced][ee-2534] in [GitLab Premium][ee] 10.5. -> - Brought to [GitLab Starter][ee] in 10.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/2534) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. +> - Brought to [GitLab Starter](https://about.gitlab.com/pricing/) in 10.7. > - [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/25975) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.10. By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group. @@ -308,7 +311,7 @@ See [the GitLab Enterprise Edition documentation](../../integration/ldap.md) for ## Epics **(ULTIMATE)** -> Introduced in [GitLab Ultimate][ee] 10.2. +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.2. Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and @@ -593,6 +596,29 @@ For performance reasons, we may delay the update up to 1 hour and 30 minutes. If your namespace shows `N/A` as the total storage usage, you can trigger a recalculation by pushing a commit to any project in that namespace. +#### Group push rules **(STARTER)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/34370) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. + +Group push rules allow group maintainers to set +[push rules](../../push_rules/push_rules.md) for newly created projects within the specific group. + +To configure push rules for a group, navigate to **{push-rules}** on the group's +sidebar. + +When set, new subgroups have push rules set for them based on either: + +- The closest parent group with push rules defined. +- Push rules set at the instance level, if no parent groups have push rules defined. + +##### Enabling the feature + +This feature comes with the `:group_push_rules` feature flag disabled by default. It can be enabled for specific group using feature flag [API endpoint](../../api/features.md#set-or-create-a-feature) or by GitLab administrator with Rails console access by running: + +```ruby +Feature.enable(:group_push_rules) +``` + ### Maximum artifacts size **(CORE ONLY)** For information about setting a maximum artifact size for a group, see @@ -623,6 +649,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[ee]: https://about.gitlab.com/pricing/ -[ee-2534]: https://gitlab.com/gitlab-org/gitlab/issues/2534 diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index edbb85962ed..cffbc013e66 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -1,5 +1,9 @@ --- type: reference, howto +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Insights **(ULTIMATE)** diff --git a/doc/user/group/issues_analytics/index.md b/doc/user/group/issues_analytics/index.md index 4477b9bb1e6..df96f2626e1 100644 --- a/doc/user/group/issues_analytics/index.md +++ b/doc/user/group/issues_analytics/index.md @@ -1,5 +1,9 @@ --- type: reference +stage: Manage +group: Analytics +To determine the technical writer assigned to the Stage/Group associated with this page, see: + https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Issues Analytics **(PREMIUM)** diff --git a/doc/user/group/roadmap/img/roadmap_view_v12_10.png b/doc/user/group/roadmap/img/roadmap_view_v12_10.png Binary files differdeleted file mode 100644 index 69579fd1c1e..00000000000 --- a/doc/user/group/roadmap/img/roadmap_view_v12_10.png +++ /dev/null diff --git a/doc/user/group/roadmap/img/roadmap_view_v13_0.png b/doc/user/group/roadmap/img/roadmap_view_v13_0.png Binary files differnew file mode 100644 index 00000000000..a5b76b84418 --- /dev/null +++ b/doc/user/group/roadmap/img/roadmap_view_v13_0.png diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index 9f068adcd47..6bee552d433 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -7,7 +7,8 @@ type: reference > - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.5. > - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/issues/198062), Roadmaps were moved to the Premium tier. > - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/issues/5164) and later, the epic bars show epics' title, progress, and completed weight percentage. -> - In [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/6802), and later, milestones appear in Roadmaps. +> - Milestones appear in Roadmaps in [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/issues/6802), and later. +> - Feature flag removed in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29641). Epics and milestones within a group containing **Start date** and/or **Due date** can be visualized in a form of a timeline (that is, a Gantt chart). The Roadmap page @@ -23,7 +24,7 @@ You can click the chevron **{chevron-down}** next to the epic title to expand an On top of the milestone bars, you can see their title. When you hover a milestone bar or title, a popover appears with its title, start date and due date. - + A dropdown menu allows you to show only open or closed epics. By default, all epics are shown. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index f49dd225146..a3d9a14df10 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -59,7 +59,7 @@ We recommend setting the NameID format to `Persistent` unless using a field (suc With this option enabled, users must use your group's GitLab single sign on URL to be added to the group or be added via SCIM. Users cannot be added manually, and may only access project/group resources via the UI by signing in through the SSO URL. -However, users will not be prompted to log via SSO on each visit. GitLab will check whether a user has authenticated through the SSO link, and will only prompt the user to login via SSO if it has been longer than 7 days. +However, users will not be prompted to log via SSO on each visit. GitLab will check whether a user has authenticated through the SSO link, and will only prompt the user to login via SSO if the session has expired. We intend to add a similar SSO requirement for [Git and API activity](https://gitlab.com/gitlab-org/gitlab/issues/9152) in the future. @@ -189,6 +189,9 @@ Once you've set up your identity provider to work with GitLab, you'll need to co  +NOTE: **Note:** +Please note that the certificate [fingerprint algorithm](#additional-setup-options) must be in SHA1. When configuring the identity provider, use a secure [signature algorithm](#additional-setup-options). + ## User access and management Once Group SSO is configured and enabled, users can access the GitLab.com group through the identity provider's dashboard. If [SCIM](scim_setup.md) is configured, please see the [user access and linking setup section on the SCIM page](scim_setup.md#user-access-and-linking-setup). @@ -254,6 +257,9 @@ Set other user attributes and claims according to the [assertions table](#assert ### Okta setup notes +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). + | GitLab Setting | Okta Field | |--------------|----------------| | Identifier | Audience URI | @@ -297,7 +303,7 @@ GitLab [isn't limited to the SAML providers listed above](#my-identity-provider- | SAML Request Binding | HTTP Redirect | GitLab (the service provider) redirects users to your Identity Provider with a base64 encoded `SAMLRequest` HTTP parameter. | | SAML Response Binding | HTTP POST | Your Identity Provider responds to users with an HTTP form including the `SAMLResponse`, which a user's browser submits back to GitLab. | | Sign SAML Response | Yes | We require this to prevent tampering. | -| X509 Certificate in response | Yes | This is used to sign the response and checked against the provided fingerprint. | +| X.509 Certificate in response | Yes | This is used to sign the response and checked against the provided fingerprint. | | Fingerprint Algorithm | SHA-1 | We need a SHA-1 hash of the certificate used to sign the SAML Response. | | Signature Algorithm | SHA-1/SHA-256/SHA-384/SHA-512 | Also known as the Digest Method, this can be specified in the SAML response. It determines how a response is signed. | | Encrypt SAML Assertion | No | TLS is used between your Identity Provider, the user's browser, and GitLab. | diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index e333fd19c1b..f66c8a788b6 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -12,25 +12,28 @@ that group is synchronized between GitLab and the identity provider. GitLab's [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). +## Features + Currently, the following actions are available: -- CREATE -- UPDATE -- DELETE (deprovisioning) +- Create users +- Update users (Azure only) +- Deactivate users The following identity providers are supported: - Azure +- Okta ## Requirements -- [Group SSO](index.md) must be configured. +- [Group Single Sign-On](index.md) must be configured. ## GitLab configuration -Once [Single sign-on](index.md) has been configured, we can: +Once [Group Single Sign-On](index.md) has been configured, we can: -1. Navigate to the group and click **Settings > SAML SSO**. +1. Navigate to the group and click **Administration > SAML SSO**. 1. Click on the **Generate a SCIM token** button. 1. Save the token and URL so they can be used in the next step. @@ -38,9 +41,12 @@ Once [Single sign-on](index.md) has been configured, we can: ## Identity Provider configuration -### Azure +- [Azure](#azure-configuration-steps) +- [Okta](#okta-configuration-steps) + +### Azure configuration steps -The SAML application that was created during [Single sign-on](index.md) setup now needs to be set up for SCIM. +The SAML application that was created during [Single sign-on](index.md) setup for [Azure](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/configure-single-sign-on-non-gallery-applications) now needs to be set up for SCIM. 1. Check the configuration for your GitLab SAML app and ensure that **Name identifier value** (NameID) points to `user.objectid` or another unique identifier. This will match the `extern_uid` used on GitLab. @@ -109,6 +115,43 @@ bottom of the **Provisioning** screen, together with a link to the audit logs. CAUTION: **Warning:** Once synchronized, changing the field mapped to `id` and `externalId` will likely cause provisioning errors, duplicate users, and prevent existing users from accessing the GitLab group. +### Okta configuration steps + +The SAML application that was created during [Single sign-on](index.md#okta-setup-notes) setup for [Okta](https://developer.okta.com/docs/guides/saml-application-setup/overview/) now needs to be set up for SCIM. +Before proceeding, be sure to complete the [GitLab configuration](#gitlab-configuration) process. + +1. Sign in to Okta. +1. If you see an **Admin** button in the top right, click the button. This will + ensure you are in the Admin area. + + TIP: **Tip:** If you're using the Developer Console, click **Developer Console** in the top + bar and select **Classic UI**. Otherwise, you may not see the buttons described + in the following steps: + +1. In the **Application** tab, click **Add Application**. +1. Search for **GitLab**, find and click on the 'GitLab' application. +1. On the GitLab application overview page, click **Add**. +1. Under **Application Visibility** select both check boxes. Currently the GitLab application does not support SAML authentication so the icon should not be shown to users. +1. Click **Done** to finish adding the application. +1. In the **Provisioning** tab, click **Configure API integration**. +1. Select **Enable API integration**. + - For **Base URL** enter the URL obtained from the GitLab SCIM configuration page + - For **API Token** enter the SCIM token obtained from the GitLab SCIM configuration page +1. Click 'Test API Credentials' to verify configuration. +1. Click **Save** to apply the settings. +1. After saving the API integration details, new settings tabs will appear on the left. Choose **To App**. +1. Click **Edit**. +1. Check the box to **Enable** for both **Create Users** and **Deactivate Users**. +1. Click **Save**. +1. Assign users in the **Assignments** tab. Assigned users will be created and + managed in your GitLab group. + +#### Okta Known Issues + +The Okta GitLab application currently only supports SCIM. Continue +using the separate Okta [SAML SSO](index.md) configuration along with the new SCIM +application described above. + ## User access and linking setup As long as [Group SAML](index.md) has been configured, prior to turning on sync, existing GitLab.com users can link to their accounts in one of the following ways, before synchronization is active: @@ -135,7 +178,9 @@ Upon the next sync, the user will be deprovisioned, which means that the user wi This section contains possible solutions for problems you might encounter. -### How do I verify my SCIM configuration is correct? +### Azure + +#### How do I verify my SCIM configuration is correct? Review the following: @@ -148,11 +193,11 @@ Review the following SCIM parameters for sensible values: - `displayName` - `emails[type eq "work"].value` -### Testing Azure connection: invalid credentials +#### Testing Azure connection: invalid credentials When testing the connection, you may encounter an error: **You appear to have entered invalid credentials. Please confirm you are using the correct information for an administrative account**. If `Tenant URL` and `secret token` are correct, check whether your group path contains characters that may be considered invalid JSON primitives (such as `.`). Removing such characters from the group path typically resolves the error. -### Azure: (Field) can't be blank sync error +#### Azure: (Field) can't be blank sync error When checking the Audit Logs for the Provisioning, you can sometimes see the error `Namespace can't be blank, Name can't be blank, and User can't be blank.` @@ -165,14 +210,7 @@ As a workaround, try an alternate mapping: 1. Delete the `name.formatted` target attribute entry. 1. Change the `displayName` source attribute to have `name.formatted` target attribute. -### Message: "SAML authentication failed: Email has already been taken" - -This message may be caused by the following: - -- Existing users have not yet signed into the new app. -- The identity provider attempts to create a new user account in GitLab with an email address that already exists in GitLab.com. - -### How do I diagnose why a user is unable to sign in +#### How do I diagnose why a user is unable to sign in The **Identity** (`extern_uid`) value stored by GitLab is updated by SCIM whenever `id` or `externalId` changes. Users won't be able to sign in unless the GitLab Identity (`extern_uid`) value matches the `NameId` sent by SAML. @@ -180,7 +218,7 @@ This value is also used by SCIM to match users on the `id`, and is updated by SC It is important that this SCIM `id` and SCIM `externalId` are configured to the same value as the SAML `NameId`. SAML responses can be traced using [debugging tools](./index.md#saml-debugging-tools), and any errors can be checked against our [SAML troubleshooting docs](./index.md#troubleshooting). -### How do I verify user's SAML NameId matches the SCIM externalId +#### How do I verify user's SAML NameId matches the SCIM externalId Group owners can see the list of users and the `externalId` stored for each user in the group SAML SSO Settings page. @@ -194,7 +232,7 @@ curl 'https://example.gitlab.com/api/scim/v2/groups/GROUP_NAME/Users?startIndex= To see how this compares to the value returned as the SAML NameId, you can have the user use a [SAML Tracer](index.md#saml-debugging-tools). -### Update or fix mismatched SCIM externalId and SAML NameId +#### Update or fix mismatched SCIM externalId and SAML NameId Whether the value was changed or you need to map to a different field, ensure `id`, `externalId`, and `NameId` all map to the same field. @@ -220,7 +258,7 @@ curl --verbose --request PATCH 'https://gitlab.com/api/scim/v2/groups/YOUR_GROUP It is important not to update these to incorrect values, since this will cause users to be unable to sign in. It is also important not to assign a value to the wrong user, as this would cause users to get signed into the wrong account. -### I need to change my SCIM app +#### I need to change my SCIM app Individual users can follow the instructions in the ["SAML authentication failed: User has already been taken"](./index.md#i-need-to-change-my-saml-app) section. diff --git a/doc/user/group/settings/img/export_panel.png b/doc/user/group/settings/img/export_panel.png Binary files differnew file mode 100644 index 00000000000..2a987f04e35 --- /dev/null +++ b/doc/user/group/settings/img/export_panel.png diff --git a/doc/user/group/settings/import_export.md b/doc/user/group/settings/import_export.md new file mode 100644 index 00000000000..3f14fea673b --- /dev/null +++ b/doc/user/group/settings/import_export.md @@ -0,0 +1,98 @@ +# Group Import/Export + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2888) in GitLab 13.0 as an experimental feature. May change in future releases. + +Existing groups running on any GitLab instance or GitLab.com can be exported with all their related data and moved to a +new GitLab instance. + +The **GitLab import/export** button is displayed if the group import option in enabled. + +See also: + +- [Group Import/Export API](../../../api/group_import_export.md) +- [Project Import/Export](../../project/settings/import_export.md) +- [Project Import/Export API](../../../api/project_import_export.md) + +To enable GitLab import/export: + +1. Navigate to **{admin}** **Admin Area >** **{settings}** **Settings > Visibility and access controls**. +1. Scroll to **Import sources** +1. Enable desired **Import sources** + +## Important Notes + +Note the following: + +- Exports are stored in a temporary [shared directory](../../../development/shared_files.md) and are deleted every 24 hours by a specific worker. +- To preserve group-level relationships from imported projects, run the Group Import/Export first, to allow projects to +be imported into the desired group structure. +- Imported groups are given a `private` visibility level, unless imported into a parent group. +- If imported into a parent group, subgroups will inherit the same level of visibility unless otherwise restricted. +- To preserve the member list and their respective permissions on imported groups, review the users in these groups. Make +sure these users exist before importing the desired groups. + +### Exported Contents + +The following items will be exported: + +- Milestones +- Labels +- Boards and Board Lists +- Badges +- Subgroups (including all the aforementioned data) +- Epics +- Events + +The following items will NOT be exported: + +- Projects +- Runners token +- SAML discovery tokens + +NOTE: **Note:** +For more details on the specific data persisted in a group export, see the +[`import_export.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/import_export/group/import_export.yml) file. + +## Exporting a Group + +1. Navigate to your group's homepage. + +1. Click **{settings}** **Settings** in the sidebar. + +1. In the **Advanced** section, click the **Export Group** button. + +  + +1. Once the export is generated, you should receive an e-mail with a link to the [exported contents](#exported-contents) + in a compressed tar archive, with contents in JSON format. + +1. Alternatively, you can come back to the project settings and download the + file from there by clicking **Download export**, or generate a new file by clicking **Regenerate export**. + +### Between CE and EE + +You can export groups from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. + +If you're exporting a group from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../README.md). + +## Version history + +GitLab can import bundles that were exported from a different GitLab deployment. +This ability is limited to two previous GitLab [minor](../../../policy/maintenance.md#versioning) +releases, which is similar to our process for [Security Releases](../../../policy/maintenance.md#security-releases). + +For example: + +| Current version | Can import bundles exported from | +|-----------------|----------------------------------| +| 13.0 | 13.0, 12.10, 12.9 | +| 13.1 | 13.1, 13.0, 12.10 | + +## Rate Limits + +To help avoid abuse, users are rate limited to: + +| Request Type | Limit | +| ---------------- | ------------------------------ | +| Export | 1 group every 5 minutes | +| Download export | 10 downloads every 10 minutes | diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index e73623a2f0e..49f6ddc3986 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -191,11 +191,6 @@ Here's a list of what you can't do with subgroups: with `group`, but can be shared with `group/subgroup02` or `group/subgroup01/subgroup03`. -[ce-2772]: https://gitlab.com/gitlab-org/gitlab-foss/issues/2772 -[permissions]: ../../permissions.md#group-members-permissions -[reserved]: ../../reserved_names.md -[issue]: https://gitlab.com/gitlab-org/gitlab-foss/issues/30472#note_27747600 - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/img/gitlab_snippet.png b/doc/user/img/gitlab_snippet.png Binary files differdeleted file mode 100644 index 718347fc2d4..00000000000 --- a/doc/user/img/gitlab_snippet.png +++ /dev/null diff --git a/doc/user/img/gitlab_snippet_v13_0.png b/doc/user/img/gitlab_snippet_v13_0.png Binary files differnew file mode 100644 index 00000000000..33194c512df --- /dev/null +++ b/doc/user/img/gitlab_snippet_v13_0.png diff --git a/doc/user/img/snippet_clone_button_v13_0.png b/doc/user/img/snippet_clone_button_v13_0.png Binary files differnew file mode 100644 index 00000000000..bf681e7349b --- /dev/null +++ b/doc/user/img/snippet_clone_button_v13_0.png diff --git a/doc/user/incident_management/index.md b/doc/user/incident_management/index.md index 8505afb375e..73d6e0e055d 100644 --- a/doc/user/incident_management/index.md +++ b/doc/user/incident_management/index.md @@ -1,141 +1,112 @@ --- -description: "GitLab - Incident Management. GitLab offers solutions for handling incidents in your applications and services" +stage: Monitor +group: Health +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Incident Management GitLab offers solutions for handling incidents in your applications and services, -from setting up an alert with Prometheus, to receiving a notification via a -monitoring tool like Slack, and automatically setting up Zoom calls with your -support team. +from setting up an alert with Prometheus, to receiving a notification through a +monitoring tool like Slack, and [setting up Zoom calls](#zoom-integration-in-issues) with your +support team. Incidents can display [metrics](#embed-metrics-in-incidents-and-issues) +and [logs](#view-logs-from-metrics-panel). -## Configuring incidents **(ULTIMATE)** +## Configure incidents **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4925) in GitLab Ultimate 11.11. -The Incident Management features can be enabled and disabled via your project's -**Settings > Operations > Incidents**. +You can enable or disable Incident Management features in your project's +**{settings}** **Settings > Operations > Incidents**. Issues can be created for +each alert triggered, and separate email notifications can be sent to users with +[Developer permissions](../permissions.md). Appropriately configured alerts include an +[embedded chart](../project/integrations/prometheus.md#embedding-metrics-based-on-alerts-in-incident-issues) +for the query corresponding to the alert. You can also configure GitLab to +[close issues](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) +when you receive notification that the alert is resolved.  -### Automatically create issues from alerts +### Create issues from alerts -GitLab issues can automatically be created as a result of an alert notification. -An issue created this way will contain the error information to help you further -debug it. +You can create GitLab issues from an alert notification. These issues contain +information about the alerts to help you diagnose the source of the alerts. -### Issue templates - -You can create your own [issue templates](../project/description_templates.md#creating-issue-templates) -that can be [used within Incident Management](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate). - -To select your issue template for use within Incident Management: - -1. Visit your project's **Settings > Operations > Incidents**. +1. Visit your project's **{settings}** **Settings > Operations > Incidents**. +1. Select the **Create an issue** checkbox for GitLab to create an issue from + the incident. 1. Select the template from the **Issue Template** dropdown. + You can create your own [issue templates](../project/description_templates.md#creating-issue-templates) + to [use within Incident Management](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate). +1. Click **Save changes**. -## Alerting +## Notify developers of alerts -GitLab can react to the alerts that your applications and services may be -triggering by automatically creating issues, and alerting developers via email. +GitLab can react to the alerts triggered from your applications and services +by creating issues and alerting developers through email. GitLab sends these emails +to [owners and maintainers](../permissions.md) of the project. They contain details +of the alert, and a link for more information. -The emails will be sent to [owners and maintainers](../permissions.md) of the project and will contain details on the alert as well as a link to see more information. +### Configure Prometheus alerts -### Prometheus alerts +You can set up Prometheus alerts in: -Prometheus alerts can be set up in both: - -- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics) and +- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics) installations. - [Self-managed Prometheus](../project/integrations/prometheus.md#external-prometheus-instances) installations. -### Alert endpoint - -GitLab can accept alerts from any source via a generic webhook receiver. When -you set up the generic alerts integration, a unique endpoint will -be created which can receive a payload in JSON format. - -[Read more on setting this up, including how to customize the payload](../project/integrations/generic_alerts.md). +Prometheus alerts are created by the special Alert Bot user. You can't remove this +user, but it does not count toward your license limit. -### Recovery alerts +### Configure external generic alerts -GitLab can [automatically close issues](../project/integrations/prometheus.md#taking-action-on-incidents-ultimate) -that have been automatically created when you receive notification that the -alert is resolved. +GitLab can accept alerts from any source through a generic webhook receiver. When +[configuring the generic alerts integration](../project/integrations/generic_alerts.md), +GitLab creates a unique endpoint which receives a JSON-formatted, customizable payload. -## Embedded metrics +## Embed metrics in incidents and issues -Metrics can be embedded anywhere where GitLab Markdown is used, for example, -descriptions and comments on issues and merge requests. - -This can be useful for when you're sharing metrics, such as for discussing -an incident or performance issues, so you can output the dashboard directly +You can embed metrics anywhere GitLab Markdown is used, such as descriptions, +comments on issues, and merge requests. Embedding metrics helps you share them +when discussing incidents or performance issues. You can output the dashboard directly into any issue, merge request, epic, or any other Markdown text field in GitLab -by simply [copying and pasting the link to the metrics dashboard](../project/integrations/prometheus.md#embedding-gitlab-managed-kubernetes-metrics). - -TIP: **Tip:** -Both GitLab-hosted and Grafana metrics can also be -[embedded in issue templates](../project/integrations/prometheus.md#embedding-metrics-in-issue-templates). - -### GitLab-hosted metrics +by [copying and pasting the link to the metrics dashboard](../project/integrations/prometheus.md#embedding-gitlab-managed-kubernetes-metrics). -Learn how to embed [GitLab hosted metric charts](../project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown). +You can embed both +[GitLab-hosted metrics](../project/integrations/prometheus.md#embedding-metric-charts-within-gitlab-flavored-markdown) and +[Grafana metrics](../project/integrations/prometheus.md#embedding-grafana-charts) +in incidents and issue templates. -#### Context menu +### Context menu From each of the embedded metrics panels, you can access more details -about the data you are viewing from a context menu. - -You can access the context menu by clicking the **{ellipsis_v}** **More actions** -dropdown box above the upper right corner of the panel: - -The options are: +about the data you're viewing from a context menu. You can access the context menu +by clicking the **{ellipsis_v}** **More actions** dropdown box above the +upper right corner of the panel. The options are: -- [View logs](#view-logs) -- [Download CSV](#download-csv) +- [View logs](#view-logs-from-metrics-panel). +- **Download CSV** - Data from embedded charts can be + [downloaded as CSV](../project/integrations/prometheus.md#downloading-data-as-csv). -##### View logs +#### View logs from metrics panel > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/201846) in GitLab Ultimate 12.8. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) to [GitLab Core](https://about.gitlab.com/pricing/) 12.9. -This can be useful if you are triaging an application incident and need to -[explore logs](../project/integrations/prometheus.md#view-logs-ultimate) -from across your application. It also helps you to understand -what is affecting your application's performance and quickly resolve any problems. - -##### Download CSV - -Data from embedded charts can be [downloaded as CSV](../project/integrations/prometheus.md#downloading-data-as-csv). - -### Grafana metrics - -Learn how to embed [Grafana hosted metric charts](../project/integrations/prometheus.md#embedding-grafana-charts). +Viewing logs from a metrics panel can be useful if you're triaging an application +incident and need to [explore logs](../project/integrations/prometheus.md#view-logs-ultimate) +from across your application. These logs help you understand what is affecting +your application's performance and resolve any problems. ## Slack integration -Slack slash commands allow you to control GitLab and view content right inside -Slack, without having to leave it. +Slack slash commands allow you to control GitLab and view GitLab content without leaving Slack. Learn how to [set up Slack slash commands](../project/integrations/slack_slash_commands.md) -and how to [use them](../../integration/slash_commands.md). - -### Slash commands - -Please refer to a list of [available slash commands](../../integration/slash_commands.md) and associated descriptions. - -## Zoom in issues - -In order to communicate synchronously for incidents management, GitLab allows you to -associate a Zoom meeting with an issue. Once you start a Zoom call for a fire-fight, -you need a way to associate the conference call with an issue, so that your team -members can join swiftly without requesting a link. - -Read more how to [add or remove a zoom meeting](../project/issues/associate_zoom_meeting.md). - -### Configuring Incidents - -Incident Management features can be easily enabled & disabled via the Project settings page. Head to Project -> Settings -> Operations -> Incidents. +and how to [use the available slash commands](../../integration/slash_commands.md). -#### Auto-creation +## Zoom integration in issues -You can automatically create GitLab issues from an Alert notification. Issues created this way contain error information to help you debug the error. Appropriately configured alerts include an [embedded chart](../project/integrations/prometheus.md#embedding-metrics-based-on-alerts-in-incident-issues) for the query corresponding to the alert. +GitLab enables you to [associate a Zoom meeting with an issue](../project/issues/associate_zoom_meeting.md) +for synchronous communication during incident management. After starting a Zoom +call for an incident, you can associate the conference call with an issue, so your +team members can join without requesting a link. diff --git a/doc/user/infrastructure/img/terraform_plan_log_v13_0.png b/doc/user/infrastructure/img/terraform_plan_log_v13_0.png Binary files differnew file mode 100644 index 00000000000..c3c6f6b2f8b --- /dev/null +++ b/doc/user/infrastructure/img/terraform_plan_log_v13_0.png diff --git a/doc/user/infrastructure/img/terraform_plan_widget_v13_0.png b/doc/user/infrastructure/img/terraform_plan_widget_v13_0.png Binary files differnew file mode 100644 index 00000000000..62bf4b279b2 --- /dev/null +++ b/doc/user/infrastructure/img/terraform_plan_widget_v13_0.png diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index a50cdf1cf0e..65237bf24e0 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -1,6 +1,343 @@ -# Infrastructure as Code +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- -GitLab can be used to manage infrastructure as code. The following are some examples: +# Infrastructure as code with Terraform and GitLab -- [A generic tutorial for Terraform with GitLab](https://medium.com/@timhberry/terraform-pipelines-in-gitlab-415b9d842596). -- [Terraform at GitLab](https://about.gitlab.com/blog/2019/11/12/gitops-part-2/). +## GitLab managed Terraform State + +[Terraform remote backends](https://www.terraform.io/docs/backends/index.html) +enable you to store the state file in a remote, shared store. GitLab uses the +[Terraform HTTP backend](https://www.terraform.io/docs/backends/types/http.html) +to securely store the state files in local storage (the default) or +[the remote store of your choice](../../administration/terraform_state.md). + +The GitLab managed Terraform state backend can store your Terraform state easily and +securely, and spares you from setting up additional remote resources like +Amazon S3 or Google Cloud Storage. Its features include: + +- Supporting encryption of the state file both in transit and at rest. +- Locking and unlocking state. +- Remote Terraform plan and apply execution. + +To get started, there are two different options when using GitLab managed Terraform State. + +- Use a local machine +- Use GitLab CI + +## Get Started using local development + +If you are planning to only run `terraform plan` and `terraform apply` commands from your local machine, this is a simple way to get started. + +First, create your project on your GitLab instance. + +Next, define the Terraform backend in your Terraform project to be: + +```hcl +terraform { + backend "http" { + } +} +``` + +Finally, you need to run `terraform init` on your local machine and pass in the following options. The below example is using GitLab.com: + +```bash +terraform init \ + -backend-config="address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>" \ + -backend-config="lock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ + -backend-config="unlock_address=https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-PROJECT-NAME>/lock" \ + -backend-config="username=<YOUR-USERNAME>" \ + -backend-config="password=<YOUR-ACCESS-TOKEN>" \ + -backend-config="lock_method=POST" \ + -backend-config="unlock_method=DELETE" \ + -backend-config="retry_wait_min=5" +``` + +This will initialize your Terraform state and store that state within your GitLab project. + +NOTE: YOUR-PROJECT-ID and YOUR-PROJECT-NAME can be accessed from the project main page. + +## Get Started using a GitLab CI + +Another route is to leverage GitLab CI to run your `terraform plan` and `terraform apply` commands. + +### Configure the CI variables + +To use the Terraform backend, [first create a Personal Access Token](../profile/personal_access_tokens.md) with the `api` scope. Keep in mind that the Terraform backend is restricted to tokens with [Maintainer access](../permissions.md) to the repository. + +To keep the Personal Access Token secure, add it as a [CI/CD environment variable](../../ci/variables/README.md). In this example we set ours to the ENV: `GITLAB_TF_PASSWORD`. + +If you are planning to use the ENV on a branch which is not protected, make sure to set the variable protection settings correctly. + +### Configure the Terraform backend + +Next we need to define the [http backend](https://www.terraform.io/docs/backends/types/http.html). In your Terraform project add the following code block in a `.tf` file such as `backend.tf` or wherever you desire to define the remote backend: + +```hcl +terraform { + backend "http" { + } +} +``` + +### Configure the CI YAML file + +Finally, configure a `.gitlab-ci.yaml`, which lives in the root of your project repository. + +In our case we are using a pre-built image: + +```yaml +image: + name: hashicorp/terraform:light + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' +``` + +We then define some environment variables to make life easier. `GITLAB_TF_ADDRESS` is the URL of the GitLab instance where this pipeline runs, and `TF_ROOT` is the directory where the Terraform commands must be executed. + +```yaml +variables: + GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} + TF_ROOT: ${CI_PROJECT_DIR}/environments/cloudflare/production + +cache: + paths: + - .terraform +``` + +In a `before_script`, pass a `terraform init` call containing configuration parameters. +These parameters correspond to variables required by the +[http backend](https://www.terraform.io/docs/backends/types/http.html): + +```yaml +before_script: + - cd ${TF_ROOT} + - terraform --version + - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" + +stages: + - validate + - build + - test + - deploy + +validate: + stage: validate + script: + - terraform validate + +plan: + stage: build + script: + - terraform plan + - terraform show + +apply: + stage: deploy + environment: + name: production + script: + - terraform apply + dependencies: + - plan + when: manual + only: + - master +``` + +### Push to GitLab + +Pushing your project to GitLab triggers a CI job pipeline, which runs the `terraform init`, `terraform validate`, and `terraform plan` commands automatically. + +The output from the above `terraform` commands should be viewable in the job logs. + +## Example project + +See [this reference project](https://gitlab.com/nicholasklick/gitlab-terraform-aws) using GitLab and Terraform to deploy a basic AWS EC2 within a custom VPC. + +## Output Terraform Plan information into a merge request + +Using the [GitLab Terraform Report Artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform), +you can expose details from `terraform plan` runs directly into a merge request widget, +enabling you to see statistics about the resources that Terraform will create, +modify, or destroy. + +Let's explore how to configure a GitLab Terraform Report Artifact: + +1. First, for simplicity, let's define a few reusable variables to allow us to + refer to these files multiple times: + + ```yaml + variables: + PLAN: plan.tfplan + PLAN_JSON: tfplan.json + ``` + +1. Next we need to install `jq`, a [lightweight and flexible command-line JSON processor](https://stedolan.github.io/jq/). We will also create an alias for a specific `jq` command that parses out the extact information we want to extract from the `terraform plan` output: + +```yaml +before_script: + - apk --no-cache add jq + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" +``` + +1. Finally, we define a `script` that runs `terraform plan` and also a `terraform show` which pipes the output and converts the relevant bits into a store variable `PLAN_JSON`. This json is then leveraged to create a [GitLab Terraform Report Artifact](../../ci/pipelines/job_artifacts.md#artifactsreportsterraform). + +The terraform report obtains a Terraform tfplan.json file. The collected Terraform plan report will be uploaded to GitLab as an artifact and will be automatically shown in merge requests. + +```yaml +plan: + stage: build + script: + - terraform plan -out=$PLAN + - terraform show --json $PLAN | convert_report > $PLAN_JSON + artifacts: + name: plan + paths: + - $PLAN + reports: + terraform: $PLAN_JSON +``` + +A full `.gitlab-ci.yaml` file could look like this: + +```yaml +image: + name: hashicorp/terraform:light + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + +# Default output file for Terraform plan +variables: + GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} + PLAN: plan.tfplan + PLAN_JSON: tfplan.json + TF_ROOT: ${CI_PROJECT_DIR} + +cache: + paths: + - .terraform + +before_script: + - apk --no-cache add jq + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" + - cd ${TF_ROOT} + - terraform --version + - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" + +stages: + - validate + - build + - deploy + +validate: + stage: validate + script: + - terraform validate + +plan: + stage: build + script: + - terraform plan -out=$PLAN + - terraform show --json $PLAN | convert_report > $PLAN_JSON + artifacts: + name: plan + paths: + - ${TF_ROOT}/plan.tfplan + reports: + terraform: ${TF_ROOT}/tfplan.json + +# Separate apply job for manual launching Terraform as it can be destructive +# action. +apply: + stage: deploy + environment: + name: production + script: + - terraform apply -input=false $PLAN + dependencies: + - plan + when: manual + only: + - master + +``` + +1. Running the pipeline displays the widget in the merge request, like this: + +  + +1. Clicking the **View Full Log** button in the widget takes you directly to the + plan output present in the pipeline logs: + +  + +### Example `.gitlab-ci.yaml` file + +```yaml +image: + name: hashicorp/terraform:light + entrypoint: + - '/usr/bin/env' + - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' + +# Default output file for Terraform plan +variables: + GITLAB_TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_PROJECT_NAME} + PLAN: plan.tfplan + PLAN_JSON: tfplan.json + TF_ROOT: ${CI_PROJECT_DIR} + +cache: + paths: + - .terraform + +before_script: + - apk --no-cache add jq + - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" + - cd ${TF_ROOT} + - terraform --version + - terraform init -backend-config="address=${GITLAB_TF_ADDRESS}" -backend-config="lock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="unlock_address=${GITLAB_TF_ADDRESS}/lock" -backend-config="username=${GITLAB_USER_LOGIN}" -backend-config="password=${GITLAB_TF_PASSWORD}" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" + +stages: + - validate + - build + - deploy + +validate: + stage: validate + script: + - terraform validate + +plan: + stage: build + script: + - terraform plan -out=$PLAN + - terraform show --json $PLAN | convert_report > $PLAN_JSON + artifacts: + name: plan + paths: + - ${TF_ROOT}/plan.tfplan + reports: + terraform: ${TF_ROOT}/tfplan.json + +# Separate apply job for manual launching Terraform as it can be destructive +# action. +apply: + stage: deploy + environment: + name: production + script: + - terraform apply -input=false $PLAN + dependencies: + - plan + when: manual + only: + - master + +``` diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md index 7b5ba14a5ae..495bfdeed6e 100644 --- a/doc/user/instance/clusters/index.md +++ b/doc/user/instance/clusters/index.md @@ -12,7 +12,7 @@ projects. ## Cluster precedence -GitLab will try [to match](../../../ci/environments.md#scoping-environments-with-specs) clusters in +GitLab will try [to match](../../../ci/environments/index.md#scoping-environments-with-specs) clusters in the following order: - Project-level clusters. @@ -20,11 +20,11 @@ the following order: - Instance-level clusters. To be selected, the cluster must be enabled and -match the [environment selector](../../../ci/environments.md#scoping-environments-with-specs). +match the [environment selector](../../../ci/environments/index.md#scoping-environments-with-specs). ## Cluster environments **(PREMIUM)** -For a consolidated view of which CI [environments](../../../ci/environments.md) +For a consolidated view of which CI [environments](../../../ci/environments/index.md) are deployed to the Kubernetes cluster, see the documentation for [cluster environments](../../clusters/environments.md). diff --git a/doc/user/instance_statistics/user_cohorts.md b/doc/user/instance_statistics/user_cohorts.md index 45140d5e852..b3ef99473e4 100644 --- a/doc/user/instance_statistics/user_cohorts.md +++ b/doc/user/instance_statistics/user_cohorts.md @@ -26,3 +26,4 @@ How do we measure the activity of users? GitLab considers a user active if: - The user has Git activity (whether push or pull). - The user visits pages related to Dashboards, Projects, Issues, and Merge Requests ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/54947) in GitLab 11.8). - The user uses the API +- The user uses the GraphQL API diff --git a/doc/user/markdown.md b/doc/user/markdown.md index a1f83a47015..102eb1f8f53 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -105,7 +105,7 @@ changing how standard Markdown is used: | Standard Markdown | Extended Markdown in GitLab | | ------------------------------------- | ------------------------- | -| [blockquotes](#blockquotes) | [multiline blockquotes](#multiline-blockquote) | +| [blockquotes](#blockquotes) | [multi-line blockquotes](#multiline-blockquote) | | [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | | [emphasis](#emphasis) | [multiple underscores in words](#multiple-underscores-in-words-and-mid-word-emphasis) | [headers](#headers) | [linkable Header IDs](#header-ids-and-links) | @@ -228,7 +228,7 @@ To make PlantUML available in GitLab, a GitLab administrator needs to enable it > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#emoji). -```md +```markdown Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: :zap: You can use emoji anywhere GFM is supported. :v: @@ -353,7 +353,7 @@ However, the wrapping tags can't be mixed: - [- deletion -} ``` -If your diff includes words in `` `code` `` font, make sure to escape each bactick `` ` `` with a +If your diff includes words in `` `code` `` font, make sure to escape each backtick `` ` `` with a backslash `\`, otherwise the diff highlight won't render correctly: ```markdown @@ -396,8 +396,8 @@ a^2+b^2=c^2 _Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -NOTE: **Note:** This also works for the asciidoctor `:stem: latexmath`. For details see -the [asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). +NOTE: **Note:** This also works for the Asciidoctor `:stem: latexmath`. For details see +the [Asciidoctor user manual](https://asciidoctor.org/docs/user-manual/#activating-stem-support). ### Special GitLab references @@ -608,7 +608,7 @@ Quote break. > If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab/blob/master/doc/user/markdown.md#multiline-blockquote). -GFM extends the standard Markdown standard by also supporting multiline blockquotes +GFM extends the standard Markdown standard by also supporting multi-line blockquotes fenced by `>>>`: ```markdown @@ -804,7 +804,7 @@ but_emphasis is_desired _here_ If you wish to emphasize only a part of a word, it can still be done with asterisks: -```md +```markdown perform*complicated*task do*this*and*do*that*and*another thing @@ -828,23 +828,31 @@ Reference tags can use letters and other characters. Avoid using lowercase `w` o (`_`) in footnote tag names until [this bug](https://gitlab.com/gitlab-org/gitlab/issues/24423) is resolved. -```markdown -A footnote reference tag looks like this: [^1] +<!-- +Do not edit the following codeblock. It uses HTML to skip the Vale ReferenceLinks test. +--> + +<pre class="highlight"><code>A footnote reference tag looks like this: [^1] This reference tag is a mix of letters and numbers. [^footnote-42] -[^1]: This is the text inside a footnote. +[^1]: This is the text inside a footnote. -[^footnote-42]: This is another footnote. -``` +[^footnote-42]: This is another footnote. +</code></pre> A footnote reference tag looks like this:[^1] This reference tag is a mix of letters and numbers.[^footnote-42] -[^1]: This is the text inside a footnote. +<!-- +Do not delete the single space before the [^1] and [^footnotes] references below. +These are used to force the Vale ReferenceLinks check to skip these examples. +--> + + [^1]: This is the text inside a footnote. -[^footnote-42]: This is another footnote. + [^footnote-42]: This is another footnote. ### Headers @@ -928,8 +936,11 @@ ___ Examples: -```markdown -Inline-style (hover to see title text): +<!-- +Do not edit the following codeblock. It uses HTML to skip the Vale ReferenceLinks test. +--> + +<pre class="highlight"><code>Inline-style (hover to see title text):  @@ -937,12 +948,12 @@ Reference-style (hover to see title text): ![alt text1][logo] -[logo]: img/markdown_logo.png "Title Text" -``` +[logo]: img/markdown_logo.png "Title Text" +</code></pre> <!-- -DO NOT change the name of markdown_logo.png. This is used for a test -in spec/controllers/help_controller_spec.rb. +DO NOT change the name of markdown_logo.png. This is used for a test in +spec/controllers/help_controller_spec.rb. --> Inline-style (hover to see title text): @@ -951,9 +962,12 @@ Inline-style (hover to see title text): Reference-style (hover to see title text): -![alt text][logo] +<!-- +The example below uses an in-line link to pass the Vale ReferenceLinks test. +Do not change to a reference style link. +--> -[logo]: img/markdown_logo.png "Title Text" + #### Videos @@ -962,7 +976,7 @@ Reference-style (hover to see title text): Image tags that link to files with a video extension are automatically converted to a video player. The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`: -```md +```markdown Here's a sample video:  @@ -979,7 +993,7 @@ Here's a sample video: Similar to videos, link tags for files with an audio extension are automatically converted to an audio player. The valid audio extensions are `.mp3`, `.oga`, `.ogg`, `.spx`, and `.wav`: -```md +```markdown Here's a sample audio clip:  @@ -1036,7 +1050,10 @@ are separated into their own lines: </dl> ``` -<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, Markdown will be fine in GitLab --> +<!-- +Note: The example below uses HTML to force correct rendering on docs.gitlab.com, +Markdown will be fine in GitLab. +--> <dl> <dt>Markdown in HTML</dt> @@ -1102,7 +1119,10 @@ PASTE LOGS HERE </details> ```` -<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, Markdown will be fine in GitLab --> +<!-- +The example below uses HTML to force correct rendering on docs.gitlab.com, Markdown +will work correctly in GitLab. +--> <details> <summary>Click this to collapse/fold.</summary> @@ -1167,8 +1187,11 @@ A new line due to the previous backslash. There are two ways to create links, inline-style and reference-style: -```markdown -- This is an [inline-style link](https://www.google.com) +<!-- +Do not edit the following codeblock. It uses HTML to skip the Vale ReferenceLinks test. +--> + +<pre class="highlight"><code>- This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) - This is a [relative link to a readme one directory higher](../README.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") @@ -1186,14 +1209,14 @@ Using references: Some text to show that the reference links can follow later. -[arbitrary case-insensitive reference text]: https://www.mozilla.org/en-US/ -[1]: https://slashdot.org -[link text itself]: https://www.reddit.com -``` +[arbitrary case-insensitive reference text]: https://www.mozilla.org/en-US/ +[1]: https://slashdot.org +[link text itself]: https://www.reddit.com +</code></pre> - This is an [inline-style link](https://www.google.com) - This is a [link to a repository file in the same directory](index.md) -- This is a [relative link to a readme one directory higher](../README.md) +- This is a [relative link to a README one directory higher](../README.md) - This is a [link that also has title text](https://www.google.com "This link takes you to Google!") Using header ID anchors: @@ -1203,15 +1226,16 @@ Using header ID anchors: Using references: -- This is a [reference-style link, see below][Arbitrary case-insensitive reference text] -- You can [use numbers for reference-style link definitions, see below][1] -- Or leave it empty and use the [link text itself][], see below. +<!-- +The example below uses in-line links to pass the Vale ReferenceLinks test. +Do not change to reference style links. +--> -Some text to show that the reference links can follow later. +- This is a [reference-style link, see below](https://www.mozilla.org/en-US/) +- You can [use numbers for reference-style link definitions, see below](https://slashdot.org) +- Or leave it empty and use the [link text itself](https://www.reddit.com), see below. -[arbitrary case-insensitive reference text]: https://www.mozilla.org/en-US/ -[1]: https://slashdot.org -[link text itself]: https://www.reddit.com +Some text to show that the reference links can follow later. NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki page, or a wiki page in a project file. The reason for this is that a wiki is always @@ -1220,7 +1244,7 @@ will point the link to `wikis/style` only when the link is inside of a wiki Mark #### URL auto-linking -GFM will autolink almost any URL you put into your text: +GFM will auto-link almost any URL you put into your text: ```markdown - https://www.google.com @@ -1251,7 +1275,7 @@ number, and count up from there. Examples: -```md +```markdown 1. First ordered list item 2. Another item - Unordered sub-list. @@ -1261,8 +1285,11 @@ Examples: 4. And another item. ``` -<!-- The "2." and "4." in the example above are changed to "1." below, to match the style standards on docs.gitlab.com --> -<!-- See https://docs.gitlab.com/ee/development/documentation/styleguide.html#lists --> +<!-- +The "2." and "4." in the example above are changed to "1." below, to match the style +standards on docs.gitlab.com. +See https://docs.gitlab.com/ee/development/documentation/styleguide.html#lists +--> 1. First ordered list item 1. Another item @@ -1275,7 +1302,7 @@ Examples: For an unordered list, add a `-`, `*` or `+`, followed by a space, at the start of each line for unordered lists, but you should not use a mix of them. -```md +```markdown Unordered lists can: - use @@ -1292,8 +1319,11 @@ They can even: + pluses ``` -<!-- The "*" and "+" in the example above are changed to "-" below, to match the style standards on docs.gitlab.com --> -<!-- See https://docs.gitlab.com/ee/development/documentation/styleguide.html#lists --> +<!-- +The "*" and "+" in the example above are changed to "-" below, to match the style +standards on docs.gitlab.com. +See https://docs.gitlab.com/ee/development/documentation/styleguide.html#lists +--> Unordered lists can: diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index ffbd8a848b5..a56a67d4959 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -20,7 +20,7 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. ## Getting started @@ -108,14 +108,19 @@ conan search Hello* --all --remote=gitlab ## Authenticating to the GitLab Conan Repository -You will need to generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +You will need a personal access token or deploy token. + +For repository authentication: + +- You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. +- You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. ### Adding a Conan user to the GitLab remote Once you have a personal access token and have [set your Conan remote](#adding-the-gitlab-package-registry-as-a-conan-remote), you can associate the token with the remote so you do not have to explicitly add them to each Conan command you run: ```shell -conan user <gitlab-username> -r gitlab -p <personal_access_token> +conan user <gitlab_username or deploy_token_username> -r gitlab -p <personal_access_token or deploy_token> ``` Note: **Note** @@ -130,7 +135,7 @@ Alternatively, you could explicitly include your credentials in any given comman For example: ```shell -CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> conan upload Hello/0.1@my-group+my-project/beta --all --remote=gitlab +CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> conan upload Hello/0.1@my-group+my-project/beta --all --remote=gitlab ``` ### Setting a default remote to your project (optional) @@ -148,7 +153,7 @@ This functionality is best suited for when you want to consume or install packag The rest of the example commands in this documentation assume that you have added a Conan user with your credentials to the `gitlab` remote and will not include the explicit credentials or remote option, but be aware that any of the commands could be run without having added a user or default remote: ```shell -`CONAN_LOGIN_USERNAME=<gitlab-username> CONAN_PASSWORD=<personal_access_token> <conan command> --remote=gitlab +`CONAN_LOGIN_USERNAME=<gitlab_username or deploy_token_username> CONAN_PASSWORD=<personal_access_token or deploy_token> <conan command> --remote=gitlab ``` ## Uploading a package @@ -274,7 +279,7 @@ To work with Conan commands within [GitLab CI/CD](./../../../ci/README.md), you It is easiest to provide the `CONAN_LOGIN_USERNAME` and `CONAN_PASSWORD` with each Conan command in your `.gitlab-ci.yml` file: -```yml +```yaml image: conanio/gcc7 create_package: diff --git a/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png b/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png Binary files differdeleted file mode 100644 index e2b606d024f..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_group_repositories_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png b/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png Binary files differnew file mode 100644 index 00000000000..14119abd56a --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_group_repositories_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png b/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png Binary files differdeleted file mode 100644 index 9e113be0a26..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png b/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png Binary files differnew file mode 100644 index 00000000000..7286007b953 --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repositories_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png Binary files differdeleted file mode 100644 index e94aab58a1d..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png Binary files differnew file mode 100644 index 00000000000..f7c3aafcc8e --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repositories_with_quickstart_v13_0.png diff --git a/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png b/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png Binary files differdeleted file mode 100644 index b911ffea935..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_repository_details_v12.10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png b/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png Binary files differnew file mode 100644 index 00000000000..088e80221de --- /dev/null +++ b/doc/user/packages/container_registry/img/container_registry_repository_details_v13.0.png diff --git a/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png b/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png Binary files differdeleted file mode 100644 index 40abc7eda9b..00000000000 --- a/doc/user/packages/container_registry/img/container_registry_tags_v12_10.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/expiration-policy-app.png b/doc/user/packages/container_registry/img/expiration-policy-app.png Binary files differdeleted file mode 100644 index e2d3d668e38..00000000000 --- a/doc/user/packages/container_registry/img/expiration-policy-app.png +++ /dev/null diff --git a/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png Binary files differnew file mode 100644 index 00000000000..93c9e00a8f5 --- /dev/null +++ b/doc/user/packages/container_registry/img/expiration_policy_app_v13_0.png diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index 5505a4503ca..5e642e1e21c 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -8,6 +8,7 @@ > login to GitLab's Container Registry. > - Multiple level image names support was added in GitLab 9.1. > - The group level Container Registry was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23315) in GitLab 12.10. +> - Searching by image repository name was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31322) in GitLab 13.0. NOTE: **Note:** This document is the user guide. To learn how to enable GitLab Container @@ -19,14 +20,14 @@ have its own space to store its Docker images. You can read more about Docker Registry at <https://docs.docker.com/registry/introduction/>. - + ## Enable the Container Registry for your project CAUTION: **Warning:** The Container Registry follows the visibility settings of the project. If the project is public, so is the Container Registry. -If you cannot find the **Packages > Container Registry** entry under your +If you cannot find the **Packages & Registries > Container Registry** entry under your project's sidebar, it is not enabled in your GitLab instance. Ask your administrator to enable GitLab Container Registry following the [administration documentation](../../../administration/packages/container_registry.md). @@ -44,7 +45,7 @@ project: projects this might be enabled by default. For existing projects (prior GitLab 8.8), you will have to explicitly enable it. 1. Press **Save changes** for the changes to take effect. You should now be able - to see the **Packages > Container Registry** link in the sidebar. + to see the **Packages & Registries > Container Registry** link in the sidebar. ## Control Container Registry from within GitLab @@ -53,13 +54,14 @@ for both projects and groups. ### Control Container Registry for your project -Navigate to your project's **{package}** **Packages > Container Registry**. +Navigate to your project's **{package}** **Packages & Registries > Container Registry**. - + This view will: - Show all the image repositories that belong to the project. +- Allow you to filter image repositories by their name. - Allow you to [delete](#delete-images-from-within-gitlab) one or more image repository. - Allow you to navigate to the image repository details page. - Show a **Quick start** dropdown with the most common commands to log in, build and push @@ -67,9 +69,9 @@ This view will: ### Control Container Registry for your group -Navigate to your groups's **{package}** **Packages > Container Registry**. +Navigate to your groups's **{package}** **Packages & Registries > Container Registry**. - + This view will: @@ -81,7 +83,7 @@ This view will: Clicking on the name of any image repository will navigate to the details. - + NOTE: **Note:** The following page has the same functionalities both in the **Group level container registry** @@ -108,7 +110,7 @@ For more information on running Docker containers, visit the ## Authenticating to the GitLab Container Registry -If you visit the **Packages > Container Registry** link under your project's +If you visit the **Packages & Registries > Container Registry** link under your project's menu, you can see the explicit instructions to login to the Container Registry using your GitLab credentials. @@ -135,7 +137,7 @@ The minimum scope needed for both of them is `read_registry`. Example of using a token: -```sh +```shell docker login registry.example.com -u <username> -p <token> ``` @@ -152,14 +154,14 @@ docker push registry.example.com/group/project/image Your image will be named after the following scheme: -```text +```plaintext <registry URL>/<namespace>/<project>/<image> ``` GitLab supports up to three levels of image repository names. The following examples of image tags are valid: -```text +```plaintext registry.example.com/group/project:some-tag registry.example.com/group/project/image:latest registry.example.com/group/project/my/image:rc1 @@ -204,7 +206,7 @@ Available for all projects, though more suitable for public ones: your Docker images and has read/write access to the Registry. This is ephemeral, so it's only valid for one job. You can use the following example as-is: - ```sh + ```shell docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY ``` @@ -219,7 +221,7 @@ For private and internal projects: Replace the `<username>` and `<access_token>` in the following example: - ```sh + ```shell docker login -u <username> -p <access_token> $CI_REGISTRY ``` @@ -229,7 +231,7 @@ For private and internal projects: Once created, you can use the special environment variables, and GitLab CI/CD will fill them in for you. You can use the following example as-is: - ```sh + ```shell docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY ``` @@ -389,7 +391,7 @@ the deleted images. To delete images from within GitLab: -1. Navigate to your project's or group's **{package}** **Packages > Container Registry**. +1. Navigate to your project's or group's **{package}** **Packages & Registries > Container Registry**. 1. From the **Container Registry** page, you can select what you want to delete, by either: @@ -401,7 +403,7 @@ To delete images from within GitLab: 1. In the dialog box, click **Remove tag**. -  +  ### Delete images using the API @@ -490,7 +492,7 @@ older tags and images are regularly removed from the Container Registry. NOTE: **Note:** Expiration policies for projects created before GitLab 12.8 may be enabled by an admin in the [CI/CD Package Registry settings](./../../admin_area/settings/index.md#cicd). -Note the inherant [risks involved](./index.md#use-with-external-container-registries). +Note the inherent [risks involved](./index.md#use-with-external-container-registries). It is possible to create a per-project expiration policy, so that you can make sure that older tags and images are regularly removed from the Container Registry. @@ -503,23 +505,25 @@ then goes through a process of excluding tags from it until only the ones to be 1. Evaluates the `name_regex`, excluding non-matching names from the list. 1. Excludes any tags that do not have a manifest (not part of the options). 1. Orders the remaining tags by `created_date`. -1. Excludes from the list the N tags based on the `keep_n` value (Expiration latest). +1. Excludes from the list the N tags based on the `keep_n` value (Number of tags to retain). 1. Excludes from the list the tags older than the `older_than` value (Expiration interval). +1. Excludes from the list any tags matching the `name_regex_keep` value (Images to preserve). 1. Finally, the remaining tags in the list are deleted from the Container Registry. ### Managing project expiration policy through the UI To manage project expiration policy, navigate to **{settings}** **Settings > CI/CD > Container Registry tag expiration policy**. - + The UI allows you to configure the following: - **Expiration policy:** enable or disable the expiration policy. - **Expiration interval:** how long tags are exempt from being deleted. - **Expiration schedule:** how often the cron job checking the tags should run. -- **Expiration latest:** how many tags to _always_ keep for each image. +- **Number of tags to retain:** how many tags to _always_ keep for each image. - **Docker tags with names matching this regex pattern will expire:** the regex used to determine what tags should be expired. To qualify all tags for expiration, use the default value of `.*`. +- **Docker tags with names matching this regex pattern will be preserved:** the regex used to determine what tags should be preserved. To preserve all tags, use the default value of `.*`. ### Managing project expiration policy through the API @@ -527,16 +531,10 @@ You can set, update, and disable the expiration policies using the GitLab API. Examples: -- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, and the policy is enabled: +- Select all tags, keep at least 1 tag per image, expire any tag older than 14 days, run once a month, preserve any images with the name `master` and the policy is enabled: ```shell - curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":".*"}}' 'https://gitlab.example.com/api/v4/projects/2' - ``` - -- Select only tags with a name that contains `stable`, keep at least 50 tag per image, expire any tag older than 7 days, run every day, and the policy is enabled: - - ```shell - curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1day","enabled":true,"keep_n":50"older_than":"7d","name_regex":"*stable"}}' 'https://gitlab.example.com/api/v4/projects/2' + curl --request PUT --header 'Content-Type: application/json;charset=UTF-8' --header "PRIVATE-TOKEN: <your_access_token>" --data-binary '{"container_expiration_policy_attributes":{"cadence":"1month","enabled":true,"keep_n":1,"older_than":"14d","name_regex":"","name_regex_delete":".*","name_regex_keep":".*-master"}}' 'https://gitlab.example.com/api/v4/projects/2' ``` See the API documentation for further details: [Edit project](../../../api/projects.md#edit-project). @@ -544,7 +542,7 @@ See the API documentation for further details: [Edit project](../../../api/proje ### Use with external container registries When using an [external container registry](./../../../administration/packages/container_registry.md#use-an-external-container-registry-with-gitlab-as-an-auth-endpoint), -running an experation policy on a project may have some performance risks. If a project is going to run +running an expiration policy on a project may have some performance risks. If a project is going to run a policy that will remove large quantities of tags (in the thousands), the GitLab background jobs that run the policy may get backed up or fail completely. It is recommended you only enable container expiration policies for projects that were created before GitLab 12.8 if you are confident the amount of tags @@ -552,10 +550,12 @@ being cleaned up will be minimal. ## Limitations -Moving or renaming existing Container Registry repositories is not supported +- Moving or renaming existing Container Registry repositories is not supported once you have pushed images, because the images are signed, and the signature includes the repository name. To move or rename a repository with a Container Registry, you will have to delete all existing images. +- Prior to GitLab 12.10, any tags that use the same image ID as the `latest` tag +will not be deleted by the expiration policy. ## Troubleshooting the GitLab Container Registry @@ -577,3 +577,47 @@ Troubleshooting the GitLab Container Registry, most of the times, requires administration access to the GitLab server. [Read how to troubleshoot the Container Registry](../../../administration/packages/container_registry.md#troubleshooting). + +### Unable to change path or transfer a project + +If you try to change a project's path or transfer a project to a new namespace, +you may receive one of the following errors: + +- "Project cannot be transferred, because tags are present in its container registry." +- "Namespace cannot be moved because at least one project has tags in container registry." + +This issue occurs when the project has images in the Container Registry. +You must delete or move these images before you can change the path or transfer +the project. + +The following procedure uses these sample project names: + +- For the current project: `example.gitlab.com/org/build/sample_project/cr:v2.9.1` +- For the new project: `example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1` + +Use your own URLs to complete the following steps: + +1. Download the Docker images on your computer: + + ```shell + docker login example.gitlab.com + docker pull example.gitlab.com/org/build/sample_project/cr:v2.9.1 + ``` + +1. Rename the images to match the new project name: + + ```shell + docker tag example.gitlab.com/org/build/sample_project/cr:v2.9.1 example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1 + ``` + +1. Delete the images in both projects by using the [UI](#delete-images) or [API](../../../api/packages.md#delete-a-project-package). + There may be a delay while the images are queued and deleted. +1. Change the path or transfer the project by going to **Settings > General** + and expanding **Advanced**. +1. Restore the images: + + ```shell + docker push example.gitlab.com/new_org/build/new_sample_project/cr:v2.9.1 + ``` + +Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/18383) for details. diff --git a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png b/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png Binary files differindex 0b94efdd83e..e550d296d5a 100644 --- a/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png +++ b/doc/user/packages/dependency_proxy/img/group_dependency_proxy.png diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index cfdcd9821fb..be9710053dd 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -12,7 +12,7 @@ receiving a request and returning the upstream image from a registry, acting as a pull-through cache. The dependency proxy is available in the group level. To access it, navigate to -a group's **Packages > Dependency Proxy**. +a group's **Packages & Registries > Dependency Proxy**.  @@ -33,7 +33,7 @@ The following dependency proxies are supported. With the Docker dependency proxy, you can use GitLab as a source for a Docker image. To get a Docker image into the dependency proxy: -1. Find the proxy URL on your group's page under **Packages > Dependency Proxy**, +1. Find the proxy URL on your group's page under **Packages & Registries > Dependency Proxy**, for example `gitlab.com/groupname/dependency_proxy/containers`. 1. Trigger GitLab to pull the Docker image you want (e.g., `alpine:latest` or `linuxserver/nextcloud:latest`) and store it in the proxy storage by using diff --git a/doc/user/packages/img/group_packages_list_v12_10.png b/doc/user/packages/img/group_packages_list_v12_10.png Binary files differdeleted file mode 100644 index ba9f2892961..00000000000 --- a/doc/user/packages/img/group_packages_list_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/group_packages_list_v13_0.png b/doc/user/packages/img/group_packages_list_v13_0.png Binary files differnew file mode 100644 index 00000000000..8cf3fd1a131 --- /dev/null +++ b/doc/user/packages/img/group_packages_list_v13_0.png diff --git a/doc/user/packages/img/package_detail_v12_10.png b/doc/user/packages/img/package_detail_v12_10.png Binary files differdeleted file mode 100644 index b2cd8e31955..00000000000 --- a/doc/user/packages/img/package_detail_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/package_detail_v13_0.png b/doc/user/packages/img/package_detail_v13_0.png Binary files differnew file mode 100644 index 00000000000..dcfbc0a4787 --- /dev/null +++ b/doc/user/packages/img/package_detail_v13_0.png diff --git a/doc/user/packages/img/project_packages_list_v12_10.png b/doc/user/packages/img/project_packages_list_v12_10.png Binary files differdeleted file mode 100644 index 2dfb92fa796..00000000000 --- a/doc/user/packages/img/project_packages_list_v12_10.png +++ /dev/null diff --git a/doc/user/packages/img/project_packages_list_v13_0.png b/doc/user/packages/img/project_packages_list_v13_0.png Binary files differnew file mode 100644 index 00000000000..468a6fe6467 --- /dev/null +++ b/doc/user/packages/img/project_packages_list_v13_0.png diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 8e98dd70346..132a64d99a3 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -18,26 +18,28 @@ The Packages feature allows GitLab to act as a repository for the following: ## Enable the Package Registry for your project -If you cannot find the **{package}** **Packages > List** entry under your -project's sidebar, it is not enabled in your GitLab instance. Ask your -administrator to enable GitLab Package Registry following the administration -documentation. +If you cannot find the **{package}** **Packages & Registries > Package +Registry** entry under your project's sidebar, ensure that: -Once enabled for your GitLab instance, to enable Package Registry for your -project: +1. The GitLab Package Registry has been enabled by your administrator (following + [this documentation](../../administration/packages/index.md)); and +1. The Package Registry has been enabled for your project. + +Once an administrator has enabled the GitLab Package Registry for your GitLab +instance, to enable Package Registry for your project: 1. Go to your project's **Settings > General** page. 1. Expand the **Visibility, project features, permissions** section and enable the **Packages** feature on your project. 1. Press **Save changes** for the changes to take effect. You should now be able to -see the **Packages > List** link in the sidebar. +see the **Packages & Registries > Package Registry** link in the sidebar. ### View Packages for your project -Navigating to your project's **{package}** **Packages > List** will show a list +Navigating to your project's **{package}** **Packages & Registries > Package Registry** will show a list of all packages that have been added to your project. - + On this page, you can: @@ -51,9 +53,9 @@ On this page, you can: ### View Packages for your group You can view all packages belonging to a group by navigating to **{package}** -**Packages > List** from the group sidebar. +**Packages & Registries > Package Registry** from the group sidebar. - + On this page, you can: @@ -68,7 +70,7 @@ On this page, you can: Additional package information can be viewed by browsing to the package details page from the either the project or group list. - + On this page you can: @@ -112,7 +114,6 @@ are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_reques | [Opkg](https://gitlab.com/gitlab-org/gitlab/issues/36894) | Optimize your work with OpenWrt using Opkg repositories. | | [P2](https://gitlab.com/gitlab-org/gitlab/issues/36895) | Host all your Eclipse plugins in your own GitLab P2 repository. | | [Puppet](https://gitlab.com/gitlab-org/gitlab/issues/36897) | Configuration management meets repository management with Puppet repositories. | -| [PyPi](https://gitlab.com/gitlab-org/gitlab/issues/10483) | Host PyPi distributions. | | [RPM](https://gitlab.com/gitlab-org/gitlab/issues/5932) | Distribute RPMs directly from GitLab. | | [RubyGems](https://gitlab.com/gitlab-org/gitlab/issues/803) | Use GitLab to host your own gems. | | [SBT](https://gitlab.com/gitlab-org/gitlab/issues/36898) | Resolve dependencies from and deploy build output to SBT repositories when running SBT builds. | diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index b4aec11d029..51e62dc871e 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -21,17 +21,19 @@ to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. Next, you must configure your project to authorize with the GitLab Maven repository. -## Getting Started +## Getting Started with Maven This section will cover installing Maven and building a package. This is a quickstart to help if you're new to building Maven packages. If you're already using Maven and understand how to build your own packages, move onto the [next section](#adding-the-gitlab-package-registry-as-a-maven-remote). +Maven repositories work well with Gradle, too. Move onto [getting started with Gradle](#getting-started-with-gradle) if you want to setup a Gradle project. + ### Installing Maven The required minimum versions are: @@ -96,20 +98,129 @@ your project has been set up successfully: You should see a new directory where you ran this command matching your `DartifactId` parameter (in this case it should be `my-project`). +## Getting started with Gradle + +This section will cover installing Gradle and initializing a Java project. This is a +quickstart to help if you're new to Gradle. If you're already +using Gradle and understand how to build your own packages, move onto the +[next section](#adding-the-gitlab-package-registry-as-a-maven-remote). + +### Installing Gradle + +Installation is needed only if you want to create a new Gradle project. Follow +instructions at [gradle.org](https://gradle.org/install/) to download and install +Gradle for your local development environment. + +Verify you can use Gradle in your terminal by running: + +```shell +gradle -version +``` + +If you want to use an existing Gradle project, installation is not necessary. +Simply execute `gradlew` (on Linux) or `gradlew.bat` (on Windows) in the project +directory instead. + +You should see something imilar to the below printed in the output: + +```plaintext +------------------------------------------------------------ +Gradle 6.0.1 +------------------------------------------------------------ + +Build time: 2019-11-18 20:25:01 UTC +Revision: fad121066a68c4701acd362daf4287a7c309a0f5 + +Kotlin: 1.3.50 +Groovy: 2.5.8 +Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 +JVM: 11.0.5 (Oracle Corporation 11.0.5+10) +OS: Windows 10 10.0 amd64 +``` + +### Creating a project in Gradle + +Understanding how to create a full Java project in Gradle is outside the scope of this +guide, but you can follow the steps below to create a new project that can be +published to the GitLab Package Registry. + +Start by opening your terminal and creating a directory where you would like to +store the project in your environment. From inside the directory, you can run +the following Maven command to initialize a new package: + +```shell +gradle init +``` + +The output should be + +```plaintext +Select type of project to generate: + 1: basic + 2: application + 3: library + 4: Gradle plugin +Enter selection (default: basic) [1..4] +``` + +Enter `3` to create a new Library project. The output should be: + +```plaintext +Select implementation language: + 1: C++ + 2: Groovy + 3: Java + 4: Kotlin + 5: Scala + 6: Swift +``` + +Enter `3` to create a new Java Library project. The output should be: + +```plaintext +Select build script DSL: + 1: Groovy + 2: Kotlin +Enter selection (default: Groovy) [1..2] +``` + +Choose `1` to create a new Java Library project which will be described in Groovy DSL. The output should be: + +```plaintext +Select test framework: + 1: JUnit 4 + 2: TestNG + 3: Spock + 4: JUnit Jupiter +``` + +Choose `1` to initialize the project with JUnit 4 testing libraries. The output should be: + +```plaintext +Project name (default: test): +``` + +Enter a project name or hit enter to use the directory name as project name. + ## Adding the GitLab Package Registry as a Maven remote The next step is to add the GitLab Package Registry as a Maven remote. If a project is private or you want to upload Maven artifacts to GitLab, credentials will need to be provided for authorization too. Support is available -for [personal access tokens](#authenticating-with-a-personal-access-token) and -[CI job tokens](#authenticating-with-a-ci-job-token) only. -[Deploy tokens](../../project/deploy_tokens/index.md) and regular username/password +for [personal access tokens](#authenticating-with-a-personal-access-token), +[CI job tokens](#authenticating-with-a-ci-job-token), and +[deploy tokens](../../project/deploy_tokens/index.md) only. Regular username/password credentials do not work. ### Authenticating with a personal access token To authenticate with a [personal access token](../../profile/personal_access_tokens.md), -set the scope to `api` and add a corresponding section to your +set the scope to `api` when creating one, and add it to your Maven or Gradle configuration +files. + +#### Authenticating with a personal access token in Maven + +Add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: ```xml @@ -130,13 +241,43 @@ set the scope to `api` and add a corresponding section to your </settings> ``` +#### Authenticating with a personal access token in Gradle + +Create a file `~/.gradle/gradle.properties` with the following content: + +```groovy +gitLabPrivateToken=REPLACE_WITH_YOUR_PERSONAL_ACCESS_TOKEN +``` + +Add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Private-Token' + value = gitLabPrivateToken + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + You should now be able to upload Maven artifacts to your project. ### Authenticating with a CI job token -If you're using Maven with GitLab CI/CD, a CI job token can be used instead +If you're using GitLab CI/CD, a CI job token can be used instead of a personal access token. +#### Authenticating with a CI job token in Maven + To authenticate with a CI job token, add a corresponding section to your [`settings.xml`](https://maven.apache.org/settings.html) file: @@ -161,6 +302,81 @@ To authenticate with a CI job token, add a corresponding section to your You can read more on [how to create Maven packages using GitLab CI/CD](#creating-maven-packages-with-gitlab-cicd). +#### Authenticating with a CI job token in Gradle + +To authenticate with a CI job token, add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Job-Token' + value = '${CI_JOB_TOKEN}' + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + +### Authenticating with a deploy token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. + +To authenticate with a [deploy token](./../../project/deploy_tokens/index.md), +set the scope to `api` when creating one, and add it to your Maven or Gradle configuration +files. + +#### Authenticating with a deploy token in Maven + +Add a corresponding section to your +[`settings.xml`](https://maven.apache.org/settings.html) file: + +```xml +<settings> + <servers> + <server> + <id>gitlab-maven</id> + <configuration> + <httpHeaders> + <property> + <name>Deploy-Token</name> + <value>REPLACE_WITH_YOUR_DEPLOY_TOKEN</value> + </property> + </httpHeaders> + </configuration> + </server> + </servers> +</settings> +``` + +#### Authenticating with a deploy token in Gradle + +To authenticate with a deploy token, add a repositories section to your +[`build.gradle`](https://docs.gradle.org/current/userguide/tutorial_using_tasks.html) +file: + +```groovy +repositories { + maven { + url "https://<gitlab-url>/api/v4/groups/<group>/-/packages/maven" + name "GitLab" + credentials(HttpHeaderCredentials) { + name = 'Deploy-Token' + value = '<deploy-token>' + } + authentication { + header(HttpHeaderAuthentication) + } + } +} +``` + ## Configuring your project to use the GitLab Maven repository URL To download and upload packages from GitLab, you need a `repository` and @@ -185,7 +401,7 @@ the `distributionManagement` section. ### Project level Maven endpoint The example below shows how the relevant `repository` section of your `pom.xml` -would look like: +would look like in Maven: ```xml <repositories> @@ -206,6 +422,17 @@ would look like: </distributionManagement> ``` +The corresponding section in Gradle would look like this: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -223,7 +450,7 @@ project's ID can be used for uploading. ### Group level Maven endpoint -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in GitLab Premium 11.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8798) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the group level endpoint for @@ -259,6 +486,17 @@ the `distributionManagement` section: </distributionManagement> ``` +For Gradle, the corresponding repositories section would look like: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/groups/GROUP_ID/-/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -275,7 +513,7 @@ For retrieving artifacts, you can use either the ### Instance level Maven endpoint -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in GitLab Premium 11.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8274) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.7. If you rely on many packages, it might be inefficient to include the `repository` section with a unique URL for each package. Instead, you can use the instance level endpoint for @@ -314,6 +552,17 @@ the `distributionManagement` section: </distributionManagement> ``` +The corresponding repositories section in Gradle would look like: + +```groovy +repositories { + maven { + url "https://gitlab.com/api/v4/packages/maven" + name "GitLab" + } +} +``` + The `id` must be the same with what you [defined in `settings.xml`](#adding-the-gitlab-package-registry-as-a-maven-remote). @@ -333,7 +582,9 @@ project's ID can be used for uploading. Once you have set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) and [configured your project](#configuring-your-project-to-use-the-gitlab-maven-repository-url), -test to upload a Maven artifact from a project of yours: +test to upload a Maven artifact from a project of yours. + +### Upload using Maven ```shell mvn deploy @@ -353,7 +604,51 @@ You should also see that the upload was uploaded to the correct registry: Uploading to gitlab-maven: https://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.jar ``` -You can then navigate to your project's **Packages** page and see the uploaded +### Upload using Gradle + +Add the Gradle plugin [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) to the plugins section: + +```groovy +plugins { + id 'java' + id 'maven-publish' +} +``` + +Add a `publishing` section: + +```groovy +publishing { + publications { + library(MavenPublication) { + from components.java + } + } + repositories { + maven { + url "https://gitlab.com/api/v4/projects/<PROJECT_ID>/packages/maven" + credentials(HttpHeaderCredentials) { + name = "Private-Token" + value = gitLabPrivateToken // the variable resides in ~/.gradle/gradle.properties + } + authentication { + header(HttpHeaderAuthentication) + } + } + } +} +``` + +Replace `PROJECT_ID` with your project ID which can be found on the home page +of your project. + +Run the publish task: + +```shell +gradle publish +``` + +You can then navigate to your project's **Packages & Registries** page and see the uploaded artifacts or even delete them. ## Installing a package @@ -362,7 +657,7 @@ Installing a package from the GitLab Package Registry requires that you set up the [remote and authentication](#adding-the-gitlab-package-registry-as-a-maven-remote) as above. Once this is completed, there are two ways for installaing a package. -### Install with `mvn install` +### Install using Maven with `mvn install` Add the dependency manually to your project `pom.xml` file. To add the example created above, the XML would look like: @@ -388,7 +683,7 @@ downloaded from the GitLab Package Registry: Downloading from gitlab-maven: http://gitlab.com/api/v4/projects/PROJECT_ID/packages/maven/com/mycompany/mydepartment/my-project/1.0-SNAPSHOT/my-project-1.0-20200128.120857-1.pom ``` -### Install with `mvn dependency:get` +#### Install with `mvn dependency:get` The second way to install packages is to use the Maven commands directly. Inside your project directory, run: @@ -404,6 +699,16 @@ TIP: **Tip:** Both the XML block and Maven command are readily copy and pastable from the Package details page, allowing for quick and easy installation. +### Install using Gradle + +Add a [dependency](https://docs.gradle.org/current/userguide/declaring_dependencies.html) to build.gradle in the dependencies section: + +```groovy +dependencies { + implementation 'com.mycompany.mydepartment:my-project:1.0-SNAPSHOT' +} +``` + ## Removing a package In the packages view of your project page, you can delete packages by clicking @@ -413,11 +718,15 @@ page. ## Creating Maven packages with GitLab CI/CD Once you have your repository configured to use the GitLab Maven Repository, -you can configure GitLab CI/CD to build new packages automatically. The example below -shows how to create a new package each time the `master` branch is updated: +you can configure GitLab CI/CD to build new packages automatically. + +### Creating Maven packages with GitLab CI/CD using Maven + +The example below shows how to create a new package each time the `master` branch +is updated: 1. Create a `ci_settings.xml` file that will serve as Maven's `settings.xml` file. - Add the server section with the same id you defined in your `pom.xml` file. + Add the server section with the same ID you defined in your `pom.xml` file. For example, in our case it's `gitlab-maven`: ```xml @@ -481,6 +790,31 @@ user's home location (in this case the user is `root` since it runs in a Docker container), and Maven will utilize the configured CI [environment variables](../../../ci/variables/README.md#predefined-environment-variables). +### Creating Maven packages with GitLab CI/CD using Gradle + +The example below shows how to create a new package each time the `master` branch +is updated: + +1. Make sure you use the Job-Token authentication as described in ["Authenticating with a CI job token in Gradle"](#authenticating-with-a-ci-job-token-in-gradle). + +1. Add a `deploy` job to your `.gitlab-ci.yml` file: + + ```yaml + deploy: + image: gradle:latest + script: + - 'gradle publish' + only: + - master + ``` + +1. Push those files to your repository. + +The next time the `deploy` job runs, it will copy `ci_settings.xml` to the +user's home location (in this case the user is `root` since it runs in a +Docker container), and Maven will use the configured CI +[environment variables](../../../ci/variables/README.md#predefined-environment-variables). + ## Troubleshooting ### Useful Maven command line options diff --git a/doc/user/packages/npm_registry/index.md b/doc/user/packages/npm_registry/index.md index e66b3d1ac63..b909646431b 100644 --- a/doc/user/packages/npm_registry/index.md +++ b/doc/user/packages/npm_registry/index.md @@ -23,7 +23,7 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. Before proceeding to authenticating with the GitLab NPM Registry, you should get familiar with the package naming convention. @@ -100,14 +100,15 @@ configure GitLab as a remote registry. If a project is private or you want to upload an NPM package to GitLab, credentials will need to be provided for authentication. [Personal access tokens](../../profile/personal_access_tokens.md) +and [deploy tokens](../../project/deploy_tokens/index.md) are preferred, but support is available for [OAuth tokens](../../../api/oauth2.md#resource-owner-password-credentials-flow). -CAUTION: **2FA is only supported with personal access tokens:** -If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api`. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. +CAUTION: **Two-factor authentication (2FA) is only supported with personal access tokens:** +If you have 2FA enabled, you need to use a [personal access token](../../profile/personal_access_tokens.md) with OAuth headers with the scope set to `api` or a [deploy token](../../project/deploy_tokens/index.md) with `read_package_registry` or `write_package_registry` scopes. Standard OAuth tokens won't be able to authenticate to the GitLab NPM Registry. -### Authenticating with a personal access token +### Authenticating with a personal access token or deploy token -To authenticate with a [personal access token](../../profile/personal_access_tokens.md), +To authenticate with a [personal access token](../../profile/personal_access_tokens.md) or [deploy token](../../project/deploy_tokens/index.md), set your NPM configuration: ```shell @@ -125,7 +126,7 @@ npm config set '//gitlab.com/api/v4/projects/<your_project_id>/packages/npm/:_au ``` Replace `<your_project_id>` with your project ID which can be found on the home page -of your project and `<your_token>` with your personal access token. +of your project and `<your_token>` with your personal access token or deploy token. If you have a self-managed GitLab installation, replace `gitlab.com` with your domain name. @@ -160,7 +161,7 @@ Then, you could run `npm publish` either locally or via GitLab CI/CD: > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/9104) in GitLab Premium 12.5. -If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token. +If you’re using NPM with GitLab CI/CD, a CI job token can be used instead of a personal access token or deploy token. The token will inherit the permissions of the user that generates the pipeline. Add a corresponding section to your `.npmrc` file: @@ -195,7 +196,7 @@ you can upload an NPM package to your project: npm publish ``` -You can then navigate to your project's **Packages** page and see the uploaded +You can then navigate to your project's **Packages & Registries** page and see the uploaded packages or even delete them. If you attempt to publish a package with a name that already exists within @@ -286,11 +287,11 @@ page. ## Publishing a package with CI/CD To work with NPM commands within [GitLab CI/CD](./../../../ci/README.md), you can use -`CI_JOB_TOKEN` in place of the personal access token in your commands. +`CI_JOB_TOKEN` in place of the personal access token or deploy token in your commands. A simple example `.gitlab-ci.yml` file for publishing NPM packages: -```yml +```yaml image: node:latest stages: @@ -323,9 +324,9 @@ info Visit https://classic.yarnpkg.com/en/docs/cli/install for documentation abo ``` In this case, try adding this to your `.npmrc` file (and replace `<your_token>` -with your personal access token): +with your personal access token or deploy token): -```text +```plaintext //gitlab.com/api/v4/projects/:_authToken=<your_token> ``` @@ -363,6 +364,14 @@ You do not need a token to run `npm install` unless your project is private (the NPM_TOKEN=<your_token> npm install ``` +### `npm install` returns `npm ERR! 403 Forbidden` + +- Check that your token is not expired and has appropriate permissions. +- Check if you have attempted to publish a package with a name that already exists within a given scope. +- Ensure the scoped packages URL includes a trailing slash: + - Correct: `//gitlab.com/api/v4/packages/npm/` + - Incorrect: `//gitlab.com/api/v4/packages/npm` + ## NPM dependencies metadata > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11867) in GitLab Premium 12.6. diff --git a/doc/user/packages/nuget_repository/index.md b/doc/user/packages/nuget_repository/index.md index ed936b546d2..d9efb3239a8 100644 --- a/doc/user/packages/nuget_repository/index.md +++ b/doc/user/packages/nuget_repository/index.md @@ -61,14 +61,16 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. ## Adding the GitLab NuGet Repository as a source to NuGet You will need the following: - Your GitLab username. -- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +- A personal access token or deploy token. For repository authentication: + - You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api`. + - You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the scope set to `read_package_registry`, `write_package_registry`, or both. - A suitable name for your source. - Your project ID which can be found on the home page of your project. @@ -83,7 +85,7 @@ You can now add a new source to NuGet with: To add the GitLab NuGet Repository as a source with `nuget`: ```shell -nuget source Add -Name <source_name> -Source "https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username> -Password <gitlab_personal_access_token> +nuget source Add -Name <source_name> -Source "https://gitlab-instance.example.com/api/v4/projects/<your_project_id>/packages/nuget/index.json" -UserName <gitlab_username or deploy_token_username> -Password <gitlab_personal_access_token or deploy_token> ``` Where: @@ -107,8 +109,8 @@ nuget source Add -Name "GitLab" -Source "https//gitlab.example/api/v4/projects/1 - **Location**: `https://gitlab.com/api/v4/projects/<your_project_id>/packages/nuget/index.json` - Replace `<your_project_id>` with your project ID. - If you have a self-managed GitLab installation, replace `gitlab.com` with your domain name. - - **Username**: Your GitLab username - - **Password**: Your personal access token + - **Username**: Your GitLab username or deploy token username + - **Password**: Your personal access token or deploy token  @@ -131,8 +133,8 @@ To add the GitLab NuGet Repository as a source for .NET, create a file named `nu </packageSources> <packageSourceCredentials> <gitlab> - <add key="Username" value="<gitlab_username>" /> - <add key="ClearTextPassword" value="<gitlab_personal_access_token>" /> + <add key="Username" value="<gitlab_username or deploy_token_username>" /> + <add key="ClearTextPassword" value="<gitlab_personal_access_token or deploy_token>" /> </gitlab> </packageSourceCredentials> </configuration> @@ -201,7 +203,7 @@ nuget install <package_id> -OutputDirectory <output_directory> \ Where: -- `<package_id>` is the package id. +- `<package_id>` is the package ID. - `<output_directory>` is the output directory, where the package will be installed. - `<package_version>` (Optional) is the package version. - `<source_name>` (Optional) is the source name. @@ -222,5 +224,5 @@ dotnet add package <package_id> \ Where: -- `<package_id>` is the package id. +- `<package_id>` is the package ID. - `<package_version>` (Optional) is the package version. diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index 11d7b828813..979f7a3c966 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -26,10 +26,132 @@ by default. To enable it for existing projects, or if you want to disable it: 1. Find the Packages feature and enable or disable it. 1. Click on **Save changes** for the changes to take effect. -You should then be able to see the **Packages** section on the left sidebar. +You should then be able to see the **Packages & Registries** section on the left sidebar. + +## Getting started + +This section will cover creating a new example PyPi package to upload. This is a +quickstart to test out the **GitLab PyPi Registry**. If you already understand how +to build and publish your own packages, move on to the [next section](#adding-the-gitlab-pypi-repository-as-a-source). + +### Create a project + +Understanding how to create a full Python project is outside the scope of this +guide, but you can create a small package to test out the registry. Start by +creating a new directory called `MyPyPiPackage`: + +```shell +mkdir MyPyPiPackage && cd MyPyPiPackage +``` + +After creating this, create another directory inside: + +```shell +mkdir mypypipackage && cd mypypipackage +``` + +Create two new files inside this directory to set up the basic project: + +```shell +touch __init__.py +touch greet.py +``` + +Inside `greet.py`, add the following code: + +```python +def SayHello(): + print("Hello from MyPyPiPackage") + return +``` + +Inside the `__init__.py` file, add the following: + +```python +from .greet import SayHello +``` + +Now that the basics of our project is completed, we can test that the code runs. +Start the Python prompt inside your top `MyPyPiPackage` directory. Then run: + +```python +>>> from mypypipackage import SayHello +>>> SayHello() +``` + +You should see an output similar to the following: + +```plaintext +Python 3.8.2 (v3.8.2:7b3ab5921f, Feb 24 2020, 17:52:18) +[Clang 6.0 (clang-600.0.57)] on darwin +Type "help", "copyright", "credits" or "license" for more information. +>>> from mypypipackage import SayHello +>>> SayHello() +Hello from MyPyPiPackage +``` + +Once we've verified that the sample project is working as above, we can next +work on creating a package. + +### Create a package + +Inside your `MyPyPiPackage` directory, we need to create a `setup.py` file. Run +the following: + +```shell +touch setup.py +``` + +This file contains all the information about our package. For more information +about this file, see [creating setup.py](https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py). +For this guide, we don't need to extensively fill out this file, simply add the +below to your `setup.py`: + +```python +import setuptools + +setuptools.setup( + name="mypypipackage", + version="0.0.1", + author="Example Author", + author_email="author@example.com", + description="A small example package", + packages=setuptools.find_packages(), + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + ], + python_requires='>=3.6', +) +``` + +Save the file, then execute the setup like so: + +```shell +python3 setup.py sdist bdist_wheel +``` + +If successful, you should be able to see the output in a newly created `dist` +folder. Run: + +```shell +ls dist +``` + +And confirm your output matches the below: + +```plaintext +mypypipackage-0.0.1-py3-none-any.whl mypypipackage-0.0.1.tar.gz +``` + +Our package is now all set up and ready to be uploaded to the **GitLab PyPi +Package Registry**. Before we do so, we next need to set up authentication. ## Adding the GitLab PyPi Repository as a source +### Authenticating with a personal access token + You will need the following: - A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. @@ -39,12 +161,37 @@ You will need the following: Edit your `~/.pypirc` file and add the following: ```ini +[distutils] +index-servers = + gitlab + [gitlab] repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi username = __token__ password = <your personal access token> ``` +### Authenticating with a deploy token + +You will need the following: + +- A deploy token. You can generate a [deploy token](./../../project/deploy_tokens/index.md) with the `read_package_registry` and/or `write_package_registry` scopes for repository authentication. +- A suitable name for your source. +- Your project ID which can be found on the home page of your project. + +Edit your `~/.pypirc` file and add the following: + +```ini +[distutils] +index-servers = + gitlab + +[gitlab] +repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +username = <deploy token username> +password = <deploy token> +``` + ## Uploading packages When uploading packages, note that: @@ -52,13 +199,33 @@ When uploading packages, note that: - The maximum allowed size is 50 Megabytes. - If you upload the same package with the same version multiple times, each consecutive upload is saved as a separate file. When installing a package, GitLab will serve the most recent file. -- When uploading packages to GitLab, they will not be displayed in the packages UI of your project - immediately. It can take up to 10 minutes to process a package. ### Upload packages with Twine -This section assumes that your project is properly built and you already [created a PyPi package with setuptools](https://packaging.python.org/tutorials/packaging-projects/). -Upload your package using the following command: +If you were following the guide above, then the `MyPyPiPackage` package should +be ready to be uploaded. Run the following command: + +```shell +python3 -m twine upload --repository gitlab dist/* +``` + +If successful, you should see the following: + +```plaintext +Uploading distributions to https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi +Uploading mypypipackage-0.0.1-py3-none-any.whl +100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s] +Uploading mypypipackage-0.0.1.tar.gz +100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s] +``` + +This indicates that the package was uploaded successfully. You can then navigate +to your project's **Packages & Registries** page and see the uploaded packages. + +If you did not follow the guide above, the you'll need to ensure your package +has been properly built and you [created a PyPi package with setuptools](https://packaging.python.org/tutorials/packaging-projects/). + +You can then upload your package using the following command: ```shell python -m twine upload --repository <source_name> dist/<package_file> @@ -80,5 +247,22 @@ pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api Where: - `<package_name>` is the package name. -- `<personal_access_token>` is your personal access token. -- `<project_id>` is your project id number. +- `<personal_access_token>` is a personal access token with the `read_api` scope. +- `<project_id>` is the project ID. + +If you were following the guide above and want to test installing the +`MyPyPiPackage` package, you can run the following: + +```shell +pip install mypypipackage --no-deps --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +``` + +This should result in the following: + +```plaintext +Looking in indexes: https://__token__:****@gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/simple +Collecting mypypipackage + Downloading https://gitlab.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB) +Installing collected packages: mypypipackage +Successfully installed mypypipackage-0.0.1 +``` diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 5a631cc59e3..e5893b291dc 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -42,7 +42,7 @@ NOTE: **Note:** In GitLab 11.0, the Master role was renamed to Maintainer. While Maintainer is the highest project-level role, some actions can only be performed by a personal namespace or group owner, -or an instance admin, who receives all permissions. +or an instance admin, who receives all permissions. For more information, see [projects members documentation](project/members/index.md). The following table depicts the various user permission levels in a project. @@ -50,7 +50,6 @@ The following table depicts the various user permission levels in a project. |---------------------------------------------------|---------|------------|-------------|----------|--------| | Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| View Insights charts **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | | View approved/blacklisted licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View Security reports **(ULTIMATE)** | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | @@ -71,6 +70,7 @@ The following table depicts the various user permission levels in a project. | View confidential issues | (*2*) | ✓ | ✓ | ✓ | ✓ | | View [Releases](project/releases/index.md) | ✓ (*6*) | ✓ | ✓ | ✓ | ✓ | | View requirements **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Manage user-starred metrics dashboards (*7*) | ✓ | ✓ | ✓ | ✓ | ✓ | | Assign issues | | ✓ | ✓ | ✓ | ✓ | | Label issues | | ✓ | ✓ | ✓ | ✓ | | Set issue weight | | ✓ | ✓ | ✓ | ✓ | @@ -83,15 +83,13 @@ The following table depicts the various user permission levels in a project. | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | -| View project statistics | | ✓ | ✓ | ✓ | ✓ | +| View project statistics | | | ✓ | ✓ | ✓ | | View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | | Create new merge request | | ✓ | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create/edit requirements **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | -| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | -| Pull from [Conan repository](packages/conan_repository/index.md), [Maven repository](packages/maven_repository/index.md), or [NPM registry](packages/npm_registry/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Publish to [Conan repository](packages/conan_repository/index.md), [Maven repository](packages/maven_repository/index.md), or [NPM registry](packages/npm_registry/index.md) **(PREMIUM)** | | | ✓ | ✓ | ✓ | +| Pull [packages](packages/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) **(PREMIUM)**| | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | | Create/edit/delete [Releases](project/releases/index.md)| | | ✓ | ✓ | ✓ | | Create new branches | | | ✓ | ✓ | ✓ | @@ -142,20 +140,28 @@ The following table depicts the various user permission levels in a project. | Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | | Remove GitLab Pages | | | | ✓ | ✓ | | Manage clusters | | | | ✓ | ✓ | -| View Pods logs **(ULTIMATE)** | | | | ✓ | ✓ | +| View Pods logs | | | | ✓ | ✓ | | Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | | Edit comments (posted by any user) | | | | ✓ | ✓ | | Manage Error Tracking | | | | ✓ | ✓ | | Delete wiki pages | | | | ✓ | ✓ | | View project Audit Events | | | | ✓ | ✓ | | Manage [push rules](../push_rules/push_rules.md) | | | | ✓ | ✓ | +| Manage [project access tokens](./project/settings/project_access_tokens.md) **(CORE ONLY)** | | | | ✓ | ✓ | | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | +| Remove fork relationship | | | | | ✓ | | Remove project | | | | | ✓ | | Delete issues | | | | | ✓ | | Disable notification emails | | | | | ✓ | | Force push to protected branches (*4*) | | | | | | | Remove protected branches (*4*) | | | | | | +| View CI\CD analytics | | ✓ | ✓ | ✓ | ✓ | +| View Code Review analytics **(STARTER)** | | ✓ | ✓ | ✓ | ✓ | +| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Issues analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Repository analytics | | ✓ | ✓ | ✓ | ✓ | +| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | \* Owner permission is only available at the group or personal namespace level (and for instance admins) and is inherited by its projects. @@ -165,6 +171,7 @@ The following table depicts the various user permission levels in a project. 1. Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [Protected Branches](./project/protected_branches.md). 1. If the [branch is protected](./project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings), this depends on the access Developers and Maintainers are given. 1. Guest users can access GitLab [**Releases**](project/releases/index.md) for downloading assets but are not allowed to download the source code nor see repository information like tags and commits. +1. Actions are limited only to records owned (referenced) by user. ## Project features permissions @@ -228,6 +235,8 @@ group. | Create/edit group epic **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | +| Pull [packages](packages/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) **(PREMIUM)** | | | ✓ | ✓ | ✓ | | View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create project in group | | | ✓ (3) | ✓ (3) | ✓ (3) | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | @@ -244,6 +253,11 @@ group. | Delete group epic **(ULTIMATE)** | | | | | ✓ | | View group Audit Events | | | | | ✓ | | Disable notification emails | | | | | ✓ | +| View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Issues analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | +| View Value Stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | 1. Groups can be set to [allow either Owners or Owners and Maintainers to create subgroups](group/subgroups/index.md#creating-a-subgroup) @@ -391,7 +405,9 @@ instance and project. In addition, all admins can use the admin interface under | See events in the system | | | | ✓ | | Admin interface | | | | ✓ | -1. Only if the job was triggered by the user +1. Only if the job was: + - Triggered by the user + - [Since GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/35069), not run for a protected branch ### Job permissions diff --git a/doc/user/profile/img/change_password_v13_0.png b/doc/user/profile/img/change_password_v13_0.png Binary files differnew file mode 100644 index 00000000000..f63b32557ac --- /dev/null +++ b/doc/user/profile/img/change_password_v13_0.png diff --git a/doc/user/profile/img/unknown_sign_in_email_v13_0.png b/doc/user/profile/img/unknown_sign_in_email_v13_0.png Binary files differnew file mode 100644 index 00000000000..51a7c29cdfa --- /dev/null +++ b/doc/user/profile/img/unknown_sign_in_email_v13_0.png diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 66ee19437ae..383c7fe73aa 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -17,6 +17,11 @@ There are several ways to create users on GitLab. See the [creating users docume There are several ways to sign into your GitLab account. See the [authentication topic](../../topics/authentication/index.md) for more details. +### Unknown sign-in + +GitLab will notify you if a sign-in occurs that is from an unknown IP address. +See [Unknown Sign-In Notification](unknown_sign_in_notification.md) for more details. + ## User profile To access your profile: @@ -44,6 +49,7 @@ To access your profile settings: From there, you can: - Update your personal information +- Change your [password](#changing-your-password) - Set a [custom status](#current-status) for your profile - Manage your [commit email](#commit-email) for your profile - Manage [2FA](account/two_factor_authentication.md) @@ -60,6 +66,18 @@ From there, you can: - [View your active sessions](active_sessions.md) and revoke any of them if necessary - Access your audit log, a security log of important events involving your account +## Changing your password + +1. Navigate to your [profile's](#profile-settings) **Settings > Password**. +1. Enter your current password in the 'Current password' field. +1. Enter your desired new password twice, once in the 'New password' field and + once in the 'Password confirmation' field. +1. Click the 'Save password' button. + +If you don't know your current password, select the 'I forgot my password' link. + + + ## Changing your username Your `username` is a unique [`namespace`](../group/index.md#namespaces) diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index 1d92f15552d..ae00f3ace57 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -83,6 +83,9 @@ Or: 1. Click the notification dropdown, marked with a bell icon. 1. Select the desired [notification level](#notification-levels). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demonstration of how to be notified when a new release is available, see [Notification for releases](https://www.youtube.com/watch?v=qyeNkGgqmH4). + #### Group notifications You can select a notification level and email address for each group. @@ -208,17 +211,17 @@ The following table lists all GitLab-specific email headers: | Header | Description | |------------------------------------|-------------------------------------------------------------------------| -| X-GitLab-Group-Id **(PREMIUM)** | The group's ID. Only present on notification emails for epics. | -| X-GitLab-Group-Path **(PREMIUM)** | The group's path. Only present on notification emails for epics. | -| X-GitLab-Project | The name of the project the notification belongs to. | -| X-GitLab-Project-Id | The project's ID. | -| X-GitLab-Project-Path | The project's path. | -| X-GitLab-(Resource)-ID | The ID of the resource the notification is for. The resource, for example, can be `Issue`, `MergeRequest`, `Commit`, or another such resource. | -| X-GitLab-Discussion-ID | The ID of the thread the comment belongs to, in notification emails for comments. | -| X-GitLab-Pipeline-Id | The ID of the pipeline the notification is for, in notification emails for pipelines. | -| X-GitLab-Reply-Key | A unique token to support reply by email. | -| X-GitLab-NotificationReason | The reason for the notification. This can be `mentioned`, `assigned`, or `own_activity`. | -| List-Id | The path of the project in an RFC 2919 mailing list identifier. This is useful for email organization with filters, for example. | +| `X-GitLab-Group-Id` **(PREMIUM)** | The group's ID. Only present on notification emails for epics. | +| `X-GitLab-Group-Path` **(PREMIUM)** | The group's path. Only present on notification emails for epics. | +| `X-GitLab-Project` | The name of the project the notification belongs to. | +| `X-GitLab-Project-Id` | The project's ID. | +| `X-GitLab-Project-Path` | The project's path. | +| `X-GitLab-(Resource)-ID` | The ID of the resource the notification is for. The resource, for example, can be `Issue`, `MergeRequest`, `Commit`, or another such resource. | +| `X-GitLab-Discussion-ID` | The ID of the thread the comment belongs to, in notification emails for comments. | +| `X-GitLab-Pipeline-Id` | The ID of the pipeline the notification is for, in notification emails for pipelines. | +| `X-GitLab-Reply-Key` | A unique token to support reply by email. | +| `X-GitLab-NotificationReason` | The reason for the notification. This can be `mentioned`, `assigned`, or `own_activity`. | +| `List-Id` | The path of the project in an RFC 2919 mailing list identifier. This is useful for email organization with filters, for example. | ### X-GitLab-NotificationReason diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 1223f7b801a..87c1fe4007a 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -4,15 +4,22 @@ type: concepts, howto # Personal access tokens -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3749) in GitLab 8.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/3749) in GitLab 8.8. +> - [Notifications about expiring tokens](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) added in GitLab 12.6. +> - [Token lifetime limits](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) added in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. -If you're unable to use [OAuth2](../../api/oauth2.md), you can use a personal access token to authenticate with the [GitLab API](../../api/README.md#personal-access-tokens). +If you're unable to use [OAuth2](../../api/oauth2.md), you can use a personal access token to authenticate with the [GitLab API](../../api/README.md#personalproject-access-tokens). You can also use personal access tokens with Git to authenticate over HTTP or SSH. Personal access tokens are required when [Two-Factor Authentication (2FA)](../account/two_factor_authentication.md) is enabled. In both cases, you can authenticate with a token in place of your password. Personal access tokens expire on the date you define, at midnight UTC. -For examples of how you can use a personal access token to authenticate with the API, see the following section from our [API Docs](../../api/README.md#personal-access-tokens). +- GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that will expire in under seven days. The owners of these tokens are notified by email. +- In GitLab Ultimate, administrators may [limit the lifetime of personal access tokens](../admin_area/settings/account_and_limit_settings.md#limiting-lifetime-of-personal-access-tokens-ultimate-only). + +For examples of how you can use a personal access token to authenticate with the API, see the following section from our [API Docs](../../api/README.md#personalproject-access-tokens). + +GitLab also offers [impersonation tokens](../../api/README.md#impersonation-tokens) which are created by administrators via the API. They're a great fit for automated authentication as a specific user. ## Creating a personal access token diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index cd195e6e7a1..55781b48a27 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -55,6 +55,11 @@ The default syntax theme is White, and you can choose among 5 different themes:  +[Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in 13.0, the theme +you choose also applies to the [Web IDE](../project/web_ide/index.md)'s code editor and [Snippets](../snippets.md). +The themes are available only in the Web IDE file editor, except for the [dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/209808), +which applies to the entire Web IDE screen. + ## Behavior The following settings allow you to customize the behavior of GitLab's layout diff --git a/doc/user/profile/unknown_sign_in_notification.md b/doc/user/profile/unknown_sign_in_notification.md new file mode 100644 index 00000000000..9400ead1922 --- /dev/null +++ b/doc/user/profile/unknown_sign_in_notification.md @@ -0,0 +1,16 @@ +# Email notification for unknown sign-ins + +When a user successfully signs in from a previously unknown IP address, +GitLab notifies the user by email. In this way, GitLab proactively alerts users of potentially +malicious or unauthorized sign-ins. + +There are two methods used to identify a known sign-in: + +- Last sign-in IP: The current sign-in IP address is checked against the last sign-in + IP address. +- Current active sessions: If the user has an existing active session from the + same IP address. See [Active Sessions](active_sessions.md). + +## Example email + + diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 7fa8ec6c5f3..712f8ea0adc 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Adding EKS clusters GitLab supports adding new and existing EKS clusters. diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index 1195421f8fb..4094828323a 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Adding GKE clusters GitLab supports adding new and existing GKE clusters. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index dce273ce602..fddc9873f17 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Adding and removing Kubernetes clusters GitLab offers integrated cluster creation for the following Kubernetes providers: diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 74a58b93442..1298a24fcac 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Kubernetes clusters > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/35954) in GitLab 10.1 for projects. @@ -30,10 +36,28 @@ Using the GitLab project Kubernetes integration, you can: - View [Logs](#logs). - Run serverless workloads on [Kubernetes with Knative](serverless/index.md). +### Supported cluster versions + +GitLab is committed to support at least two production-ready Kubernetes minor versions at any given time. We regularly review the versions we support, and provide a four-month deprecation period before we remove support of a specific version. The range of supported versions is based on the evaluation of: + +- Our own needs. +- The versions supported by major managed Kubernetes providers. +- The versions [supported by the Kubernetes community](https://kubernetes.io/docs/setup/release/version-skew-policy/#supported-versions). + +Currently, GitLab supports the following Kubernetes versions: + +- 1.15 +- 1.14 +- 1.13 (deprecated, support ends on November 22, 2020) +- 1.12 (deprecated, support ends on September 22, 2020) + +NOTE: **Note:** +Some GitLab features may support versions outside the range provided here. + ### Deploy Boards **(PREMIUM)** GitLab's Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../../ci/environments.md) running on Kubernetes, +status of each CI [environment](../../../ci/environments/index.md) running on Kubernetes, displaying the status of the pods in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes. @@ -78,8 +102,8 @@ Kubernetes clusters can be used without Auto DevOps. > Introduced in GitLab 8.15. -When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments.md#web-terminals) -support to your [environments](../../../ci/environments.md). This is based on the `exec` functionality found in +When enabled, the Kubernetes integration adds [web terminal](../../../ci/environments/index.md#web-terminals) +support to your [environments](../../../ci/environments/index.md). This is based on the `exec` functionality found in Docker and Kubernetes, so you get a new shell session within your existing containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any deployments, replica sets, and @@ -181,8 +205,8 @@ you can either: ### Setting the environment scope **(PREMIUM)** When adding more than one Kubernetes cluster to your project, you need to differentiate -them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments.md) similar to how the -[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables) work. +them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the +[environment-specific variables](../../../ci/variables/README.md#limit-the-environment-scopes-of-environment-variables) work. The default environment scope is `*`, which means all jobs, regardless of their environment, will use that cluster. Each scope can only be used by a single @@ -262,7 +286,7 @@ A Kubernetes cluster can be the destination for a deployment job. If the cluster from your jobs using tools such as `kubectl` or `helm`. - You don't use GitLab's cluster integration you can still deploy to your cluster. However, you will need configure Kubernetes tools yourself - using [environment variables](../../../ci/variables/README.md#creating-a-custom-environment-variable) + using [environment variables](../../../ci/variables/README.md#custom-environment-variables) before you can interact with the cluster from your jobs. ### Deployment variables @@ -297,7 +321,7 @@ of the form `<project_name>-<project_id>-<environment>` (see [Deployment variables](#deployment-variables)). For **non**-GitLab-managed clusters, the namespace can be customized using -[`environment:kubernetes:namespace`](../../../ci/environments.md#configuring-kubernetes-deployments) +[`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) in `.gitlab-ci.yml`. NOTE: **Note:** When using a [GitLab-managed cluster](#gitlab-managed-clusters), the @@ -314,7 +338,7 @@ the deployment job: However, sometimes GitLab can not create them. In such instances, your job will fail with the message: -```text +```plaintext This job failed because the necessary resources were not successfully created. ``` @@ -325,7 +349,7 @@ Reasons for failure include: - The token you gave GitLab does not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) privileges required by GitLab. - Missing `KUBECONFIG` or `KUBE_TOKEN` variables. To be passed to your job, they must have a matching - [`environment:name`](../../../ci/environments.md#defining-environments). If your job has no + [`environment:name`](../../../ci/environments/index.md#defining-environments). If your job has no `environment:name` set, it will not be passed the Kubernetes credentials. NOTE: **NOTE:** diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 5543187b6de..8577231b69b 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -1,24 +1,36 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Kubernetes Logs > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4752) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25455) to [GitLab Core](https://about.gitlab.com/pricing/) 12.9. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26383) to [GitLab Core](https://about.gitlab.com/pricing/) 12.9. GitLab makes it easy to view the logs of running pods in [connected Kubernetes clusters](index.md). -By displaying the logs directly in GitLab, developers can avoid having to manage console tools or jump to a different interface. +By displaying the logs directly in GitLab in the **Log Explorer**, developers can avoid +managing console tools or jumping to a different interface. NOTE: **Kubernetes + GitLab** -Everything you need to build, test, deploy, and run your app at scale. +Everything you need to build, test, deploy, and run your application at scale. [Learn more](https://about.gitlab.com/solutions/kubernetes/). ## Overview -[Kubernetes](https://kubernetes.io) logs can be viewed directly within GitLab. +[Kubernetes](https://kubernetes.io) logs can be viewed directly within GitLab with +the **Log Explorer**.  +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +To learn more, see [APM - Log Explorer](https://www.youtube.com/watch?v=hWclZHA7Dgw). + ## Requirements -[Deploying to a Kubernetes environment](../deploy_boards.md#enabling-deploy-boards) is required in order to be able to use Logs. +[Deploying to a Kubernetes environment](../deploy_boards.md#enabling-deploy-boards) +is required to use Logs. ## Usage @@ -30,7 +42,8 @@ You can access them in two ways. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22011) in GitLab 12.5. -Go to **{cloud-gear}** **Operations > Logs** on the sidebar menu. +Go to **{cloud-gear}** **Operations > Pod logs** on the sidebar menu to display +the **Log Explorer**.  @@ -38,34 +51,42 @@ Go to **{cloud-gear}** **Operations > Logs** on the sidebar menu. Logs can be displayed by clicking on a specific pod from [Deploy Boards](../deploy_boards.md): -1. Go to **{cloud-gear}** **Operations > Environments** and find the environment which contains the desired pod, like `production`. -1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md). -1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status. +1. Go to **{cloud-gear}** **Operations > Environments** and find the environment + which contains the desired pod, like `production`. +1. On the **Environments** page, you should see the status of the environment's + pods with [Deploy Boards](../deploy_boards.md). +1. When mousing over the list of pods, a tooltip will appear with the exact pod name + and status.  -1. Click on the desired pod to bring up the logs view. +1. Click on the desired pod to display the **Log Explorer**. ### Logs view -The logs view lets you filter the logs by: +The **Log Explorer** lets you filter the logs by: - Pods. - [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/issues/5769), environments. -- [From GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21656), [full text search](#full-text-search). +- [From GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21656), + [full text search](#full-text-search). - [From GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/issues/197879), dates. -Loading more than 500 log lines is possible from [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/198050) onwards. +Loading more than 500 log lines is possible from +[GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/198050) onward. -Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab/issues/13404). +Support for pods with multiple containers is coming +[in a future release](https://gitlab.com/gitlab-org/gitlab/issues/13404). -Support for historical data is coming [in a future release](https://gitlab.com/gitlab-org/gitlab/issues/196191). +Support for historical data is coming +[in a future release](https://gitlab.com/gitlab-org/gitlab/issues/196191). ### Filter by date > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/197879) in GitLab 12.8. -When you enable [Elastic Stack](../../clusters/applications.md#elastic-stack) on your cluster, you can filter by date. +When you enable [Elastic Stack](../../clusters/applications.md#elastic-stack) +on your cluster, you can filter logs displayed in the **Log Explorer** by date. -Click on **Show last** to see the available options. +Click **Show last** in the **Log Explorer** to see the available options. ### Full text search @@ -74,7 +95,8 @@ Click on **Show last** to see the available options. When you enable [Elastic Stack](../../clusters/applications.md#elastic-stack) on your cluster, you can search the content of your logs through a search bar. -The search is passed on to Elasticsearch using the [simple_query_string](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html) +The search is passed on to Elasticsearch using the +[simple_query_string](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html) Elasticsearch function, which supports the following operators: | Operator | Description | diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 5575cd1d2d4..dfed43470bc 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -1,14 +1,19 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Runbooks Runbooks are a collection of documented procedures that explain how to carry out a particular process, be it starting, stopping, debugging, or troubleshooting a particular system. -Using [Jupyter Notebooks](https://jupyter.org/) and the [Rubix library](https://github.com/Nurtch/rubix), +Using [Jupyter Notebooks](https://jupyter.org/) and the +[Rubix library](https://github.com/Nurtch/rubix), users can get started writing their own executable runbooks. -## Overview - Historically, runbooks took the form of a decision tree or a detailed step-by-step guide depending on the condition or system. @@ -22,121 +27,128 @@ pre-written code blocks or database queries against a given environment. The JupyterHub app offered via GitLab’s Kubernetes integration now ships with Nurtch’s Rubix library, providing a simple way to create DevOps -runbooks. A sample runbook is provided, showcasing common operations. While Rubix makes it -simple to create common Kubernetes and AWS workflows, you can also create them manually without -Rubix. +runbooks. A sample runbook is provided, showcasing common operations. While +Rubix makes it simple to create common Kubernetes and AWS workflows, you can +also create them manually without Rubix. -**<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch this [video](https://www.youtube.com/watch?v=Q_OqHIIUPjE) -for an overview of how this is accomplished in GitLab!** +for an overview of how this is accomplished in GitLab! ## Requirements To create an executable runbook, you will need: -1. **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications. - The simplest way to get started is to add a cluster using one of [GitLab's integrations](../add_remove_clusters.md#create-new-cluster). -1. **Helm Tiller** - Helm is a package manager for Kubernetes and is required to install - all the other applications. It is installed in its own pod inside the cluster which - can run the Helm CLI in a safe environment. -1. **Ingress** - Ingress can provide load balancing, SSL termination, and name-based - virtual hosting. It acts as a web proxy for your applications. -1. **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user service for managing notebooks across - a team. Jupyter Notebooks provide a web-based interactive programming environment - used for data analysis, visualization, and machine learning. +- **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the + applications. The simplest way to get started is to add a cluster using one + of [GitLab's integrations](../add_remove_clusters.md#create-new-cluster). +- **Helm Tiller** - Helm is a package manager for Kubernetes and is required to + install all the other applications. It's installed in its own pod inside the + cluster which can run the Helm CLI in a safe environment. +- **Ingress** - Ingress can provide load balancing, SSL termination, and name-based + virtual hosting. It acts as a web proxy for your applications. +- **JupyterHub** - [JupyterHub](https://jupyterhub.readthedocs.io/) is a multi-user + service for managing notebooks across a team. Jupyter Notebooks provide a + web-based interactive programming environment used for data analysis, + visualization, and machine learning. ## Nurtch -Nurtch is the company behind the [Rubix library](https://github.com/Nurtch/rubix). Rubix is -an open-source Python library that makes it easy to perform common DevOps tasks inside Jupyter Notebooks. -Tasks such as plotting Cloudwatch metrics and rolling your ECS/Kubernetes app are simplified -down to a couple of lines of code. See the [Nurtch Documentation](http://docs.nurtch.com/en/latest/) -for more information. +Nurtch is the company behind the [Rubix library](https://github.com/Nurtch/rubix). +Rubix is an open-source Python library that makes it easy to perform common +DevOps tasks inside Jupyter Notebooks. Tasks such as plotting Cloudwatch metrics +and rolling your ECS/Kubernetes app are simplified down to a couple of lines of +code. See the [Nurtch Documentation](http://docs.nurtch.com/en/latest/) for more +information. ## Configure an executable runbook with GitLab Follow this step-by-step guide to configure an executable runbook in GitLab using -the components outlined above and the preloaded demo runbook. - -### 1. Add a Kubernetes cluster - -Follow the steps outlined in [Create new cluster](../add_remove_clusters.md#create-new-cluster) -to add a Kubernetes cluster to your project. - -### 2. Install Helm Tiller, Ingress, and JupyterHub - -Once the cluster has been provisioned in GKE, click the **Install** button next to the **Helm Tiller** app. - - +the components outlined above and the pre-loaded demo runbook. -Once Tiller has been installed successfully, click the **Install** button next to the **Ingress** app. +1. Add a Kubernetes cluster to your project by following the steps outlined in + [Create new cluster](../add_remove_clusters.md#create-new-cluster). +1. After the cluster has been provisioned in GKE, click the **Install** button + next to the **Helm Tiller** application to install Helm Tiller. - +  -Once Ingress has been installed successfully, click the **Install** button next to the **JupyterHub** app. +1. After Helm Tiller has been installed successfully, click the **Install** button next + to the **Ingress** application. - +  -### 3. Login to JupyterHub and start the server +1. After Ingress has been installed successfully, click the **Install** button next + to the **JupyterHub** application. You will need the **Jupyter Hostname** provided + here in the next step. -Once JupyterHub has been installed successfully, navigate to the displayed **Jupyter Hostname** URL and click -**Sign in with GitLab**. Authentication is automatically enabled for any user of the GitLab instance via OAuth2. This -will redirect to GitLab in order to authorize JupyterHub to use your GitLab account. Click **Authorize**. +  - +1. After JupyterHub has been installed successfully, open the **Jupyter Hostname** + in your browser. Click the **Sign in with GitLab** button to log in to + JupyterHub and start the server. Authentication is enabled for any user of the + GitLab instance with OAuth2. This button redirects you to a page at GitLab + requesting authorization for JupyterHub to use your GitLab account. -Once the application has been authorized you will taken back to the JupyterHub application. Click **Start My Server**. -The server will take a couple of seconds to start. +  -### 4. Configure access +1. Click **Authorize**, and you will be redirected to the JupyterHub application. +1. Click **Start My Server**, and the server will start in a few seconds. +1. To configure the runbook's access to your GitLab project, you must enter your + [GitLab Access Token](../../../profile/personal_access_tokens.md) + and your Project ID in the **Setup** section of the demo runbook: -In order for the runbook to access your GitLab project, you will need to enter a -[GitLab Access Token](../../../profile/personal_access_tokens.md) -as well as your Project ID in the **Setup** section of the demo runbook. + 1. Double-click the **DevOps-Runbook-Demo** folder located on the left panel. -Double-click the **DevOps-Runbook-Demo** folder located on the left panel. +  - + 1. Double-click the `Nurtch-DevOps-Demo.ipynb` runbook. -Double-click the "Nurtch-DevOps-Demo.ipynb" runbook. +  - + Jupyter displays the runbook's contents in the right-hand side of the screen. + The **Setup** section displays your `PRIVATE_TOKEN` and your `PROJECT_ID`. + Enter these values, maintaining the single quotes as follows: -The contents on the runbook will be displayed on the right side of the screen. Under the "Setup" section, you will find -entries for both your `PRIVATE_TOKEN` and your `PROJECT_ID`. Enter both these values, conserving the single quotes as follows: + ```sql + PRIVATE_TOKEN = 'n671WNGecHugsdEDPsyo' + PROJECT_ID = '1234567' + ``` -```sql -PRIVATE_TOKEN = 'n671WNGecHugsdEDPsyo' -PROJECT_ID = '1234567' -``` + 1. Update the `VARIABLE_NAME` on the last line of this section to match the name of + the variable you're using for your access token. In this example, our variable + name is `PRIVATE_TOKEN`. -Update the `VARIABLE_NAME` on the last line of this section to match the name of the variable you are using for your -access token. In this example our variable name is `PRIVATE_TOKEN`. + ```sql + VARIABLE_VALUE = project.variables.get('PRIVATE_TOKEN').value + ``` -```sql -VARIABLE_VALUE = project.variables.get('PRIVATE_TOKEN').value -``` +1. To configure the operation of a runbook, create and configure variables: -### 5. Configure an operation + NOTE: **Note:** + For this example, we are using the **Run SQL queries in Notebook** section in the + sample runbook to query a PostgreSQL database. The first four lines of the following + code block define the variables that are required for this query to function: -For this example we'll use the "**Run SQL queries in Notebook**" section in the sample runbook to query -a PostgreSQL database. The first 4 lines of the section define the variables that are required for this query to function. + ```sql + %env DB_USER={project.variables.get('DB_USER').value} + %env DB_PASSWORD={project.variables.get('DB_PASSWORD').value} + %env DB_ENDPOINT={project.variables.get('DB_ENDPOINT').value} + %env DB_NAME={project.variables.get('DB_NAME').value} + ``` -```sql -%env DB_USER={project.variables.get('DB_USER').value} -%env DB_PASSWORD={project.variables.get('DB_PASSWORD').value} -%env DB_ENDPOINT={project.variables.get('DB_ENDPOINT').value} -%env DB_NAME={project.variables.get('DB_NAME').value} -``` + 1. Navigate to **{settings}** **Settings >> CI/CD >> Variables** to create + the variables in your project. -Create the matching variables in your project's **Settings >> CI/CD >> Variables** +  - + 1. Click **Save variables**. -Back in Jupyter, click the "Run SQL queries in Notebook" heading and the click *Run*. The results will be -displayed in-line as follows: + 1. In Jupyter, click the **Run SQL queries in Notebook** heading, and then click + **Run**. The results are displayed inline as follows: - +  -You can try other operations such as running shell scripts or interacting with a Kubernetes cluster. Visit the +You can try other operations, such as running shell scripts or interacting with a +Kubernetes cluster. Visit the [Nurtch Documentation](http://docs.nurtch.com/) for more information. diff --git a/doc/user/project/clusters/serverless/aws.md b/doc/user/project/clusters/serverless/aws.md index 3df57e3a7a5..124a0d4bf9f 100644 --- a/doc/user/project/clusters/serverless/aws.md +++ b/doc/user/project/clusters/serverless/aws.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Deploying AWS Lambda function using GitLab CI/CD GitLab allows users to easily deploy AWS Lambda functions and create rich serverless applications. @@ -150,7 +156,7 @@ endpoints: Running the following `curl` command should trigger your function. NOTE: **Note:** - Your url should be the one retrieved from the GitLab deploy stage log. +Your URL should be the one retrieved from the GitLab deploy stage log. ```shell curl https://u768nzby1j.execute-api.us-east-1.amazonaws.com/production/hello diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 418e16aa0c1..2156d96f92a 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -1,3 +1,9 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Serverless > Introduced in GitLab 11.5. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index 49083be77dc..45d9e8f04e0 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -9,6 +9,33 @@ in [GitLab Starter](https://about.gitlab.com/pricing/) 11.3. > - [Support for group namespaces](https://gitlab.com/gitlab-org/gitlab-foss/issues/53182) added in GitLab Starter 12.1. > - Code Owners for Merge Request approvals was [introduced](https://gitlab.com/gitlab-org/gitlab/issues/4418) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.9. +## Introduction + +When contributing to a project, it can often be difficult +to find out who should review or approve merge requests. +Additionally, if you have a question over a specific file or +code block, it may be difficult to know who to find the answer from. + +GitLab Code Owners is a feature to define who owns specific +files or paths in a repository, allowing other users to understand +who is responsible for each file or path. + +## Why is this useful? + +Code Owners allows for a version controlled single source of +truth file outlining the exact GitLab users or groups that +own certain files or paths in a repository. Code Owners can be +utilized in the merge request approval process which can streamline +the process of finding the right reviewers and approvers for a given +merge request. + +In larger organizations or popular open source projects, Code Owners +can also be useful to understand who to contact if you have +a question that may not be related to code review or a merge request +approval. + +## How to set up Code Owners + You can use a `CODEOWNERS` file to specify users or [shared groups](members/share_project_with_groups.md) that are responsible for certain files in a repository. @@ -41,7 +68,7 @@ The user that would show for `README.md` would be `@user2`. ## Approvals by Code Owners Once you've set Code Owners to a project, you can configure it to -receive approvals: +be used for merge request approvals: - As [merge request eligible approvers](merge_requests/merge_request_approvals.md#code-owners-as-eligible-approvers). - As required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium). **(PREMIUM)** @@ -50,6 +77,9 @@ Once set, Code Owners are displayed in merge requests widgets:  +NOTE: **Note**: + While the`CODEOWNERS` file can be used in addition to Merge Request [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules) it can also be used as the sole driver of a Merge Request approval (without using [Approval Rules](merge_requests/merge_request_approvals.md#approval-rules)) by simply creating the file in one of the three locations specified above, configuring the Code Owners to be required approvers for [protected branches](protected_branches.md#protected-branches-approval-by-code-owners-premium) and then using [the syntax of Code Owners files](code_owners.md#the-syntax-of-code-owners-files) to specify the actual owners and granular permissions. + ## The syntax of Code Owners files Files can be specified using the same kind of patterns you would use @@ -69,23 +99,28 @@ escaped using `\#` to address files for which the name starts with a Example `CODEOWNERS` file: ```plaintext -# This is an example code owners file, lines starting with a `#` will -# be ignored. +# This is an example of a code owners file +# lines starting with a `#` will be ignored. # app/ @commented-rule # We can specify a default match using wildcards: * @default-codeowner +# We can also specify "multiple tab or space" separated codeowners: +* @multiple @code @owners + # Rules defined later in the file take precedence over the rules # defined before. # This will match all files for which the file name ends in `.rb` *.rb @ruby-owner -# Files with a `#` can still be accesssed by escaping the pound sign +# Files with a `#` can still be accessed by escaping the pound sign \#file_with_pound.rb @owner-file-with-pound # Multiple codeowners can be specified, separated by spaces or tabs +# In the following case the CODEOWNERS file from the root of the repo +# has 3 code owners (@multiple @code @owners) CODEOWNERS @multiple @code @owners # Both usernames or email addresses can be used to match diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index c479f610ff1..8f7bb844e37 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -3,7 +3,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/1589) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.0. GitLab's Deploy Boards offer a consolidated view of the current health and -status of each CI [environment](../../ci/environments.md) running on [Kubernetes](https://kubernetes.io), displaying the status +status of each CI [environment](../../ci/environments/index.md) running on [Kubernetes](https://kubernetes.io), displaying the status of the pods in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes. @@ -57,9 +57,9 @@ specific environment, there are a lot of use cases. To name a few: ## Enabling Deploy Boards -To display the Deploy Boards for a specific [environment](../../ci/environments.md) you should: +To display the Deploy Boards for a specific [environment](../../ci/environments/index.md) you should: -1. Have [defined an environment](../../ci/environments.md#defining-environments) with a deploy stage. +1. Have [defined an environment](../../ci/environments/index.md#defining-environments) with a deploy stage. 1. Have a Kubernetes cluster up and running. @@ -113,7 +113,7 @@ metadata: name: "APPLICATION_NAME" annotations: app.gitlab.com/app: ${CI_PROJECT_PATH_SLUG} - app.gitlab.com/env: ${CI_ENVIRONMENT_SLUG} + app.gitlab.com/env: ${CI_ENVIRONMENT_SLUG} spec: replicas: 1 selector: @@ -130,6 +130,11 @@ spec: The annotations will be applied to the deployments, replica sets, and pods. By changing the number of replicas, like `kubectl scale --replicas=3 deploy APPLICATION_NAME -n ${KUBE_NAMESPACE}`, you can follow the instances' pods from the board. +NOTE: **Note:** +The YAML file is static. If you apply it using `kubectl apply`, you must +manually provide the project and environment slugs, or create a script to +replace the variables in the YAML before applying. + ## Canary Deployments A popular CI strategy, where a small portion of the fleet is updated to the new @@ -141,5 +146,5 @@ version of your application. - [GitLab Autodeploy](../../topics/autodevops/stages.md#auto-deploy) - [GitLab CI/CD environment variables](../../ci/variables/README.md) -- [Environments and deployments](../../ci/environments.md) +- [Environments and deployments](../../ci/environments/index.md) - [Kubernetes deploy example](https://gitlab.com/gitlab-examples/kubernetes-deploy) diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index ebb12a6ed5d..2d42debed68 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -3,8 +3,10 @@ > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17894) in GitLab 10.7. > - [Moved](https://gitlab.com/gitlab-org/gitlab/issues/199370) from **Settings > Repository** in GitLab 12.9. > - [Added `write_registry` scope](https://gitlab.com/gitlab-org/gitlab/-/issues/22743) in GitLab 12.10. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29280) from **Settings > CI / CD** in GitLab 12.10.1. +> - [Added package registry scopes](https://gitlab.com/gitlab-org/gitlab/-/issues/213566) from **Settings > CI / CD** in GitLab 13.0. -Deploy tokens allow you to download (`git clone`) or push and pull the container registry images of a project without having a user and a password. +Deploy tokens allow you to download (`git clone`) or push and pull packages and container registry images of a project without having a user and a password. Deploy tokens can be managed by [maintainers only](../../permissions.md). @@ -16,7 +18,7 @@ You can create as many deploy tokens as you like from the settings of your proje 1. Log in to your GitLab account. 1. Go to the project (or group) you want to create Deploy Tokens for. -1. Go to **{settings}** **Settings** > **CI / CD**. +1. Go to **{settings}** **Settings** > **Repository**. 1. Click on "Expand" on **Deploy Tokens** section. 1. Choose a name, expiry date (optional), and username (optional) for the token. 1. Choose the [desired scopes](#limiting-scopes-of-a-deploy-token). @@ -65,7 +67,7 @@ To download a repository using a Deploy Token, you just need to: 1. `git clone` the project using the Deploy Token: ```shell - git clone http://<username>:<deploy_token>@gitlab.example.com/tanuki/awesome_project.git + git clone https://<username>:<deploy_token>@gitlab.example.com/tanuki/awesome_project.git ``` Replace `<username>` and `<deploy_token>` with the proper values. @@ -100,6 +102,22 @@ To push the container registry images, you'll need to: Just replace `<username>` and `<deploy_token>` with the proper values. Then you can simply push images to your Container Registry. +### Read or pull packages + +To pull packages in the GitLab package registry, you'll need to: + +1. Create a Deploy Token with `read_package_registry` as a scope. +1. Take note of your `username` and `token`. +1. For the [package type of your choice](./../../packages/index.md), follow the authentication instructions for deploy tokens. + +### Push or upload packages + +To upload packages in the GitLab package registry, you'll need to: + +1. Create a Deploy Token with `write_package_registry` as a scope. +1. Take note of your `username` and `token`. +1. For the [package type of your choice](./../../packages/index.md), follow the authentication instructions for deploy tokens. + ### Group Deploy Token > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/21765) in GitLab 12.9. @@ -107,6 +125,9 @@ push images to your Container Registry. A deploy token created at the group level can be used across all projects that belong either to the specific group or to one of its subgroups. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Group Deploy Tokens](https://youtu.be/8kxTJvaD9ks). + To use a group deploy token: 1. [Create](#creating-a-deploy-token) a deploy token for a group. @@ -132,3 +153,6 @@ those variables: ```shell docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY ``` + +NOTE: **Note:** +The special handling for the `gitlab-deploy-token` deploy token is not currently implemented for group deploy tokens. For the deploy token to be available for CI/CD jobs, it must be created at the project level. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214014) for details. diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index a02dc016f03..16ac53a2b52 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -39,6 +39,26 @@ templates of the default branch will be taken into account. Create a new Markdown (`.md`) file inside the `.gitlab/issue_templates/` directory in your repository. Commit and push to your default branch. +To create a Markdown file: + + 1. Click the `+` button next to `master` and click **New file**. + 1. Add the name of your issue template to the **File name** text field next to `master`. + Make sure words are separated with underscores and that your file has the `.md` extension, for + example `feature_request.md`. + 1. Commit and push to your default branch. + +If you don't have a `.gitlab/issue_templates` directory in your repository, you'll need to create it. + +To create the `.gitlab/issue_templates` directory: + + 1. Click the `+` button next to `master` and select **New directory**. + 1. Name this new directory `.gitlab` and commit to your default branch. + 1. Click the `+` button next to `master` again and select **New directory**.This time, n + 1. Name your directory `issue_templates` and commit to your default branch. + +To check if this has worked correctly, [create a new issue](./issues/managing_issues.md#create-a-new-issue) +and see if you can choose a description template. + ## Creating merge request templates Similarly to issue templates, create a new Markdown (`.md`) file inside the diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index d5f35051e9a..9069a231db4 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -2,9 +2,10 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in [GitLab Premium](https://about.gitlab.com/pricing/) 8.9. -File Locking helps you avoid merge conflicts and better manage your binary files. -Lock any file or directory, make your changes, and then unlock it so another -member of the team can edit it. +Working with multiple people on the same file can be a risk. Conflicts when merging a non-text file are hard to overcome and will require a lot of manual work to resolve. File Locking helps you avoid these merge conflicts and better manage your binary files. + +With File Locking, you can lock any file or directory, make your changes, and +then unlock it so another member of the team can edit it. ## Overview diff --git a/doc/user/project/git_attributes.md b/doc/user/project/git_attributes.md index 21ef94e61f7..260e618ba03 100644 --- a/doc/user/project/git_attributes.md +++ b/doc/user/project/git_attributes.md @@ -1,6 +1,6 @@ # Git Attributes -GitLab supports defining custom [Git attributes][gitattributes] such as what +GitLab supports defining custom [Git attributes](https://git-scm.com/docs/gitattributes) such as what files to treat as binary, and what language to use for syntax highlighting diffs. @@ -18,5 +18,3 @@ ignored. The `.gitattributes` file can be used to define which language to use when syntax highlighting files and diffs. See ["Syntax Highlighting"](highlighting.md) for more information. - -[gitattributes]: https://git-scm.com/docs/gitattributes diff --git a/doc/user/project/img/service_desk_custom_email_address_v13_0.png b/doc/user/project/img/service_desk_custom_email_address_v13_0.png Binary files differnew file mode 100644 index 00000000000..6ce8bf45085 --- /dev/null +++ b/doc/user/project/img/service_desk_custom_email_address_v13_0.png diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index 77fc2761e07..56717858b53 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -24,7 +24,7 @@ Import your projects from Bitbucket Cloud to GitLab with minimal effort. ## Requirements -The [Bitbucket Cloud integration][bb-import] must be first enabled in order to be +The [Bitbucket Cloud integration](../../../integration/bitbucket.md) must be first enabled in order to be able to import your projects from Bitbucket Cloud. Ask your GitLab administrator to enable this if not already. @@ -42,7 +42,7 @@ The importer will create any new namespaces (groups) if they don't exist or in the case the namespace is taken, the repository will be imported under the user's namespace that started the import process. -## Importing your Bitbucket repositories +## Import your Bitbucket repositories 1. Sign in to GitLab and go to your dashboard. 1. Click on **New project**. @@ -61,5 +61,11 @@ namespace that started the import process.  -[bb-import]: ../../../integration/bitbucket.md -[social sign-in]: ../../profile/account/social_sign_in.md +## Troubleshooting + +If you have more than one Bitbucket account, be sure to sign in to the correct account. +If you've accidentally started the import process with the wrong account, follow these steps: + +1. Revoke GitLab access to your Bitbucket account, essentially reversing the process in the following procedure: [Import your Bitbucket repositories](#import-your-bitbucket-repositories). + +1. Sign out of the Bitbucket account. Follow the procedure linked from the previous step. diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index 99179c3190e..55df2d7294d 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -73,3 +73,7 @@ namespace that started the import process. imported.  + +## Troubleshooting + +See the [troubleshooting](bitbucket.md#troubleshooting) section for [Bitbucket](bitbucket.md). diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index db8d33b58da..4c213f21920 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -8,7 +8,7 @@ your self-managed GitLab instance. NOTE: **Note:** These instructions work for users on GitLab.com, but if you are an administrator of a self-managed GitLab instance or if you are importing from GitHub Enterprise, -you must enable [GitHub integration][gh-import]. GitHub integration is the only method for +you must enable [GitHub integration](../../../integration/github.md). GitHub integration is the only method for importing from GitHub Enterprise. If you are using GitLab.com, you can alternatively import GitHub repositories using a [personal access token](#using-a-github-token), but this method is not recommended because it cannot associate all user activity @@ -81,7 +81,7 @@ the user account that is performing the import. NOTE: **Note:** If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured -[GitHub integration][gh-import]. +[GitHub integration](../../../integration/github.md). 1. From the top navigation bar, click **+** and select **New project**. 1. Select the **Import project** tab and then select **GitHub**. @@ -155,5 +155,3 @@ servers. For 4 servers with 8 cores this means you can import up to 32 objects ( Reducing the time spent in cloning a repository can be done by increasing network throughput, CPU capacity, and disk performance (e.g., by using high performance SSDs) of the disks that store the Git repositories (for your GitLab instance). Increasing the number of Sidekiq workers will *not* reduce the time spent cloning repositories. - -[gh-import]: ../../../integration/github.md "GitHub integration" diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index 49224001fe6..db48282a8f3 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -9,6 +9,15 @@ Jira issues import is an MVC, project-level feature, meaning that issues from mu Jira projects can be imported into a GitLab project. MVC version imports issue title and description as well as some other issue metadata as a section in the issue description. +## Future iterations + +As of GitLab 12.10, the Jira issue importer only brings across the title and description of +an issue. + +There is an [epic](https://gitlab.com/groups/gitlab-org/-/epics/2738) tracking the +addition of items such as issue assignees, labels, comments, user mapping, and much more. +These will be included in the future iterations of the GitLab Jira importer. + ## Prerequisites ### Permissions diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 585c45e35df..50272f0e007 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -242,7 +242,7 @@ field. For example: -```text +```plaintext machine example.gitlab.com login <gitlab_user_name> password <personal_access_token> diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 8c7e6edbf38..db8f24fc1e1 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -27,7 +27,7 @@ need to be configured in a Bamboo build plan before GitLab can integrate. 1. In the left pane, select a build stage. If you have multiple build stages you want to select the last stage that contains the Git checkout task. 1. Select the 'Miscellaneous' tab. -1. Under 'Pattern Match Labelling' put `${bamboo.repository.revision.number}` +1. Under 'Pattern Match Labeling' put `${bamboo.repository.revision.number}` in the 'Labels' box. 1. Save diff --git a/doc/user/project/integrations/generic_alerts.md b/doc/user/project/integrations/generic_alerts.md index 1a000fd1c44..2234727dd82 100644 --- a/doc/user/project/integrations/generic_alerts.md +++ b/doc/user/project/integrations/generic_alerts.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: Health +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Generic alerts integration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13203) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.4. @@ -26,12 +32,16 @@ You can customize the payload by sending the following parameters. All fields ar | Property | Type | Description | | -------- | ---- | ----------- | -| `title` | String | The title of the incident. If none is provided, then `New: Incident #N` will be used, where `#N` is the number of incident | +| `title` | String | The title of the incident. Required. | | `description` | String | A high-level summary of the problem. | | `start_time` | DateTime | The time of the incident. If none is provided, a timestamp of the issue will be used. | | `service` | String | The affected service. | | `monitoring_tool` | String | The name of the associated monitoring tool. | | `hosts` | String or Array | One or more hosts, as to where this incident occurred. | +| `severity` | String | The severity of the alert. Must be one of `critical`, `high`, `medium`, `low`, `info`, `unknown`. Default is `critical`. | + +TIP: **Payload size:** +Ensure your requests are smaller than the [payload application limits](../../../administration/instance_limits.md#generic-alert-json-payloads). Example request: diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 1ef2f593621..49b6a3f6450 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -1,13 +1,14 @@ # GitLab Slack application **(FREE ONLY)** +> - Introduced in GitLab 9.4. +> - Distributed to Slack App Directory in GitLab 10.2. + NOTE: **Note:** The GitLab Slack application is only configurable for GitLab.com. It will **not** work for on-premises installations where you can configure the [Slack slash commands](slack_slash_commands.md) service instead. We're planning to make this configurable for all GitLab installations, but there's no ETA - see [#28164](https://gitlab.com/gitlab-org/gitlab/issues/28164). -It was first introduced in GitLab 9.4 and distributed to Slack App Directory in -GitLab 10.2. Slack provides a native application which you can enable via your project's integrations on GitLab.com. @@ -35,12 +36,30 @@ docs on [Adding an app to your team](https://slack.com/help/articles/202035138). To enable GitLab's service for your Slack team: -1. Go to your project's **Settings > Integration > Slack application** (only - visible on GitLab.com) -1. Click the "Add to Slack" button +1. Go to your project's **{settings}** **Settings > Integration > Slack application** (only + visible on GitLab.com). +1. Click **Add to Slack**. That's all! You can now start using the Slack slash commands. +## Create a project alias for Slack + +To create a project alias on GitLab.com for Slack integration: + +1. Go to your project's home page. +1. Navigate to **{settings}** **Settings > Integrations** (only visible on GitLab.com) +1. On the **Integrations** page, click **Slack application**. +1. The current **Project Alias**, if any, is displayed. To edit this value, + click **Edit**. +1. Enter your desired alias, and click **Save changes**. + +Some Slack commands require a project alias, and fail with the following error +if the project alias is incorrect or missing from the command: + +```plaintext +GitLab error: project or alias not found +``` + ## Usage After confirming the installation, you, and everyone else in your Slack team, diff --git a/doc/user/project/integrations/img/metrics_dashboard_annotations_ui_v13.0.png b/doc/user/project/integrations/img/metrics_dashboard_annotations_ui_v13.0.png Binary files differnew file mode 100644 index 00000000000..a042fbbcf4e --- /dev/null +++ b/doc/user/project/integrations/img/metrics_dashboard_annotations_ui_v13.0.png diff --git a/doc/user/project/integrations/img/panel_context_menu_v12_10.png b/doc/user/project/integrations/img/panel_context_menu_v12_10.png Binary files differdeleted file mode 100644 index 096262fea91..00000000000 --- a/doc/user/project/integrations/img/panel_context_menu_v12_10.png +++ /dev/null diff --git a/doc/user/project/integrations/img/panel_context_menu_v13_0.png b/doc/user/project/integrations/img/panel_context_menu_v13_0.png Binary files differnew file mode 100644 index 00000000000..2d7cb923981 --- /dev/null +++ b/doc/user/project/integrations/img/panel_context_menu_v13_0.png diff --git a/doc/user/project/integrations/img/prometheus_dashboard_select_v_13_0.png b/doc/user/project/integrations/img/prometheus_dashboard_select_v_13_0.png Binary files differnew file mode 100644 index 00000000000..2f0309ce664 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_dashboard_select_v_13_0.png diff --git a/doc/user/project/integrations/img/toggle_metrics_user_starred_dashboard_v13_0.png b/doc/user/project/integrations/img/toggle_metrics_user_starred_dashboard_v13_0.png Binary files differnew file mode 100644 index 00000000000..59dc9ccfd30 --- /dev/null +++ b/doc/user/project/integrations/img/toggle_metrics_user_starred_dashboard_v13_0.png diff --git a/doc/user/project/integrations/img/webex_teams_configuration.png b/doc/user/project/integrations/img/webex_teams_configuration.png Binary files differnew file mode 100644 index 00000000000..66993e0887d --- /dev/null +++ b/doc/user/project/integrations/img/webex_teams_configuration.png diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index a23d8d7306d..6a202c9a130 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -17,21 +17,21 @@ If you have the Omnibus GitLab package installed, Mattermost is already bundled in it. All you have to do is configure it. Read more in the [Omnibus GitLab Mattermost documentation](https://docs.gitlab.com/omnibus/gitlab-mattermost/). -## Automated Configuration +## Automated configuration If Mattermost is installed on the same server as GitLab, the configuration process can be done for you by GitLab. Go to the Mattermost Slash Command service on your project and click the 'Add to Mattermost' button. -## Manual Configuration +## Manual configuration The configuration consists of two parts. First you need to enable the slash commands in Mattermost and then enable the service in GitLab. ### Step 1. Enable custom slash commands in Mattermost -This step is only required when using a source install, omnibus installs will be +This step is only required when using a source install, Omnibus installs will be preconfigured with the right settings. The first thing to do in Mattermost is to enable custom slash commands from @@ -145,6 +145,15 @@ trigger word followed by <kbd>help</kbd>. Example: `/gitlab help` The permissions to run the [available commands](#available-slash-commands) derive from the [permissions you have on the project](../../permissions.md#project-members-permissions). +## Troubleshooting + +If an event is not being triggered, confirm that the channel you're using is a public one, as +Mattermost webhooks do not have access to private channels. + +If a private channel is required, you can edit the webhook's channel in Mattermost and +select a private channel. It is not possible to use different channels for +different types of notifications - all events will be sent to the specified channel. + ## Further reading - [Mattermost slash commands documentation](https://docs.mattermost.com/developer/slash-commands.html) diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index b973abbe210..88668ab6c7d 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -47,7 +47,7 @@ Click on the service links to see further configuration instructions and details | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | No | | Packagist | Update your project on Packagist, the main Composer repository | Yes | | Pipelines emails | Email the pipeline status to a list of recipients | No | -| [Slack Notifications](slack.md) | Send GitLab events (e.g. issue created) to Slack as notifications | No | +| [Slack Notifications](slack.md) | Send GitLab events (for example, an issue was created) to Slack as notifications | No | | [Slack slash commands](slack_slash_commands.md) **(CORE ONLY)** | Use slash commands in Slack to control GitLab | No | | [GitLab Slack application](gitlab_slack_application.md) **(FREE ONLY)** | Use Slack's official application | No | | PivotalTracker | Project Management Software (Source Commits Endpoint) | No | @@ -55,6 +55,7 @@ Click on the service links to see further configuration instructions and details | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | No | | [Redmine](redmine.md) | Redmine issue tracker | No | | [Unify Circuit](unify_circuit.md) | Receive events notifications in Unify Circuit | No | +| [Webex Teams](webex_teams.md) | Receive events notifications in Webex Teams | No | | [YouTrack](youtrack.md) | YouTrack issue tracker | No | ## Push hooks limit @@ -93,6 +94,15 @@ From this page, you can repeat delivery with the same data by clicking **Resend  +### Uninitialized repositories + +Some integrations fail with an error `Test Failed. Save Anyway` when you attempt to set them up on +uninitialized repositories. This is because the default service test uses push data to build the +payload for the test request, and it fails, because there are no push events for the project. + +To resolve this error, initialize the repository by pushing a test file to the project and set up +the integration again. + ## Contributing to integrations Because GitLab is open source we can ship with the code and tests for all @@ -100,9 +110,6 @@ plugins. This allows the community to keep the plugins up to date so that they always work in newer GitLab versions. For an overview of what integrations are available, please see the -[project_services source directory][projects-code]. +[project_services source directory](https://gitlab.com/gitlab-org/gitlab/tree/master/app/models/project_services). Contributions are welcome! - -[projects-code]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master/app/models/project_services -[permissions]: ../../permissions.md diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index bbed14ea93f..6c40e5b9696 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Prometheus integration > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8935) in GitLab 9.0. @@ -66,6 +72,25 @@ It enables you to search as you type through all environments and select the one  +##### Select a dashboard + +The **dashboard** dropdown box above the dashboard displays the list of all dashboards available for the project. +It enables you to search as you type through all dashboards and select the one you're looking for. + + + +##### Mark a dashboard as favorite + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0. + +When viewing a dashboard, click the empty **Star dashboard** **{star-o}** button to mark a +dashboard as a favorite. Starred dashboards display a solid star **{star}** button, +and appear at the top of the dashboard select list. + +To remove dashboard from the favorites list, click the solid **Unstar Dashboard** **{star}** button. + + + #### About managed Prometheus deployments Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). @@ -93,7 +118,7 @@ Integration with Prometheus requires the following: #### Getting started -Installing and configuring Prometheus to monitor applications is fairly straight forward. +Installing and configuring Prometheus to monitor applications is fairly straightforward. 1. [Install Prometheus](https://prometheus.io/docs/prometheus/latest/installation/) 1. Set up one of the [supported monitoring targets](prometheus_library/index.md) @@ -123,14 +148,32 @@ to integrate with. 1. Provide the domain name or IP address of your server, for example `http://thanos.example.com/` or `http://192.0.2.1/`. 1. Click **Save changes**. +### Precedence with multiple Prometheus configurations + +Although you can enable both a [manual configuration](#manual-configuration-of-prometheus) +and [auto configuration](#managed-prometheus-on-kubernetes) of Prometheus, only +one of them will be used: + +- If you have enabled a + [Prometheus manual configuration](#manual-configuration-of-prometheus) + and a [managed Prometheus on Kubernetes](#managed-prometheus-on-kubernetes), + the manual configuration takes precedence and is used to run queries from + [dashboards](#defining-custom-dashboards-per-project) and [custom metrics](#adding-custom-metrics). +- If you have managed Prometheus applications installed on Kubernetes clusters + at **different** levels (project, group, instance), the order of precedence is described in + [Cluster precedence](../../instance/clusters/index.md#cluster-precedence). +- If you have managed Prometheus applications installed on multiple Kubernetes + clusters at the **same** level, the Prometheus application of a cluster with a + matching [environment scope](../../../ci/environments/index.md#scoping-environments-with-specs) is used. + ## Monitoring CI/CD Environments Once configured, GitLab will attempt to retrieve performance metrics for any environment which has had a successful deployment. -GitLab will automatically scan the Prometheus server for metrics from known servers like Kubernetes and NGINX, and attempt to identify individual environment. The supported metrics and scan process is detailed in our [Prometheus Metrics Library documentation](prometheus_library/index.md). +GitLab will automatically scan the Prometheus server for metrics from known servers like Kubernetes and NGINX, and attempt to identify individual environments. The supported metrics and scan process is detailed in our [Prometheus Metrics Library documentation](prometheus_library/index.md). -You can view the performance dashboard for an environment by [clicking on the monitoring button](../../../ci/environments.md#monitoring-environments). +You can view the performance dashboard for an environment by [clicking on the monitoring button](../../../ci/environments/index.md#monitoring-environments). ### Adding custom metrics @@ -152,10 +195,12 @@ A few fields are required: - **Y-axis label**: Y axis title to display on the dashboard. - **Unit label**: Query units, for example `req / sec`. Shown next to the value. -Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**, and **Y-axis label** match between metrics. For example, a metric with **Name** `Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display on the same chart as a second metric with the same values. A **Legend label** is suggested if this feature used. +Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**, and **Y-axis label** match between metrics. For example, a metric with **Name** `Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display on the same chart as a second metric with the same values. A **Legend label** is suggested if this feature is used. #### Query Variables +##### Predefined variables + GitLab supports a limited set of [CI variables](../../../ci/variables/README.md) in the Prometheus query. This is particularly useful for identifying a specific environment, for example with `ci_environment_slug`. The supported variables are: - `ci_environment_slug` @@ -168,10 +213,34 @@ GitLab supports a limited set of [CI variables](../../../ci/variables/README.md) NOTE: **Note:** Variables for Prometheus queries must be lowercase. -There are 2 methods to specify a variable in a query or dashboard: +##### User-defined variables + +[Variables can be defined](#templating-templating-properties) in a custom dashboard YAML file. + +##### Using variables + +Variables can be specified using double curly braces, such as `"{{ci_environment_slug}}"` ([added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20793) in GitLab 12.7). + +Support for the `"%{ci_environment_slug}"` format was +[removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31581) in GitLab 13.0. +Queries that continue to use the old format will show no data. + +#### Query Variables from URL -1. Variables can be specified using the [Liquid template format](https://shopify.dev/docs/liquid/reference/basics), for example `{{ci_environment_slug}}` ([added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20793) in GitLab 12.6). -1. You can also enclose it in quotation marks with curly braces with a leading percent, for example `"%{ci_environment_slug}"`. This method is deprecated though and support will be [removed in the next major release](https://gitlab.com/gitlab-org/gitlab/issues/37990). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214500) in GitLab 13.0. + +GitLab supports setting custom variables through URL parameters. Surround the variable +name with double curly braces (`{{example}}`) to interpolate the variable in a query: + +```plaintext +avg(sum(container_memory_usage_bytes{container_name!="{{pod}}"}) by (job)) without (job) /1024/1024/1024' +``` + +The URL for this query would be: + +```plaintext +http://gitlab.com/<user>/<project>/-/environments/<environment_id>/metrics?dashboard=.gitlab%2Fdashboards%2Fcustom.yml&pod=POD +``` #### Editing additional metrics from the dashboard @@ -261,19 +330,29 @@ If you select another branch, this branch should be merged to your **default** b Dashboards have several components: -- Panel groups, which comprise of panels. +- Templating variables. +- Panel groups, which consist of panels. - Panels, which support one or more metrics. The following tables outline the details of expected properties. -**Dashboard properties:** +##### **Dashboard (top-level) properties** | Property | Type | Required | Description | | ------ | ------ | ------ | ------ | | `dashboard` | string | yes | Heading for the dashboard. Only one dashboard should be defined per file. | | `panel_groups` | array | yes | The panel groups which should be on the dashboard. | +| `templating` | Hash | no | Top level key under which templating related options can be added. | + +##### **Templating (`templating`) properties** + +| Property | Type | Required | Description | +| -------- | ---- | -------- | ----------- | +| `variables` | Hash | no | Variables can be defined here. | -**Panel group (`panel_groups`) properties:** +Read the documentation on [templating](#templating-variables-for-metrics-dashboards). + +##### **Panel group (`panel_groups`) properties** | Property | Type | Required | Description | | ------ | ------ | ------ | ------ | @@ -281,7 +360,7 @@ The following tables outline the details of expected properties. | `priority` | number | optional, defaults to order in file | Order to appear on the dashboard. Higher number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | | `panels` | array | required | The panels which should be in the panel group. | -**Panel (`panels`) properties:** +##### **Panel (`panels`) properties** | Property | Type | Required | Description | | ------ | ------ | ------ | ------- | @@ -293,19 +372,19 @@ The following tables outline the details of expected properties. | `weight` | number | no, defaults to order in file | Order to appear within the grouping. Lower number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | | `metrics` | array | yes | The metrics which should be displayed in the panel. Any number of metrics can be displayed when `type` is `area-chart` or `line-chart`, whereas only 3 can be displayed when `type` is `anomaly-chart`. | -**Axis (`panels[].y_axis`) properties:** +##### **Axis (`panels[].y_axis`) properties** -| Property | Type | Required | Description | -| ----------- | ------ | ------------------------- | -------------------------------------------------------------------- | -| `name` | string | no, but highly encouraged | Y-Axis label for the panel, it will replace `y_label` if set. | -| `format` | string | no, defaults to `number` | Unit format used. See the [full list of units](prometheus_units.md). | -| `precision` | number | no, defaults to `2` | Number of decimals to display in the number. | +| Property | Type | Required | Description | +| ----------- | ------ | ----------------------------- | -------------------------------------------------------------------- | +| `name` | string | no, but highly encouraged | Y-Axis label for the panel. Replaces `y_label` if set. | +| `format` | string | no, defaults to `engineering` | Unit format used. See the [full list of units](prometheus_units.md). | +| `precision` | number | no, defaults to `2` | Number of decimal places to display in the number. | | -**Metrics (`metrics`) properties:** +##### **Metrics (`metrics`) properties** | Property | Type | Required | Description | | ------ | ------ | ------ | ------ | -| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/60319)). | +| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). | | `unit` | string | yes | Defines the unit of the query's return data. | | `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. | | `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | @@ -610,21 +689,122 @@ Note the following properties:  +### Templating variables for metrics dashboards + +Templating variables can be used to make your metrics dashboard more versatile. + +#### Templating variable types + +`templating` is a top-level key in the +[dashboard YAML](#dashboard-top-level-properties). +Define your variables in the `variables` key, under `templating`. The value of +the `variables` key should be a hash, and each key under `variables` +defines a templating variable on the dashboard. + +A variable can be used in a Prometheus query in the same dashboard using the syntax +described [here](#using-variables). + +##### `text` variable type + +CAUTION: **Warning:** +This variable type is an _alpha_ feature, and is subject to change at any time +without prior notice! + +For each `text` variable defined in the dashboard YAML, there will be a free text +box on the dashboard UI, allowing you to enter a value for each variable. + +The `text` variable type supports a simple and a full syntax. + +###### Simple syntax + +This example creates a variable called `variable1`, with a default value +of `default value`: + +```yaml +templating: + variables: + variable1: 'default value' # `text` type variable with `default value` as its default. +``` + +###### Full syntax + +This example creates a variable called `variable1`, with a default value of `default`. +The label for the text box on the UI will be the value of the `label` key: + +```yaml +templating: + variables: + variable1: # The variable name that can be used in queries. + label: 'Variable 1' # (Optional) label that will appear in the UI for this text box. + type: text + options: + default_value: 'default' # (Optional) default value. +``` + +##### `custom` variable type + +CAUTION: **Warning:** +This variable type is an _alpha_ feature, and is subject to change at any time +without prior notice! + +Each `custom` variable defined in the dashboard YAML creates a dropdown +selector on the dashboard UI, allowing you to select a value for each variable. + +The `custom` variable type supports a simple and a full syntax. + +###### Simple syntax + +This example creates a variable called `variable1`, with a default value of `value1`. +The dashboard UI will display a dropdown with `value1`, `value2` and `value3` +as the choices. + +```yaml +templating: + variables: + variable1: ['value1', 'value2', 'value3'] +``` + +###### Full syntax + +This example creates a variable called `variable1`, with a default value of `var1_option_2`. +The label for the text box on the UI will be the value of the `label` key. +The dashboard UI will display a dropdown with `Option 1` and `Option 2` +as the choices. + +If you select `Option 1` from the dropdown, the variable will be replaced with `value option 1`. +Similarly, if you select `Option 2`, the variable will be replaced with `value_option_2`: + +```yaml +templating: + variables: + variable1: # The variable name that can be used in queries. + label: 'Variable 1' # (Optional) label that will appear in the UI for this dropdown. + type: custom + options: + values: + - value: 'value option 1' # The value that will replace the variable in queries. + text: 'Option 1' # (Optional) Text that will appear in the UI dropdown. + - value: 'value_option_2' + text: 'Option 2' + default: true # (Optional) This option should be the default value of this variable. +``` + ### View and edit the source file of a custom dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/34779) in GitLab 12.5. When viewing a custom dashboard of a project, you can view the original -`.yml` file by clicking on **Edit dashboard** button. +`.yml` file by clicking on the **Edit dashboard** button. ### Chart Context Menu From each of the panels in the dashboard, you can access the context menu by clicking the **{ellipsis_v}** **More actions** dropdown box above the upper right corner of the panel to take actions related to the chart's data. - + The options are: +- [Expand panel](#expand-panel) - [View logs](#view-logs-ultimate) - [Download CSV](#downloading-data-as-csv) - [Copy link to chart](#embedding-gitlab-managed-kubernetes-metrics) @@ -632,7 +812,8 @@ The options are: ### Dashboard Annotations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`). +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/215224) in GitLab 13.0. You can use **Metrics Dashboard Annotations** to mark any important events on every metrics dashboard by adding annotations to it. While viewing a dashboard, @@ -644,6 +825,18 @@ its description. You can create annotations by making requests to the [Metrics dashboard annotations API](../../../api/metrics_dashboard_annotations.md) + + +### Expand panel + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3100) in GitLab 13.0. + +To view a larger version of a visualization, expand the panel by clicking the +**{ellipsis_v}** **More actions** icon and selecting **Expand panel**. + +To return to the metrics dashboard, click the **Back** button in your +browser, or pressing the <kbd>Esc</kbd> key. + ### View Logs **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/122013) in GitLab 12.8. @@ -708,14 +901,14 @@ receivers: ... ``` -In order for GitLab to associate your alerts with an [environment](../../../ci/environments.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab. +In order for GitLab to associate your alerts with an [environment](../../../ci/environments/index.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab. ### Taking action on incidents **(ULTIMATE)** >- [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11. >- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue. -Alerts can be used to trigger actions, like open an issue automatically (enabled by default since `12.1`). To configure the actions: +Alerts can be used to trigger actions, like opening an issue automatically (enabled by default since `12.1`). To configure the actions: 1. Navigate to your project's **Settings > Operations > Incidents**. 1. Enable the option to create issues. @@ -734,7 +927,7 @@ Once enabled, an issue will be opened automatically when an alert is triggered w - Optional list of attached annotations extracted from `annotations/*` - Alert [GFM](../../markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown` -When GitLab receives a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicated that it was closed automatically by the GitLab Alert bot. +When GitLab receives a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicating that it was closed automatically by the GitLab Alert bot. To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field. @@ -812,6 +1005,8 @@ Metric charts may also be hidden:  +You can open the link directly into your browser for a [detailed view of the data](#expand-panel). + ### Embedding metrics in issue templates It is also possible to embed either the default dashboard metrics or individual metrics in issue templates. For charts to render side-by-side, links to the entire metrics dashboard or individual metrics should be separated by either a comma or a space. @@ -907,12 +1102,20 @@ Prerequisites for embedding from a Grafana instance: 1. In the upper-left corner of the page, select a specific value for each variable required for the queries in the chart.  1. In Grafana, click on a panel's title, then click **Share** to open the panel's sharing dialog to the **Link** tab. If you click the _dashboard's_ share panel instead, GitLab will attempt to embed the first supported panel on the dashboard (if available). -1. If your Prometheus queries use Grafana's custom template variables, ensure that "Template variables" option is toggled to **On**. Of Grafana global template variables, only `$__interval`, `$__from`, and `$__to` are currently supported. Toggle **On** the "Current time range" option to specify the time range of the chart. Otherwise, the default range will be the last 8 hours. +1. If your Prometheus queries use Grafana's custom template variables, ensure that the "Template variables" option is toggled to **On**. Of Grafana global template variables, only `$__interval`, `$__from`, and `$__to` are currently supported. Toggle **On** the "Current time range" option to specify the time range of the chart. Otherwise, the default range will be the last 8 hours.  1. Click **Copy** to copy the URL to the clipboard. 1. In GitLab, paste the URL into a Markdown field and save. The chart will take a few moments to render.  +## Metrics dashboard visibility + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201924) in GitLab 13.0. + +You can set the visibility of the metrics dashboard to **Only Project Members** +or **Everyone With Access**. When set to **Everyone with Access**, the metrics +dashboard is visible to authenticated and non-authenticated users. + ## Troubleshooting When troubleshooting issues with a managed Prometheus app, it is often useful to diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index 143130aebea..911493cdae9 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring AWS Resources > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12621) in GitLab 9.4 diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index fa3590af8cf..712805b75f2 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring HAProxy > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12621) in GitLab 9.4 diff --git a/doc/user/project/integrations/prometheus_library/index.md b/doc/user/project/integrations/prometheus_library/index.md index c2b3676b23f..6f2c2477eee 100644 --- a/doc/user/project/integrations/prometheus_library/index.md +++ b/doc/user/project/integrations/prometheus_library/index.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Prometheus Metrics library > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8935) in GitLab 9.0. diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index ca1555c793b..29efe08e53d 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring Kubernetes > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/8935) in GitLab 9.0. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index d5f078f3ddf..eda6f64ccac 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring NGINX > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12621) in GitLab 9.4 diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 62f8c08e298..b2bc217e8bf 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring NGINX Ingress Controller > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22133) in GitLab 11.7. diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md index af3b725deb6..6ba0a7610f6 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress_vts.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Monitoring NGINX Ingress Controller with VTS metrics > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/13438) in GitLab 9.5. diff --git a/doc/user/project/integrations/prometheus_units.md b/doc/user/project/integrations/prometheus_units.md index 9df9f52ceb1..691d20e5de2 100644 --- a/doc/user/project/integrations/prometheus_units.md +++ b/doc/user/project/integrations/prometheus_units.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Unit formats reference > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/201999) in GitLab 12.9. @@ -5,19 +11,78 @@ You can select units to format your charts by adding `format` to your [axis configuration](prometheus.md#dashboard-yaml-properties). +## Internationalization and localization + +Currently, your [internationalization and localization options](https://en.wikipedia.org/wiki/Internationalization_and_localization) for number formatting are dependent on the system you are using i.e. your OS or browser. + +## Engineering Notation + +For generic or default data, numbers are formatted according to the current locale in [engineering notation](https://en.wikipedia.org/wiki/Engineering_notation). + +While an [engineering notation exists for the web](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NumberFormat), GitLab uses a version based off the [scientific notation](https://en.wikipedia.org/wiki/Scientific_notation). GitLab formatting acts in accordance with SI prefixes. For example, using GitLab notation, `1500.00` becomes `1.5k` instead of `1.5E3`. Keep this distinction in mind when using the engineering notation for your metrics. + +Formats: `engineering` + +SI prefixes: + +| Name | Symbol | Value | +| ---------- | ------- | -------------------------- | +| `yotta` | Y | 1000000000000000000000000 | +| `zetta` | Z | 1000000000000000000000 | +| `exa` | E | 1000000000000000000 | +| `peta` | P | 1000000000000000 | +| `tera` | T | 1000000000000 | +| `giga` | G | 1000000000 | +| `mega` | M | 1000000 | +| `kilo` | k | 1000 | +| `milli` | m | 0.001 | +| `micro` | μ | 0.000001 | +| `nano` | n | 0.000000001 | +| `pico` | p | 0.000000000001 | +| `femto` | f | 0.000000000000001 | +| `atto` | a | 0.000000000000000001 | +| `zepto` | z | 0.000000000000000000001 | +| `yocto` | y | 0.000000000000000000000001 | + +**Examples:** + +| Data | Displayed | +| --------------------------------- | --------- | +| `0.000000000000000000000008` | 8y | +| `0.000000000000000000008` | 8z | +| `0.000000000000000008` | 8a | +| `0.000000000000008` | 8f | +| `0.000000000008` | 8p | +| `0.000000008` | 8n | +| `0.000008` | 8μ | +| `0.008` | 8m | +| `10` | 10 | +| `1080` | 1.08k | +| `18000` | 18k | +| `18888` | 18.9k | +| `188888` | 189k | +| `18888888` | 18.9M | +| `1888888888` | 1.89G | +| `1888888888888` | 1.89T | +| `1888888888888888` | 1.89P | +| `1888888888888888888` | 1.89E | +| `1888888888888888888888` | 1.89Z | +| `1888888888888888888888888` | 1.89Y | +| `1888888888888888888888888888` | 1.89e+27 | + ## Numbers -For generic data, numbers are formatted according to the current locale. +For number data, numbers are formatted according to the current locale. Formats: `number` **Examples:** -| Data | Displayed | -| --------- | --------- | -| `10` | 1 | -| `1000` | 1,000 | -| `1000000` | 1,000,000 | +| Data | Displayed | +| ---------- | --------- | +| `10` | 1 | +| `1000` | 1,000 | +| `1000000` | 1,000,000 | ## Percentage diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index ba2a8f55d84..419340c14ef 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -41,7 +41,7 @@ an error message and keep troubleshooting from there. You may see an entry similar to the following in your Sidekiq log: -```text +```plaintext 2019-01-10_13:22:08.42572 2019-01-10T13:22:08.425Z 6877 TID-abcdefg ProjectServiceWorker JID-3bade5fb3dd47a85db6d78c5 ERROR: {:class=>"ProjectServiceWorker", :service_class=>"SlackService", :message=>"SSL_connect returned=1 errno=0 state=error: certificate verify failed"} ``` diff --git a/doc/user/project/integrations/webex_teams.md b/doc/user/project/integrations/webex_teams.md new file mode 100644 index 00000000000..a6e688887b6 --- /dev/null +++ b/doc/user/project/integrations/webex_teams.md @@ -0,0 +1,24 @@ +# Webex Teams service + +You can configure GitLab to send notifications to a Webex Teams space. + +## Create a webhook for the space + +1. Go to the [Incoming Webooks app page](https://apphub.webex.com/teams/applications/incoming-webhooks-cisco-systems). +1. Click **Connect** and log in to Webex Teams, if required. +1. Enter a name for the webhook and select the space that will receive the notifications. +1. Click **ADD**. +1. Copy the **Webhook URL**. + +## Configure settings in GitLab + +Once you have a webhook URL for your Webex Teams space, you can configure GitLab to send notifications. + +1. Navigate to **Project > Settings > Integrations**. +1. Select the **Webex Teams** integration. +1. Ensure that the **Active** toggle is enabled. +1. Select the checkboxes corresponding to the GitLab events you want to receive in Webex Teams. +1. Paste the **Webhook** URL for the Webex Teams space. +1. Configure the remaining options and then click **Test settings and save changes**. + +The Webex Teams space will begin to receive all applicable GitLab events. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 6dd2fd3b61b..89e84dda0e8 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1294,7 +1294,7 @@ X-Gitlab-Event: Job Hook } ``` -Note that `commit.id` is the id of the pipeline, not the id of the commit. +Note that `commit.id` is the ID of the pipeline, not the ID of the commit. ## Image URL rewriting diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index a72eaaa9ff1..119a53f5c35 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -14,8 +14,8 @@ To enable YouTrack integration in a project: | Field | Description | |:----------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **Description** | Name for the issue tracker (to differentiate between instances, for example). | - | **Project url** | URL to the project in YouTrack which is being linked to this GitLab project. | - | **Issues url** | URL to the issue in YouTrack project that is linked to this GitLab project. Note that the **Issues url** requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | + | **Project URL** | URL to the project in YouTrack which is being linked to this GitLab project. | + | **Issues URL** | URL to the issue in YouTrack project that is linked to this GitLab project. Note that the **Issues URL** requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | 1. Click the **Save changes** button. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 5bc71337e44..9903a711987 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -9,220 +9,271 @@ organize, and visualize a workflow for a feature or product release. It can be used as a [Kanban](https://en.wikipedia.org/wiki/Kanban_(development)) or a [Scrum](https://en.wikipedia.org/wiki/Scrum_(software_development)) board. -It provides perfect pairing between issue tracking and project management, +It pairs issue tracking and project management, keeping everything in the same place, so that you don't need to jump between different platforms to organize your workflow. -With GitLab Issue Boards, you organize your issues in lists that correspond to +With issue boards, you organize your issues in lists that correspond to their assigned labels, visualizing issues designed as cards throughout those lists. -You define your process and GitLab organizes it for you. You add your labels +You define your process, and GitLab organizes it for you. You add your labels then create the corresponding list to pull in your existing issues. When you're ready, you can drag and drop your issue cards from one step to the next. - + -**<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a [video presentation](https://youtu.be/UWsJ8tkHAa8) of -Issue Boards** (version introduced in GitLab 8.11 - August 2016). +the Issue Board feature (introduced in GitLab 8.11 - August 2016). -### Advanced features of Issue Boards +### Advanced features of issue boards - Create multiple issue boards per project. - Create multiple issue boards per group. **(PREMIUM)** - Add lists for [assignees](#assignee-lists-premium) and [milestones](#milestone-lists-premium). **(PREMIUM)** -Check all the [advanced features of Issue Boards](#gitlab-enterprise-features-for-issue-boards) -below. +Check all the [GitLab Enterprise features for issue boards](#gitlab-enterprise-features-for-issue-boards). - + + +--- ## How it works -The Issue Board builds on GitLab's existing +The Issue Board feature builds on GitLab's existing [issue tracking functionality](issues/index.md#issues-list) and -leverages the power of [labels](labels.md) by utilizing them as lists of the scrum board. +[labels](labels.md) by using them as lists of the Scrum board. -With the Issue Board you can have a different view of your issues while +With issue boards you can have a different view of your issues while maintaining the same filtering and sorting abilities you see across the -issue tracker. An Issue Board is based on its project's label structure, therefore, it +issue tracker. An issue board is based on its project's label structure, so it applies the same descriptive labels to indicate placement on the board, keeping consistency throughout the entire development lifecycle. -An Issue Board shows you what issues your team is working on, who is assigned to each, +An issue board shows you what issues your team is working on, who is assigned to each, and where in the workflow those issues are. You create issues, host code, perform reviews, build, test, -and deploy from one single platform. Issue Boards help you to visualize -and manage the entire process _in_ GitLab. +and deploy from one single platform. Issue boards help you to visualize +and manage the entire process in GitLab. -With [Multiple Issue Boards](#use-cases-for-multiple-issue-boards), +With [multiple issue boards](#use-cases-for-multiple-issue-boards), you go even further, as you can not only keep yourself and your project -organized from a broader perspective with one Issue Board per project, +organized from a broader perspective with one issue board per project, but also allow your team members to organize their own workflow by creating -multiple Issue Boards within the same project. +multiple issue boards within the same project. ## Use cases -There are many ways to use GitLab Issue Boards tailored to your own preferred workflow. -Here are some common use cases for Issue Boards. +There are many ways to use GitLab issue boards tailored to your own preferred workflow. +Here are some common use cases for issue boards. -### Use cases for a single Issue Board +### Use cases for a single issue board -GitLab Workflow allows you to discuss proposals in issues, categorize them -with labels, and from there organize and prioritize them with Issue Boards. +With the GitLab Workflow you can discuss proposals in issues, categorize them +with labels, and from there, organize and prioritize them with issue boards. For example, let's consider this simplified development workflow: -1. You have a repository hosting your app's codebase - and your team actively contributing to code -1. Your **backend** team starts working a new - implementation, gathers feedback and approval, and pass it over to **frontend** -1. When frontend is complete, the new feature is deployed to **staging** to be tested -1. When successful, it is deployed to **production** +1. You have a repository that hosts your application's codebase, and your team actively contributes code. +1. Your **backend** team starts working on a new implementation, gathers feedback and approval, and + passes it over to the **frontend** team. +1. When frontend is complete, the new feature is deployed to a **staging** environment to be tested. +1. When successful, it's deployed to **production**. -If we have the labels "**backend**", "**frontend**", "**staging**", and -"**production**", and an Issue Board with a list for each, we can: +If you have the labels "**backend**", "**frontend**", "**staging**", and +"**production**", and an issue board with a list for each, you can: -- Visualize the entire flow of implementations since the - beginning of the development life cycle until deployed to production -- Prioritize the issues in a list by moving them vertically -- Move issues between lists to organize them according to the labels you've set -- Add multiple issues to lists in the board by selecting one or more existing issues +- Visualize the entire flow of implementations since the beginning of the development life cycle + until deployed to production. +- Prioritize the issues in a list by moving them vertically. +- Move issues between lists to organize them according to the labels you've set. +- Add multiple issues to lists in the board by selecting one or more existing issues.  -### Use cases for Multiple Issue Boards +--- + +### Use cases for multiple issue boards -With [Multiple Issue Boards](#multiple-issue-boards), +With [multiple issue boards](#multiple-issue-boards), each team can have their own board to organize their workflow individually. #### Scrum team -With Multiple Issue Boards, each team has one board. Now you can move issues through each +With multiple issue boards, each team has one board. Now you can move issues through each part of the process. For instance: **To Do**, **Doing**, and **Done**. #### Organization of topics -Create lists to order things by topic and quickly change them between topics or groups, -such as between **UX**, **Frontend**, and **Backend**. The changes will be reflected across boards, -as changing lists will update the label accordingly. +Create lists to order issues by topic and quickly change them between topics or groups, +such as between **UX**, **Frontend**, and **Backend**. The changes are reflected across boards, +as changing lists updates the labels on each issue accordingly. #### Advanced team handover -For example, suppose we have a UX team with an Issue Board that contains: +For example, suppose we have a UX team with an issue board that contains: - **To Do** - **Doing** - **Frontend** -When done with something, they move the card to **Frontend**. The Frontend team's board looks like: +When finished with something, they move the card to **Frontend**. The Frontend team's board looks like: - **Frontend** - **Doing** - **Done** -Cards finished by the UX team will automatically appear in the **Frontend** column when they're ready for them. +Cards finished by the UX team automatically appear in the **Frontend** column when they are ready +for them. NOTE: **Note:** For a broader use case, please see the blog post [GitLab Workflow, an Overview](https://about.gitlab.com/blog/2016/10/25/gitlab-workflow-an-overview/#gitlab-workflow-use-case-scenario). For a real use case example, you can read why -[Codepen decided to adopt Issue Boards](https://about.gitlab.com/blog/2017/01/27/codepen-welcome-to-gitlab/#project-management-everything-in-one-place) +[Codepen decided to adopt issue boards](https://about.gitlab.com/blog/2017/01/27/codepen-welcome-to-gitlab/#project-management-everything-in-one-place) to improve their workflow with multiple boards. #### Quick assignments -Create lists for each of your team members and quickly drag-and-drop issues onto each team member. +Create lists for each of your team members and quickly drag and drop issues onto each team member's +list. + +## Issue board terminology + +An **issue board** represents a unique view of your issues. It can have multiple lists with each +list consisting of issues represented by cards. + +A **list** is a column on the issue board that displays issues matching certain attributes. +In addition to the default "Open" and "Closed" lists, each additional list shows issues matching +your chosen label, assignee, or milestone. On the top of each list you can see the number of issues +that belong to it. Types of lists include: + +- **Open** (default): all open issues that do not belong to one of the other lists. + Always appears as the leftmost list. +- **Closed** (default): all closed issues. Always appears as the rightmost list. +- **Label list**: all open issues for a label. +- [**Assignee list**](#assignee-lists-premium): all open issues assigned to a user. +- [**Milestone list**](#milestone-lists-premium): all open issues for a milestone. -## Issue Board terminology +A **Card** is a box on a list, and it represents an issue. You can drag cards from one list to +another to change their label, assignee, or milestone. The information you can see on a +card includes: -- **Issue Board** - Each board represents a unique view for your issues. It can have multiple lists with each list consisting of issues represented by cards. -- **List** - A column on the issue board that displays issues matching certain attributes. In addition to the default lists of 'Open' and 'Closed' issue, each additional list will show issues matching your chosen label or assignee. On the top of that list you can see the number of issues that belong to it. - - **Label list**: a list based on a label. It shows all opened issues with that label. - - **Assignee list**: a list which includes all issues assigned to a user. - - **Open** (default): shows all open issues that do not belong to one of the other lists. Always appears as the leftmost list. - - **Closed** (default): shows all closed issues. Always appears as the rightmost list. -- **Card** - A box in the list that represents an individual issue. The information you can see on a card consists of the issue number, the issue title, the assignee, and the labels associated with the issue. You can drag cards from one list to another to change their label or assignee from that of the source list to that of the destination list. +- Issue title +- Associated labels +- Issue number +- Assignee ## Permissions -[Reporters and up](../permissions.md) can use all the functionality of the -Issue Board to create or delete lists, and drag issues from one list to another. +Users with the [Reporter and higher roles](../permissions.md) can use all the functionality of the +Issue Board feature to create or delete lists and drag issues from one list to another. -## GitLab Enterprise features for Issue Boards +## GitLab Enterprise features for issue boards -GitLab Issue Boards are available on GitLab Core and GitLab.com Free, but some -advanced functionalities are only present in higher tiers: GitLab.com Bronze, -Silver, or Gold, or GitLab self-managed Starter, Premium, and Ultimate, as described -in the following sections. +GitLab issue boards are available on GitLab Core and GitLab.com Free tiers, but some +advanced functionality is present in [higher tiers only](https://about.gitlab.com/pricing/). -For a collection of [features per tier](#summary-of-features-per-tier), check the summary below. +### Summary of features per tier + +Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), +as shown in the following table: -### Multiple Issue Boards +| Tier | Number of Project issue boards | Number of Group issue boards | Configurable issue boards | Assignee lists | +|------------------|--------------------------------|------------------------------|---------------------------|----------------| +| Core / Free | Multiple | 1 | No | No | +| Starter / Bronze | Multiple | 1 | Yes | No | +| Premium / Silver | Multiple | Multiple | Yes | Yes | +| Ultimate / Gold | Multiple | Multiple | Yes | Yes | -> - Multiple Issue Boards per project [moved](https://gitlab.com/gitlab-org/gitlab-foss/issues/53811) to [GitLab Core](https://about.gitlab.com/pricing/) in GitLab 12.1. -> - Multiple Issue Boards per group is available in [GitLab Premium Edition](https://about.gitlab.com/pricing/). +### Multiple issue boards -Multiple Issue Boards, as the name suggests, allow for more than one Issue Board -for a given project or group. This is great for large projects with more than one team -or in situations where a repository is used to host the code of multiple -products. +> - [Introduced](https://about.gitlab.com/releases/2016/10/22/gitlab-8-13-released/) in GitLab 8.13. +> - Multiple issue boards per project [moved](https://gitlab.com/gitlab-org/gitlab-foss/issues/53811) to [GitLab Core](https://about.gitlab.com/pricing/) in GitLab 12.1. +> - Multiple issue boards per group are available in [GitLab Premium](https://about.gitlab.com/pricing/). + +Multiple issue boards allow for more than one issue board for a given project or group. +This is great for large projects with more than one team or in situations where a repository is used +to host the code of multiple products. -Clicking on the current board name in the upper left corner will reveal a -menu from where you can create another Issue Board or delete the existing one. Using the search box at the top of the menu, you can filter the listed boards. -When you have 10 or more boards available, a "Recent" section is also shown in the menu. -These are shortcuts to your last 4 visited boards. +When you have ten or more boards available, a **Recent** section is also shown in the menu, with +shortcuts to your last four visited boards. - + When you're revisiting an issue board in a project or group with multiple boards, -GitLab will automatically load the last board you visited. +GitLab automatically loads the last board you visited. + +#### Create an issue board -### Configurable Issue Boards **(STARTER)** +To create a new issue board: -> Introduced in [GitLab Starter Edition 10.2](https://about.gitlab.com/releases/2017/11/22/gitlab-10-2-released/#issue-boards-configuration). +1. Click the dropdown with the current board name in the upper left corner of the Issue Boards page. +1. Click **Create new board**. +1. Enter the new board's name and select its scope: milestone, labels, assignee, or weight. -An Issue Board can be associated with a GitLab [Milestone](milestones/index.md#milestones), +#### Delete an issue board + +To delete the currently active issue board: + +1. Click the dropdown with the current board name in the upper left corner of the Issue Boards page. +1. Click **Delete board**. +1. Click **Delete** to confirm. + +### Configurable issue boards **(STARTER)** + +> [Introduced](https://about.gitlab.com/releases/2017/11/22/gitlab-10-2-released/#issue-boards-configuration) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.2. + +An issue board can be associated with a GitLab [Milestone](milestones/index.md#milestones), [Labels](labels.md), Assignee and Weight which will automatically filter the Board issues according to these fields. This allows you to create unique boards according to your team's need.  -You can define the scope of your board when creating it or by clicking on the "Edit board" button. -Once a milestone, assignee or weight is assigned to an Issue Board, you will no longer be able to +You can define the scope of your board when creating it or by clicking the "Edit board" button. +Once a milestone, assignee or weight is assigned to an issue board, you will no longer be able to filter through these in the search bar. In order to do that, you need to remove the desired scope -(for example, milestone, assignee, or weight) from the Issue Board. +(for example, milestone, assignee, or weight) from the issue board.  -If you don't have editing permission in a board, you're still able to see the configuration by clicking on "View scope". +If you don't have editing permission in a board, you're still able to see the configuration by +clicking **View scope**.  +--- + ### Focus mode -> - Introduced in [GitLab Starter 9.1](https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#issue-boards-focus-mode-ees-eep). -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212331) to GitLab Core in 12.10. +> - [Introduced]((https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#issue-boards-focus-mode-ees-eep)) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.1. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28597) to the Free tier of GitLab.com in 12.10. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212331) to GitLab Core in 13.0. -Click the button at the top right to toggle focus mode on and off. In focus mode, the navigation UI is hidden, allowing you to focus on issues in the board. +Click the button at the top right to toggle focus mode on and off. In focus mode, the navigation UI +is hidden, allowing you to focus on issues in the board.  -### Sum of Issue Weights **(STARTER)** +--- + +### Sum of issue weights **(STARTER)** The top of each list indicates the sum of issue weights for the issues that belong to that list. This is useful when using boards for capacity allocation, especially in combination with [assignee lists](#assignee-lists-premium). - + + +--- -### Group Issue Boards **(PREMIUM)** +### Group issue boards **(PREMIUM)** -> Introduced in [GitLab Premium 10.0](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards). +> [Introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0. Accessible at the group navigation level, a group issue board offers the same features as a project-level board, but it can display issues from all projects in that @@ -231,102 +282,114 @@ boards. When updating milestones and labels for an issue through the sidebar upd group-level objects are available. NOTE: **Note:** -Multiple group issue boards were originally introduced in [GitLab 10.0 Premium](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) and -one group issue board per group was made available in GitLab 10.6 Core. +Multiple group issue boards were originally [introduced](https://about.gitlab.com/releases/2017/09/22/gitlab-10-0-released/#group-issue-boards) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.0, and one group issue board per group was made available in GitLab Core 10.6.  +--- + ### Assignee lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5784) in GitLab 11.0 Premium. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5784) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. -Like a regular list that shows all issues that have the list label, you can add -an assignee list that shows all issues assigned to the given user. +Like in a regular list that shows all issues with a chosen label, you can add +an assignee list that shows all issues assigned to a user. You can have a board with both label lists and assignee lists. To add an assignee list: 1. Click **Add list**. 1. Select the **Assignee list** tab. -1. Search and click on the user you want to add as an assignee. +1. Search and click the user you want to add as an assignee. Now that the assignee list is added, you can assign or unassign issues to that user -by [dragging issues](#dragging-issues-between-lists) to and/or from an assignee list. +by [dragging issues](#drag-issues-between-lists) to and from an assignee list. To remove an assignee list, just as with a label list, click the trash icon.  +--- + ### Milestone lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6469) in GitLab 11.2 Premium. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6469) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.2. -As of 11.2, you're also able to create lists of a milestone. As the name states, -these are lists that filter issues by the assigned milestone, giving you more -freedom and visibility on the Issue Board. To do so: +You're also able to create lists of a milestone. These are lists that filter issues by the assigned +milestone, giving you more freedom and visibility on the issue board. To add a milestone list: 1. Click **Add list**. 1. Select the **Milestone** tab. -1. Search and click on the milestone. +1. Search and click the milestone. -Similar to the assignee lists, you're now able to [drag issues](#dragging-issues-between-lists) -to and/or from a milestone list to manipulate the milestone of the dragged issues. -As on another list types, click on the trash icon to remove it. +Similar to the assignee lists, you're now able to [drag issues](#drag-issues-between-lists) +to and from a milestone list to manipulate the milestone of the dragged issues. +As in other list types, click the trash icon to remove a list.  +--- + ## Work In Progress limits **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/11403) in GitLab 12.7 -You can set Work In Progress (WIP) limits per issues list. When a limit is set, the list's header shows the number of issues in the list and the soft limit of issues. For example, for a list with 4 issues, and a limit of 5, the header will show `4/5`. If you exceed the limit, the current number of issues is shown in red. For example, you have a list with 5 issues with a limit of 5. When you move another issue to that list, the list's header displays `6/5`, with the `6` shown in red. +You can set Work In Progress (WIP) limits per issues list. When a limit is set, the list's header +shows the number of issues in the list and the soft limit of issues. -To set a WIP limit for a list: +Examples: -1. Navigate to a Project or Group board for which you have membership and click on the Settings icon (gear) in a list's header. -1. Next to **Work In Progress Limit**, click **Edit** and enter the maximum number of issues. Press `Enter` to save. +- You have a list with four issues, and a limit of five, the header will show **4/5**. + If you exceed the limit, the current number of issues is shown in red. +- You have a list with five issues with a limit of five. When you move another issue to that list, + the list's header displays **6/5**, with the six shown in red. -### Summary of features per tier - -Different issue board features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), as shown in the following table: +To set a WIP limit for a list: -| Tier | Number of Project Issue Boards | Number of Group Issue Boards | Configurable Issue Boards | Assignee Lists | -|----------|--------------------------------|------------------------------|---------------------------|----------------| -| Core / Free | Multiple | 1 | No | No | -| Starter / Bronze | Multiple | 1 | Yes | No | -| Premium / Silver | Multiple | Multiple | Yes | Yes | -| Ultimate / Gold | Multiple | Multiple | Yes | Yes | +1. Navigate to a Project or Group board of which you're a member. +1. Click the Settings icon (**{settings}**) in a list's header. +1. Next to **Work In Progress Limit**, click **Edit**. +1. Enter the maximum number of issues. +1. Press <kbd>Enter</kbd> to save. ## Blocked issues > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/34723) in GitLab 12.8. -If an issue is blocked by another issue, an icon will display next to its title to differentiate it from unblocked issues. +If an issue is blocked by another issue, an icon appears next to its title to indicate its blocked +status.  -## Actions you can take on an Issue Board +--- + +## Actions you can take on an issue board + +- [Create a new list](#create-a-new-list). +- [Delete an existing list](#delete-a-list). +- [Add issues to a list](#add-issues-to-a-list). +- [Remove an issue from a list](#remove-an-issue-from-a-list). +- [Filter issues](#filter-issues) that appear across your issue board. +- [Create workflows](#create-workflows). +- [Drag issues between lists](#drag-issues-between-lists). +- [Mulit-select issue cards](#multi-select-issue-cards). +- [Re-order issues in lists](#issue-ordering-in-a-list). +- Drag and reorder the lists. +- Change issue labels (by dragging an issue between lists). +- Close an issue (by dragging it to the **Done** list). -- [Create a new list](#creating-a-new-list). -- [Delete an existing list](#deleting-a-list). -- Drag issues between lists. -- Re-order issues in lists. -- Drag and reorder the lists themselves. -- Change issue labels on-the-fly while dragging issues between lists. -- Close an issue if you drag it to the **Done** list. -- Create a new list from a non-existing label by [creating the label on-the-fly](#creating-a-new-list) - within the Issue Board. -- [Filter issues](#filtering-issues) that appear across your Issue Board. +If you're not able to do some of the things above, make sure you have the right +[permissions](#permissions). -If you are not able to perform one or more of the things above, make sure you -have the right [permissions](#permissions). +### First time using an issue board -### First time using the Issue Board +The first time you open an issue board, you are presented with +the default lists (**Open** and **Closed**) and a welcome message that gives +you two options. You can either: -The first time you navigate to your Issue Board, you will be presented with -a default list (**Done**) and a welcoming message that gives -you two options. You can either create a predefined set of labels and create -their corresponding lists to the Issue Board or opt-out and use your own lists. +- Create a predefined set of labels (by default: **To Do** and **Doing**) and create their + corresponding lists to the issue board. +- Opt-out and use your own lists. - + If you choose to use and create the predefined lists, they will appear as empty because the labels associated to them will not exist up until that moment, @@ -334,99 +397,74 @@ which means the system has no way of populating them automatically. That's of course if the predefined labels don't already exist. If any of them does exist, the list will be created and filled with the issues that have that label. -### Creating a new list +### Create a new list -Create a new list by clicking on the **Add list** button at the upper -right corner of the Issue Board. +Create a new list by clicking the **Add list** button in the upper right corner of the issue board. - + -Simply choose the label or user to create the list from. The new list will be inserted +Then, choose the label or user to create the list from. The new list will be inserted at the end of the lists, before **Done**. Moving and reordering lists is as easy as dragging them around. -To create a list for a label that doesn't yet exist, simply create the label by -choosing **Create new label**. The label will be created on-the-fly and it will -be immediately added to the dropdown. You can now choose it to create a list. +To create a list for a label that doesn't yet exist, create the label by +choosing **Create new label**. This creates the label immediately and adds it to the dropdown. +You can now choose it to create a list. -### Deleting a list +### Delete a list -To delete a list from the Issue Board use the small trash icon that is present +To delete a list from the issue board, use the small trash icon present in the list's heading. A confirmation dialog will appear for you to confirm. Deleting a list doesn't have any effect in issues and labels, it's just the list view that is removed. You can always add it back later if you need. -### Adding issues to a list +### Add issues to a list -You can add issues to a list by clicking the **Add issues** button that is -present in the upper right corner of the Issue Board. This will open up a modal +You can add issues to a list by clicking the **Add issues** button +present in the upper right corner of the issue board. This will open up a modal window where you can see all the issues that do not belong to any list. -Select one or more issues by clicking on the cards and then click **Add issues** +Select one or more issues by clicking the cards and then click **Add issues** to add them to the selected list. You can limit the issues you want to add to the list by filtering by author, assignee, milestone, and label.  -### Removing an issue from a list - -Removing an issue from a list can be done by clicking on the issue card and then -clicking the **Remove from board** button in the sidebar. Under the hood, the -respective label is removed, and as such it's also removed from the list and the -board itself. - - - -### Issue ordering in a list +--- -When visiting a board, issues appear ordered in any list. You are able to change -that order simply by dragging and dropping the issues. The changed order will be saved -to the system so that anybody who visits the same board later will see the reordering, -with some exceptions. +### Remove an issue from a list -The first time a given issue appears in any board (that is, the first time a user -loads a board containing that issue), it is ordered with -respect to other issues in that list according to [Priority order](labels.md#label-priority). +Removing an issue from a list can be done by clicking the issue card and then +clicking the **Remove from board** button in the sidebar. The +respective label is removed. -At that point, that issue is assigned a relative order value by the system -representing its relative order with respect to the other issues in the list. Any time -you drag-and-drop reorder that issue, its relative order value changes accordingly. - -Also, any time that issue appears in any board when it's loaded by a user, -the updated relative order value is used for the ordering. (It's only the first -time an issue appears that it takes from the Priority order mentioned above.) This means that -if issue `A` is drag-and-drop reordered to be above issue `B` by any user in -a given board inside your GitLab instance, any time those two issues are subsequently -loaded in any board in the same instance (could be a different project board or a different group -board, for example), that ordering is maintained. + -This ordering also affects [issue lists](issues/sorting_issue_lists.md). -Changing the order in an issue board changes the ordering in an issue list, -and vice versa. +--- -### Filtering issues +### Filter issues -You should be able to use the filters on top of your Issue Board to show only +You should be able to use the filters on top of your issue board to show only the results you want. This is similar to the filtering used in the issue tracker -since the metadata from the issues and labels are re-used in the Issue Board. +since the metadata from the issues and labels are re-used in the issue board. You can filter by author, assignee, milestone, and label. -### Creating workflows +### Create workflows -By reordering your lists, you can create workflows. As lists in Issue Boards are +By reordering your lists, you can create workflows. As lists in issue boards are based on labels, it works out of the box with your existing issues. So if you've already labeled things with 'Backend' and 'Frontend', the issue appears in the lists as you create them. In addition, this means you can easily move something between lists by changing a label. -A typical workflow of using the Issue Board would be: +A typical workflow of using an issue board would be: 1. You have [created](labels.md#label-management) and [prioritized](labels.md#label-priority) labels so that you can easily categorize your issues. 1. You have a bunch of issues (ideally labeled). -1. You visit the Issue Board and start [creating lists](#creating-a-new-list) to +1. You visit the issue board and start [creating lists](#create-a-new-list) to create a workflow. 1. You move issues around in lists so that your team knows who should be working on what issue. @@ -446,9 +484,11 @@ issue. This process can be seen clearly when visiting an issue since with every move to another list the label changes and a system not is recorded. - + -### Dragging issues between lists +--- + +### Drag issues between lists When dragging issues between lists, different behavior occurs depending on the source list and the target list. @@ -472,6 +512,35 @@ To select and move multiple cards:  +--- + +### Issue ordering in a list + +When visiting a board, issues appear ordered in any list. You're able to change +that order by dragging and dropping the issues. The changed order will be saved +to the system so that anybody who visits the same board later will see the reordering, +with some exceptions. + +The first time a given issue appears in any board (that is, the first time a user +loads a board containing that issue), it is ordered with +respect to other issues in that list according to [Priority order](labels.md#label-priority). + +At that point, that issue is assigned a relative order value by the system +representing its relative order with respect to the other issues in the list. Any time +you drag-and-drop reorder that issue, its relative order value changes accordingly. + +Also, any time that issue appears in any board when it's loaded by a user, +the updated relative order value is used for the ordering. (It's only the first +time an issue appears that it takes from the Priority order mentioned above.) This means that +if issue `A` is drag-and-drop reordered to be above issue `B` by any user in +a given board inside your GitLab instance, any time those two issues are subsequently +loaded in any board in the same instance (could be a different project board or a different group +board, for example), that ordering is maintained. + +This ordering also affects [issue lists](issues/sorting_issue_lists.md). +Changing the order in an issue board changes the ordering in an issue list, +and vice versa. + ## Tips A few things to remember: @@ -480,8 +549,8 @@ A few things to remember: and adds the label from the list it goes to. - An issue can exist in multiple lists if it has more than one label. - Lists are populated with issues automatically if the issues are labeled. -- Clicking on the issue title inside a card takes you to that issue. -- Clicking on a label inside a card quickly filters the entire Issue Board +- Clicking the issue title inside a card takes you to that issue. +- Clicking a label inside a card quickly filters the entire issue board and show only the issues from all lists that have that label. - For performance and visibility reasons, each list shows the first 20 issues by default. If you have more than 20 issues, start scrolling down and the next diff --git a/doc/user/project/issues/csv_export.md b/doc/user/project/issues/csv_export.md index ec53b3dbbba..af01534a67f 100644 --- a/doc/user/project/issues/csv_export.md +++ b/doc/user/project/issues/csv_export.md @@ -70,7 +70,7 @@ Data will be encoded with a comma as the column delimiter, with `"` used to quot | Labels | Title of any labels joined with a `,` | | Time Estimate | [Time estimate](../time_tracking.md#estimates) in seconds | | Time Spent | [Time spent](../time_tracking.md#time-spent) in seconds | -| Epic ID | Id of the parent epic **(ULTIMATE)**, introduced in 12.7 | +| Epic ID | ID of the parent epic **(ULTIMATE)**, introduced in 12.7 | | Epic Title | Title of the parent epic **(ULTIMATE)**, introduced in 12.7 | ## Limitations diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index 1078d0410ed..64f56221632 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -1,6 +1,7 @@ -# Design Management **(PREMIUM)** +# Design Management -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212566) to GitLab Core in 13.0. CAUTION: **Warning:** This an **alpha** feature and is subject to change at any time without @@ -86,6 +87,7 @@ Copy-and-pasting has some limitations: - You can paste only one image at a time. When copy/pasting multiple files, only the first one will be uploaded. - All images will be converted to `png` format under the hood, so when you want to copy/paste `gif` file, it will result in broken animation. +- If you are pasting a screenshot from the clipboard, it will be renamed to `design_<timestamp>.png` - Copy/pasting designs is not supported on Internet Explorer. Designs with the same filename as an existing uploaded design will create a new version diff --git a/doc/user/project/issues/due_dates.md b/doc/user/project/issues/due_dates.md index f70597f6875..0be0cdd11bd 100644 --- a/doc/user/project/issues/due_dates.md +++ b/doc/user/project/issues/due_dates.md @@ -24,6 +24,11 @@ Changes are saved immediately.  +The last way to set a due date is by using [quick actions](../quick_actions.md), directly in an issue's description or comment: + +- `/due <date>`: set due date. Examples of valid `<date>` include `in 2 days`, `this Friday`, and `December 31st`. +- `/remove_due_date`: remove due date. + ## Making use of due dates Issues that have a due date can be easily seen in the issue tracker, diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 0f9295e1afd..22221531360 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -28,6 +28,11 @@ you can also view all the issues collectively at the group level. See also [Always start a discussion with an issue](https://about.gitlab.com/blog/2016/03/03/start-with-an-issue/). +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +To learn how GitLab's Strategic Marketing department uses GitLab issues with [labels](../labels.md) and +[issue boards](../issue_board.md), see the video on +[Managing Commitments with Issues](https://www.youtube.com/watch?v=cuIHNintg1o&t=3). + ## Parts of an issue Issues contain a variety of content and metadata, enabling a large range of flexibility diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 4ee29f3357d..4e329889e7c 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -50,7 +50,7 @@ create issues for the same project.  -### New issue via Service Desk **(PREMIUM)** +### New issue via Service Desk **(STARTER)** Enable [Service Desk](../service_desk.md) for your project and offer email support. By doing so, when your customer sends a new email, a new issue can be created in @@ -92,9 +92,17 @@ field values using query string parameters in a URL. This is useful for embeddin a URL in an external HTML page, and also certain scenarios where you want the user to create an issue with certain fields prefilled. -The title, description, and description template fields can be prefilled using -this method. You cannot pre-fill both the description and description template fields -in the same URL (since a description template also populates the description field). +The title, description, description template, and confidential fields can be prefilled +using this method. You cannot pre-fill both the description and description template +fields in the same URL (since a description template also populates the description +field). + +| Field | URL Parameter Name | Notes | +|----------------------|-----------------------|-------------------------------------------------------| +| title | `issue[title]` | | +| description | `issue[description]` | | +| description template | `issuable_template` | | +| confidential | `issue[confidential]` | Parameter value must be `true` to set to confidential | Follow these examples to form your new issue URL with prefilled fields. @@ -102,6 +110,8 @@ Follow these examples to form your new issue URL with prefilled fields. and a pre-filled description, the URL would be `https://gitlab.com/gitlab-org/gitlab-foss/issues/new?issue[title]=Validate%20new%20concept&issue[description]=Research%20idea` - For a new issue in the GitLab Community Edition project with a pre-filled title and a pre-filled description template, the URL would be `https://gitlab.com/gitlab-org/gitlab-foss/issues/new?issue[title]=Validate%20new%20concept&issuable_template=Research%20proposal` +- For a new issue in the GitLab Community Edition project with a pre-filled title, + a pre-filled description, and the confidential flag set, the URL would be `https://gitlab.com/gitlab-org/gitlab-foss/issues/new?issue[title]=Validate%20new%20concept&issue[description]=Research%20idea&issue[confidential]=true` ## Moving Issues @@ -117,7 +127,10 @@ The "Move issue" button is at the bottom of the right-sidebar when viewing the i If you have advanced technical skills you can also bulk move all the issues from one project to another in the rails console. The below script will move all the issues from one project to another that are not in status **closed**. -To access rails console run `sudo gitlab-rails console` on the GitLab server and run the below script. Please be sure to change **project**, **admin_user** and **target_project** to your values. We do also recommend [creating a backup](../../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) before attempting any changes in the console. +To access rails console run `sudo gitlab-rails console` on the GitLab server and run the below +script. Please be sure to change **project**, **admin_user** and **target_project** to your values. +We do also recommend [creating a backup](../../../raketasks/backup_restore.md#back-up-gitlab) before +attempting any changes in the console. ```ruby project = Project.find_by_full_path('full path of the project where issues are moved from') @@ -170,7 +183,7 @@ but it will not close automatically. If the issue is in a different repository than the MR, add the full URL for the issue(s): -```md +```markdown Closes #4, #6, and https://gitlab.com/<username>/<projectname>/issues/<xxx> ``` @@ -179,7 +192,7 @@ Closes #4, #6, and https://gitlab.com/<username>/<projectname>/issues/<xxx> When not specified, the default issue closing pattern as shown below will be used: ```shell -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing)|[Ii]mplement(?:s|ed|ing)?)(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) +\b((?:[Cc]los(?:e[sd]?|ing)|\b[Ff]ix(?:e[sd]|ing)?|\b[Rr]esolv(?:e[sd]?|ing)|\b[Ii]mplement(?:s|ed|ing)?)(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?: *,? +and +| *,? *)?)|([A-Z][A-Z0-9_]+-\d+))+) ``` This translates to the following keywords: diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index 5fba73c2971..8002b528a80 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -10,12 +10,14 @@ The relationship only shows up in the UI if the user can see both issues. ## Adding a related issue +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2035) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/34239) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.0. +> When you try to close an issue with open blockers, you'll see a warning that you can dismiss. + You can relate one issue to another by clicking the related issues "+" button in the header of the related issue block. Then, input the issue reference number or paste in the full URL of the issue. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/2035) in GitLab 12.8. - Additionally, you can select whether the current issue relates to, blocks, or is blocked by the issues being entered.  diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 9cf50d3aaed..f3b59147d5b 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -2,13 +2,18 @@ ## Overview -Labels allow you to categorize epics, issues, and merge requests using descriptive titles like -`bug`, `feature request`, or `docs`, as well as customizable colors. They allow you to quickly -and dynamically filter and manage epics, issues, and merge requests, and are a key -part of [issue boards](issue_board.md). +As your count of issues, merge requests, and epics grows in GitLab, it's more and more challenging +to keep track of those items. Especially as your organization grows from just a few people to +hundreds or thousands. This is where labels come in. They help you organize and tag your work +so you can track and find the work items you're interested in. -You can use labels to help [search](../search/index.md#issues-and-merge-requests) in -lists of issues, merge requests, and epics, as well as [search in issue boards](../search/index.md#issue-boards). +Labels are a key part of [issue boards](issue_board.md). With labels you can: + +- Categorize epics, issues, and merge requests using colors and descriptive titles like +`bug`, `feature request`, or `docs`. +- Dynamically filter and manage epics, issues, and merge requests. +- [Search lists of issues, merge requests, and epics](../search/index.md#issues-and-merge-requests), + as well as [issue boards](../search/index.md#issue-boards). ## Project labels and group labels @@ -164,7 +169,7 @@ Suppose you have the labels `workflow::development`, `workflow::review`, and applied, and a developer wanted to advance the issue to `workflow::review`, they would simply apply that label, and the `workflow::development` label would automatically be removed. This behavior already exists when you move issues -across label lists in an [issue board](issue_board.md#creating-workflows), but +across label lists in an [issue board](issue_board.md#create-workflows), but now, team members who may not be working in an issue board directly would still be able to advance workflow states consistently in issues themselves. diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 27a5701e6c2..9020dc335b6 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -1,4 +1,4 @@ -# Project's members +# Members of a project You can manage the groups and users and their access levels in all of your projects. You can also personalize the access level you give each user, diff --git a/doc/user/project/merge_requests/accessibility_testing.md b/doc/user/project/merge_requests/accessibility_testing.md index 755bf0447e3..427761281f6 100644 --- a/doc/user/project/merge_requests/accessibility_testing.md +++ b/doc/user/project/merge_requests/accessibility_testing.md @@ -18,6 +18,23 @@ measuring the accessibility of web sites, and has built a simple This job outputs accessibility violations, warnings, and notices for each page analyzed to a file called `accessibility`. +## Accessibility Merge Request widget + +[Since GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/39425), in addition to the report artifact that is created, GitLab will also show the +Accessibility Report in the merge request widget area: + + + +This widget comes with the `:accessibility_report_view` feature flag disabled by default while we test feature stability. +Once we have determined the widget is stable, this feature will be enabled by default. + +To enable this feature, ask a GitLab administrator with [Rails console access](../../../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags) to run the +following command: + +```ruby +Feature.enable(:accessibility_report_view) +``` + ## Configure Accessibility Testing This example shows how to run [pa11y](https://pa11y.org/) @@ -46,10 +63,13 @@ Pa11y against the webpages defined in `a11y_urls`, and builds an HTML report for The report for each URL is saved as an artifact that can be [viewed directly in your browser](../../../ci/pipelines/job_artifacts.md#browsing-artifacts). -A single `accessibility.json` artifact is created and saved along with the individual HTML reports. +A single `gl-accessibility.json` artifact is created and saved along with the individual HTML reports. It includes report data for all URLs scanned. NOTE: **Note:** +For GitLab 12.10 and earlier, the [artifact generated is named `accessibility.json`](https://gitlab.com/gitlab-org/ci-cd/accessibility/-/merge_requests/9). + +NOTE: **Note:** For GitLab versions earlier than 12.9, you can use `include:remote` and use a link to the [current template in `master`](https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml) diff --git a/doc/user/project/merge_requests/browser_performance_testing.md b/doc/user/project/merge_requests/browser_performance_testing.md index 1bca5d2a212..0fa3d7be265 100644 --- a/doc/user/project/merge_requests/browser_performance_testing.md +++ b/doc/user/project/merge_requests/browser_performance_testing.md @@ -6,55 +6,50 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3. -If your application offers a web interface and you are using +If your application offers a web interface and you're using [GitLab CI/CD](../../../ci/README.md), you can quickly determine the performance impact of pending code changes. ## Overview GitLab uses [Sitespeed.io](https://www.sitespeed.io), a free and open source -tool for measuring the performance of web sites, and has built a simple -[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) -which outputs the results in a file called `performance.json`. This plugin -outputs the performance score for each page that is analyzed. - +tool, for measuring the performance of web sites. GitLab has built a simple +[Sitespeed plugin](https://gitlab.com/gitlab-org/gl-performance) which outputs +the performance score for each page analyzed in a file called `performance.json`. The [Sitespeed.io performance score](https://examples.sitespeed.io/6.0/2017-11-23-23-43-35/help.html) -is a composite value based on best practices, and we will be expanding support -for [additional metrics](https://gitlab.com/gitlab-org/gitlab/issues/4370) -in a future release. +is a composite value based on best practices. -Going a step further, GitLab can show the Performance report right -in the merge request widget area (see below). +GitLab can [show the Performance report](#how-browser-performance-testing-works) +in the merge request widget area. ## Use cases -For instance, consider the following workflow: +Consider the following workflow: 1. A member of the marketing team is attempting to track engagement by adding a new tool. 1. With browser performance metrics, they see how their changes are impacting the usability of the page for end users. -1. The metrics show that after their changes the performance score of the page has gone down. -1. When looking at the detailed report, they see that the new JavaScript library was - included in `<head>` which affects loading page speed. -1. They ask a front end developer to help them, who sets the library to load asynchronously. -1. The frontend developer approves the merge request and authorizes its deployment to production. - -## How it works +1. The metrics show that after their changes, the performance score of the page has gone down. +1. When looking at the detailed report, they see the new JavaScript library was + included in `<head>`, which affects loading page speed. +1. They ask for help from a front end developer, who sets the library to load asynchronously. +1. The frontend developer approves the merge request, and authorizes its deployment to production. -First of all, you need to define a job in your `.gitlab-ci.yml` file that generates the -[Performance report artifact](../../../ci/yaml/README.md#artifactsreportsperformance-premium). -For more information on how the Performance job should look like, check the -example on [Configuring Browser Performance Testing](#configuring-browser-performance-testing). +## How browser performance testing works +First, define a job in your `.gitlab-ci.yml` file that generates the +[Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium). GitLab then checks this report, compares key performance metrics for each page -between the source and target branches, and shows the information right on the merge request. +between the source and target branches, and shows the information in the merge request. + +For an example Performance job, see +[Configuring Browser Performance Testing](#configuring-browser-performance-testing). NOTE: **Note:** -If the Performance report doesn't have anything to compare to, no information -will be displayed in the merge request area. That is the case when you add the -Performance job in your `.gitlab-ci.yml` for the very first time. -Consecutive merge requests will have something to compare to, and the Performance -report will be shown properly. +If the Performance report has no data to compare, such as when you add the +Performance job in your `.gitlab-ci.yml` for the very first time, no information +displays in the merge request widget area. Consecutive merge requests will have data for +comparison, and the Performance report will be shown properly.  @@ -64,52 +59,51 @@ This example shows how to run the [sitespeed.io container](https://hub.docker.co on your code by using GitLab CI/CD and [sitespeed.io](https://www.sitespeed.io) using Docker-in-Docker. -First, you need GitLab Runner with -[docker-in-docker build](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). +1. First, set up GitLab Runner with a + [docker-in-docker build](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). +1. After configuring the Runner, add a new job to `.gitlab-ci.yml` that generates + the expected report. +1. Define the `performance` job according to your version of GitLab: -Once you set up the Runner, add a new job to `.gitlab-ci.yml` that generates the -expected report. + - For GitLab 12.4 and later - [include](../../../ci/yaml/README.md#includetemplate) the + [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml) provided as a part of your GitLab installation. + - For GitLab versions earlier than 12.4 - Copy and use the job as defined in the + [`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). -For GitLab 12.4 and later, to define the `performance` job, you must -[include](../../../ci/yaml/README.md#includetemplate) the -[`Browser-Performance.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml) -that's provided as a part of your GitLab installation. -For GitLab versions earlier than 12.4, you can copy and use the job as defined -in that template. + CAUTION: **Caution:** + The job definition provided by the template does not support Kubernetes yet. + For a complete example of a more complex setup that works in Kubernetes, see + [`Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml). -CAUTION: **Caution:** -The job definition provided by the template does not support Kubernetes yet. For a complete example of a more complex setup -that works in Kubernetes, see [here](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml). +1. Add the following to your `.gitlab-ci.yml` file: -Add the following to your `.gitlab-ci.yml` file: + ```yaml + include: + template: Verify/Browser-Performance.gitlab-ci.yml -```yaml -include: - template: Verify/Browser-Performance.gitlab-ci.yml + performance: + variables: + URL: https://example.com + ``` -performance: - variables: - URL: https://example.com -``` - -CAUTION: **Caution:** -The job definition provided by the template is supported in GitLab 11.5 and later versions. -It also requires GitLab Runner 11.5 or later. For earlier versions, use the -[previous job definitions](#previous-job-definitions). + CAUTION: **Caution:** + The job definition provided by the template is supported in GitLab 11.5 and later versions. + It also requires GitLab Runner 11.5 or later. For earlier versions, use the + [previous job definitions](#previous-job-definitions). -The above example will create a `performance` job in your CI/CD pipeline and will run +The above example creates a `performance` job in your CI/CD pipeline and runs sitespeed.io against the webpage you defined in `URL` to gather key metrics. The [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance) -is downloaded in order to save the report as a [Performance report artifact](../../../ci/yaml/README.md#artifactsreportsperformance-premium) -that you can later download and analyze. Due to implementation limitations we always +is downloaded to save the report as a [Performance report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsperformance-premium) +that you can later download and analyze. Due to implementation limitations, we always take the latest Performance artifact available. -The full HTML sitespeed.io report will also be saved as an artifact, and if you have -[GitLab Pages](../pages/index.md) enabled, it can be viewed directly in your browser. +The full HTML sitespeed.io report is saved as an artifact, and if +[GitLab Pages](../pages/index.md) is enabled, it can be viewed directly in your browser. -It is also possible to customize options by setting the `SITESPEED_OPTIONS` variable. -For example, this is how to override the number of runs sitespeed.io -will make on the given URL: +You can also customize options by setting the `SITESPEED_OPTIONS` variable. +For example, you can override the number of runs sitespeed.io +makes on the given URL: ```yaml include: @@ -122,27 +116,47 @@ performance: ``` For further customization options for sitespeed.io, including the ability to provide a -list of URLs to test, please see the [Sitespeed.io Configuration](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) +list of URLs to test, please see the +[Sitespeed.io Configuration](https://www.sitespeed.io/documentation/sitespeed.io/configuration/) documentation. TIP: **Tip:** Key metrics are automatically extracted and shown in the merge request widget. +### Configuring degradation threshold + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27599) in GitLab 13.0. + +You can configure the sensitivity of degradation alerts to avoid getting alerts for minor drops in metrics. +This is done by setting the `DEGRADATION_THRESHOLD` variable. In the example below, the alert will only show up +if the `Total Score` metric degrades by 5 points or more: + +```yaml +include: + template: Verify/Browser-Performance.gitlab-ci.yml + +performance: + variables: + URL: https://example.com + DEGRADATION_THRESHOLD: 5 +``` + +The `Total Score` metric is based on sitespeed.io's [coach performance score](https://www.sitespeed.io/documentation/sitespeed.io/metrics/#performance-score). There is more information in [the coach documentation](https://www.sitespeed.io/documentation/coach/how-to/#what-do-the-coach-do). + ### Performance testing on Review Apps -The above CI YML is great for testing against static environments, and it can -be extended for dynamic environments. There are a few extra steps to take to -set this up: +The above CI YAML configuration is great for testing against static environments, and it can +be extended for dynamic environments, but a few extra steps are required: 1. The `performance` job should run after the dynamic environment has started. 1. In the `review` job, persist the hostname and upload it as an artifact so - it's available to the `performance` job (the same can be done for static - environments like staging and production to unify the code path). Saving it - as an artifact is as simple as `echo $CI_ENVIRONMENT_URL > environment_url.txt` + it's available to the `performance` job. The same can be done for static + environments like staging and production to unify the code path. You can save it + as an artifact with `echo $CI_ENVIRONMENT_URL > environment_url.txt` in your job's `script`. 1. In the `performance` job, read the previous artifact into an environment - variable, in this case `$URL` because this is what our sitespeed.io command - uses for the URL parameter. Because Review App URLs are dynamic, we define + variable. In this case, use `$URL` because the sitespeed.io command + uses it for the URL parameter. Because Review App URLs are dynamic, define the `URL` variable through `before_script` instead of `variables`. 1. You can now run the sitespeed.io container against the desired hostname and paths. @@ -183,11 +197,11 @@ performance: ### Previous job definitions CAUTION: **Caution:** -Before GitLab 11.5, Performance job and artifact had to be named specifically +Before GitLab 11.5, the Performance job and artifact had to be named specifically to automatically extract report data and show it in the merge request widget. -While these old job definitions are still maintained they have been deprecated +While these old job definitions are still maintained, they have been deprecated and may be removed in next major release, GitLab 12.0. -You are advised to update your current `.gitlab-ci.yml` configuration to reflect that change. +GitLab recommends you update your current `.gitlab-ci.yml` configuration to reflect that change. For GitLab 11.4 and earlier, the job should look like: diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index a7e712a4c0a..beb90e30906 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -27,7 +27,19 @@ in the merge request widget area:  -For more information, see the Code Climate list of [Supported Languages for Maintainability](https://docs.codeclimate.com/docs/supported-languages-for-maintainability). +Watch a quick walkthrough of Code Quality in action: + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=B32LxtJKo9M">Code Quality: Speed Run</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/B32LxtJKo9M" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +NOTE: **Note:** +For one customer, the auditor found that having Code Quality, SAST, and Container Scanning all automated in GitLab CI/CD was almost better than a manual review! [Read more](https://about.gitlab.com/customers/bi_worldwide/). + +See also the Code Climate list of [Supported Languages for Maintainability](https://docs.codeclimate.com/docs/supported-languages-for-maintainability). ## Use cases @@ -37,7 +49,7 @@ For instance, consider the following workflow: feature in your app faster. 1. With Code Quality reports, they analyze how their implementation is impacting the code quality. -1. The metrics show that their code degrade the quality in 10 points. +1. The metrics show that their code degrades the quality by 10 points. 1. You ask a co-worker to help them with this modification. 1. They both work on the changes until Code Quality report displays no degradations, only improvements. @@ -53,10 +65,10 @@ also requires the GitLab Runner 11.5 or later. For earlier versions, use the This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker. -First, you need GitLab Runner with: +First, you need GitLab Runner configured: -- The [docker-in-docker executor](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). -- Enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB. +- For the [docker-in-docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor). +- With enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB. Once you set up the Runner, include the CodeQuality template in your CI config: @@ -67,7 +79,7 @@ include: The above example will create a `code_quality` job in your CI/CD pipeline which will scan your source code for code quality issues. The report will be saved as a -[Code Quality report artifact](../../../ci/yaml/README.md#artifactsreportscodequality-starter) +[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter) that you can later download and analyze. Due to implementation limitations we always take the latest Code Quality artifact available. @@ -227,7 +239,7 @@ do this: 1. Define a job in your `.gitlab-ci.yml` file that generates the [Code Quality report - artifact](../../../ci/yaml/README.md#artifactsreportscodequality-starter). + artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality-starter). 1. Configure your tool to generate the Code Quality report artifact as a JSON file that implements subset of the [Code Climate spec](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types). diff --git a/doc/user/project/merge_requests/creating_merge_requests.md b/doc/user/project/merge_requests/creating_merge_requests.md index d8a2b427288..544727380e7 100644 --- a/doc/user/project/merge_requests/creating_merge_requests.md +++ b/doc/user/project/merge_requests/creating_merge_requests.md @@ -171,7 +171,7 @@ When the changes are merged, your changes are added to the upstream repository a the branch as per specification. After your work is merged, if you don't intend to make any other contributions to the upstream project, you can unlink your fork from its upstream project in the **Settings > Advanced Settings** section by -[removing the forking relashionship](../settings/index.md#removing-a-fork-relationship). +[removing the forking relationship](../settings/index.md#removing-a-fork-relationship). For further details, [see the forking workflow documentation](../repository/forking_workflow.md). diff --git a/doc/user/project/merge_requests/img/accessibility_mr_widget_v13_0.png b/doc/user/project/merge_requests/img/accessibility_mr_widget_v13_0.png Binary files differnew file mode 100644 index 00000000000..c52bf9964f1 --- /dev/null +++ b/doc/user/project/merge_requests/img/accessibility_mr_widget_v13_0.png diff --git a/doc/user/project/merge_requests/img/code_quality.png b/doc/user/project/merge_requests/img/code_quality.png Binary files differindex a20f6476fb8..3c6c92baad2 100644 --- a/doc/user/project/merge_requests/img/code_quality.png +++ b/doc/user/project/merge_requests/img/code_quality.png diff --git a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md index bb5aadfa9b9..fdcb1049ef7 100644 --- a/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md +++ b/doc/user/project/merge_requests/reviewing_and_managing_merge_requests.md @@ -64,6 +64,15 @@ list.  +### Merge requests commit navigation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18140) in GitLab 13.0. + +To seamlessly navigate among commits in a merge request, from the **Commits** tab, click one of +the commits to open the single-commit view. From there, you can navigate among the commits +by clicking the **Prev** and **Next** buttons on the top-right of the page or by using the +<kbd>X</kbd> and <kbd>C</kbd> keyboard shortcuts. + ### Incrementally expand merge request diffs By default, the diff shows only the parts of a file which are changed. @@ -102,7 +111,7 @@ you will be able to see: - Both pre and post-merge pipelines and the environment information if any. - Which deployments are in progress. -If there's an [environment](../../../ci/environments.md) and the application is +If there's an [environment](../../../ci/environments/index.md) and the application is successfully deployed to it, the deployed environment and the link to the Review App will be shown as well. diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index 71fbdaf112f..84d60fca72d 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -17,14 +17,14 @@ MR is merged. ## How test coverage visualization works Collecting the coverage information is done via GitLab CI/CD's -[artifacts reports feature](../../../ci/yaml/README.md#artifactsreports). +[artifacts reports feature](../../../ci/pipelines/job_artifacts.md#artifactsreports). You can specify one or more coverage reports to collect, including wildcard paths. GitLab will then take the coverage information in all the files and combine it together. For the coverage analysis to work, you have to provide a properly formatted [Cobertura XML](https://cobertura.github.io/cobertura/) report to -[`artifacts:reports:cobertura`](../../../ci/yaml/README.md#artifactsreportscobertura). +[`artifacts:reports:cobertura`](../../../ci/pipelines/job_artifacts.md#artifactsreportscobertura). This format was originally developed for Java, but most coverage analysis frameworks for other languages have plugins to add support for it, like: diff --git a/doc/user/project/merge_requests/versions.md b/doc/user/project/merge_requests/versions.md index 2f51af24a95..84934148bdc 100644 --- a/doc/user/project/merge_requests/versions.md +++ b/doc/user/project/merge_requests/versions.md @@ -57,7 +57,7 @@ source and target branch can be shown mixed together making it hard to understand which changes are being added and which already exist in the target branch. -In GitLab 12.10, we added an **experimental** comparison mode, which +In GitLab 12.10, we added a comparison mode, which shows a diff calculated by simulating how it would look like once merged - a more accurate representation of the changes rather than using the base of the two branches. The new mode is available from the comparison target drop down @@ -67,26 +67,6 @@ current default comparison.  -### Enable or disable `HEAD` comparison mode **(CORE ONLY)** - -`HEAD` comparison mode is under development and not ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session) -can enable it for your instance. You're welcome to test it, but use it at your -own risk. - -To enable it: - -```ruby -Feature.enable(:diff_compare_with_head) -``` - -To disable it: - -```ruby -Feature.disable(:diff_compare_with_head) -``` - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md index c5ab8d66852..120c7a35cb2 100644 --- a/doc/user/project/new_ci_build_permissions_model.md +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -229,5 +229,5 @@ to projects and their project permissions. ### API -GitLab API cannot be used via `CI_JOB_TOKEN` but there is a [proposal](https://gitlab.com/gitlab-org/gitlab-foss/issues/29566) +GitLab API cannot be used via `CI_JOB_TOKEN` but there is a [proposal](https://gitlab.com/gitlab-org/gitlab/-/issues/35067) to support it. diff --git a/doc/user/project/operations/alert_management.md b/doc/user/project/operations/alert_management.md new file mode 100644 index 00000000000..2dcf72aaf01 --- /dev/null +++ b/doc/user/project/operations/alert_management.md @@ -0,0 +1,79 @@ +# Alert Management + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2877) in GitLab 13.0. + +Alert Management enables developers to easily discover and view the alerts +generated by their application. By surfacing alert information where the code is +being developed, efficiency and awareness can be increased. + +## Enable Alert Management + +NOTE: **Note:** +You will need at least Maintainer [permissions](../../permissions.md) to enable the Alert Management feature. + +1. Follow the [instructions for toggling generic alerts](../integrations/generic_alerts.md#setting-up-generic-alerts) +1. You can now visit **{cloud-gear}** **Operations > Alerts** in your project's sidebar to [view a list](#alert-management-list) of alerts. + + + +## Populate Alert data + +To populate data, see the instructions for +[customizing the payload](../integrations/generic_alerts.md) and making a +request to the alerts endpoint. + +## Alert Management severity + +Each level of alert contains a uniquely shaped and color-coded icon to help +you identify the severity of a particular alert. These severity icons help you +immediately identify which alerts you should prioritize investigating: + + + +Alerts contain one of the following icons: + +- **Critical**: **{severity-critical}** and hexadecimal color `#8b2615` +- **High**: **{severity-high}** and hexadecimal color `#c0341d` +- **Medium**: **{severity-medium}** and hexadecimal color `#fca429` +- **Low**: **{severity-low}** and hexadecimal color `#fdbc60` +- **Info**: **{severity-info}** and hexadecimal color `#418cd8` +- **Unknown**: **{severity-unknown}** and hexadecimal color `#bababa` + +## Alert Management list + +NOTE: **Note:** +You will need at least Developer [permissions](../../permissions.md) to view the Alert Management list. + +You can find the Alert Management list at **{cloud-gear}** **Operations > Alerts** in your project's sidebar. +Each alert contains the following metrics: + + + +- **Severity** - The current importance of a alert and how much attention it should receive. +- **Start time** - How long ago the alert fired. This field uses the standard GitLab pattern of `X time ago`, but is supported by a granular date/time tooltip depending on the user's locale. +- **End time** - How long ago the alert fired was resolved. This field uses the standard GitLab pattern of `X time ago`, but is supported by a granular date/time tooltip depending on the user's locale. +- **Alert description** - The description of the alert, which attempts to capture the most meaningful data. +- **Event count** - The number of times that an alert has fired. +- **Status** - The [current status](#alert-management-statuses) of the alert. + +### Alert Management statuses + +Each alert contains a status dropdown to indicate which alerts need investigation. +Standard alert statuses include `triggered`, `acknowledged`, and `resolved`: + +- **Triggered**: No one has begun investigation. +- **Acknowledged**: Someone is actively investigating the problem. +- **Resolved**: No further work is required. + +## Alert Management details + +NOTE: **Note:** +You will need at least Developer [permissions](../../permissions.md) to view Alert Management details. + +Navigate to the Alert Management detail view by visiting the [Alert Management list](#alert-management-list) and selecting an Alert from the list. + + + +### Update an Alert's status + +The Alert Management detail view allows users to update the Alert Status. See [Alert Management statuses](#alert-management-statuses) for more details. diff --git a/doc/user/project/operations/error_tracking.md b/doc/user/project/operations/error_tracking.md index a95459e093a..23a50fd7766 100644 --- a/doc/user/project/operations/error_tracking.md +++ b/doc/user/project/operations/error_tracking.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: Health +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Error Tracking > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/169) in GitLab 11.8. @@ -53,7 +59,7 @@ From error list, users can navigate to the error details page by clicking the ti This page has: - A link to the Sentry issue. -- A link to the GitLab commit if the Sentry [release id/version](https://docs.sentry.io/workflow/releases/?platform=javascript#configure-sdk) on the Sentry Issue's first release matches a commit SHA in your GitLab hosted project. +- A link to the GitLab commit if the Sentry [release ID/version](https://docs.sentry.io/workflow/releases/?platform=javascript#configure-sdk) on the Sentry Issue's first release matches a commit SHA in your GitLab hosted project. - Other details about the issue, including a full stack trace. - In [GitLab 12.7 and newer](https://gitlab.com/gitlab-org/gitlab/issues/36246), language and urgency are displayed. diff --git a/doc/user/project/operations/feature_flags.md b/doc/user/project/operations/feature_flags.md index f13fbcb2722..8716d5feb4a 100644 --- a/doc/user/project/operations/feature_flags.md +++ b/doc/user/project/operations/feature_flags.md @@ -1,3 +1,9 @@ +--- +stage: Release +group: Progressive Delivery +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Feature Flags **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/7433) in GitLab 11.4. @@ -15,6 +21,13 @@ This helps reducing risk and allows you to easily manage which features to enabl GitLab offers a Feature Flags interface that allows you to create, toggle and remove feature flags. +<div class="video-fallback"> + <a href="https://www.youtube.com/watch?v=5tw2p6lwXxo">Watch</a> a use case between Feature Flags and Sentry Error Tracking +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/5tw2p6lwXxo" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + ## How it works Underneath, GitLab uses [unleash](https://github.com/Unleash/unleash), a feature @@ -36,10 +49,10 @@ To add a new feature flag: 1. Click on the **New Feature Flag** button. 1. Give it a name. - NOTE: **Note:** - A name can contain only lowercase letters, digits, underscores (`_`) - and dashes (`-`), must start with a letter, and cannot end with a dash (`-`) - or an underscore (`_`). + NOTE: **Note:** + A name can contain only lowercase letters, digits, underscores (`_`) + and dashes (`-`), must start with a letter, and cannot end with a dash (`-`) + or an underscore (`_`). 1. Give it a description (optional, 255 characters max). 1. Define environment [specs](#define-environment-specs). If you want the flag on by default, enable the catch-all [wildcard spec (`*`)](#define-environment-specs) @@ -66,24 +79,59 @@ For example, you may not want to enable a feature flag on production until your first confirmed that the feature is working correctly on testing environments. To handle these situations, you can enable a feature flag on a particular environment -with [Environment specs](../../../ci/environments.md#scoping-environments-with-specs). +with [Environment specs](../../../ci/environments/index.md#scoping-environments-with-specs). You can define multiple specs per flag so that you can control your feature flag more granularly. To define specs for each environment: 1. Navigate to your project's **Operations > Feature Flags**. 1. Click on the **New Feature Flag** button or edit an existing flag. -1. Set the status of the default [spec](../../../ci/environments.md#scoping-environments-with-specs) (`*`). Choose a rollout strategy. This status and rollout strategy combination will be used for _all_ environments. -1. If you want to enable/disable the feature on a specific environment, create a new [spec](../../../ci/environments.md#scoping-environments-with-specs) and type the environment name. +1. Set the status of the default [spec](../../../ci/environments/index.md#scoping-environments-with-specs) (`*`). Choose a rollout strategy. This status and rollout strategy combination will be used for _all_ environments. +1. If you want to enable/disable the feature on a specific environment, create a new [spec](../../../ci/environments/index.md#scoping-environments-with-specs) and type the environment name. 1. Set the status and rollout strategy of the additional spec. This status and rollout strategy combination takes precedence over the default spec since we always use the most specific match available. 1. Click **Create feature flag** or **Update feature flag**.  NOTE: **NOTE** -We'd highly recommend you to use the [Environment](../../../ci/environments.md) +We'd highly recommend you to use the [Environment](../../../ci/environments/index.md) feature in order to quickly assess which flag is enabled per environment. +## Feature flag behavior change in 13.0 + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35555) in GitLab 13.0. + +Starting in GitLab 13.0, you can apply a feature flag strategy across multiple environment specs, +without defining the strategy multiple times. + +This feature is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can enable it for your instance. + +To enable it: + +```ruby +Feature.enable(:feature_flags_new_version) +``` + +To disable it: + +```ruby +Feature.disable(:feature_flags_new_version) +``` + +### Applying a strategy to environments + +After a strategy is defined, it applies to **All Environments** by default. To +make a strategy apply to a specific environment spec: + +1. Click the **Add Environment** button. +1. Create a new + [spec](../../../ci/environments/index.md#scoping-environments-with-specs). + +To apply the strategy to multiple environment specs, repeat these steps. + ## Feature Flag strategies GitLab Feature Flag adopts [Unleash](https://unleash.github.io) @@ -148,12 +196,12 @@ To get the access credentials that your application will need to talk to GitLab: 1. Navigate to your project's **Operations > Feature Flags**. 1. Click on the **Configure** button to see the values: - - **API URL**: URL where the client (application) connects to get a list of feature flags. - - **Instance ID**: Unique token that authorizes the retrieval of the feature flags. - - **Application name**: The name of the running environment. For instance, - if the application runs for production server, application name would be - `production` or similar. This value is used for - [the environment spec evaluation](#define-environment-specs). + - **API URL**: URL where the client (application) connects to get a list of feature flags. + - **Instance ID**: Unique token that authorizes the retrieval of the feature flags. + - **Application name**: The name of the running environment. For instance, + if the application runs for a production server, application name would be + `production` or similar. This value is used for + [the environment spec evaluation](#define-environment-specs). NOTE: **Note:** The meaning of these fields might change over time. For example, we are not sure @@ -231,7 +279,7 @@ func main() { Here's an example of how to integrate the feature flags in a Ruby application. -The Unleash client is given a user id for use with a **Percent rollout (logged in users)** rollout strategy or a list of **Target Users**. +The Unleash client is given a user ID for use with a **Percent rollout (logged in users)** rollout strategy or a list of **Target Users**. ```ruby #!/usr/bin/env ruby @@ -265,3 +313,4 @@ to control them in an automated flow: - [Feature Flags API](../../../api/feature_flags.md) - [Feature Flag Specs API](../../../api/feature_flag_specs.md) +- [Feature Flag User Lists API](../../../api/feature_flag_user_lists.md) diff --git a/doc/user/project/operations/img/alert_detail_v13_0.png b/doc/user/project/operations/img/alert_detail_v13_0.png Binary files differnew file mode 100644 index 00000000000..7da09407cd5 --- /dev/null +++ b/doc/user/project/operations/img/alert_detail_v13_0.png diff --git a/doc/user/project/operations/img/alert_management_1_v13_0.png b/doc/user/project/operations/img/alert_management_1_v13_0.png Binary files differnew file mode 100644 index 00000000000..dbc1e795b16 --- /dev/null +++ b/doc/user/project/operations/img/alert_management_1_v13_0.png diff --git a/doc/user/project/operations/img/alert_management_1_v13_1.png b/doc/user/project/operations/img/alert_management_1_v13_1.png Binary files differnew file mode 100644 index 00000000000..c01b4749eda --- /dev/null +++ b/doc/user/project/operations/img/alert_management_1_v13_1.png diff --git a/doc/user/project/operations/img/alert_management_severity_v13_0.png b/doc/user/project/operations/img/alert_management_severity_v13_0.png Binary files differnew file mode 100644 index 00000000000..f996d6e88f4 --- /dev/null +++ b/doc/user/project/operations/img/alert_management_severity_v13_0.png diff --git a/doc/user/project/operations/index.md b/doc/user/project/operations/index.md index df7ce61525e..954f88fe4f2 100644 --- a/doc/user/project/operations/index.md +++ b/doc/user/project/operations/index.md @@ -4,7 +4,7 @@ GitLab provides a variety of tools to help operate and maintain your applications: - Collect [Prometheus metrics](../integrations/prometheus_library/index.md). -- Deploy to different [environments](../../../ci/environments.md). +- Deploy to different [environments](../../../ci/environments/index.md). - Connect your project to a [Kubernetes cluster](../clusters/index.md). - Manage your infrastructure with [Infrastructure as Code](../../infrastructure/index.md) approaches. - Discover and view errors generated by your applications with [Error Tracking](error_tracking.md). diff --git a/doc/user/project/operations/linking_to_an_external_dashboard.md b/doc/user/project/operations/linking_to_an_external_dashboard.md index 620d9d70e4d..8e1117de4c7 100644 --- a/doc/user/project/operations/linking_to_an_external_dashboard.md +++ b/doc/user/project/operations/linking_to_an_external_dashboard.md @@ -13,7 +13,7 @@ You can add a button to the Monitoring dashboard linking directly to your existi  1. There should now be a button on your - [Monitoring dashboard](../../../ci/environments.md#monitoring-environments) which + [Monitoring dashboard](../../../ci/environments/index.md#monitoring-environments) which will open the URL you entered in the above step.  diff --git a/doc/user/project/operations/tracing.md b/doc/user/project/operations/tracing.md index 8282a980ced..07f60c37f1b 100644 --- a/doc/user/project/operations/tracing.md +++ b/doc/user/project/operations/tracing.md @@ -1,3 +1,9 @@ +--- +stage: Monitor +group: APM +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + # Tracing **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/7903) in GitLab Ultimate 11.5. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md index 172f5d4377f..6e48afad96a 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md @@ -1,5 +1,8 @@ --- type: concepts +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # DNS records overview diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 07bd2a61eb8..a5fc5b00b53 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -2,6 +2,9 @@ last_updated: 2019-07-04 type: reference, howto disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pages/getting_started_part_three.html' +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Custom domains and SSL/TLS Certificates @@ -60,6 +63,11 @@ according to the type of domain you want to use with your Pages site: - [For subdomains](#for-subdomains), `subdomain.example.com`. - [For both](#for-both-root-and-subdomains). +NOTE: **Note:** +You can [configure IPv6 on self-managed instances](../../../../administration/pages/index.md#advanced-configuration), +but IPv6 is not currently configured for Pages on GitLab.com. +Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214718) for details. + ##### For root domains Root domains (`example.com`) require: diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index f80b741fb77..8b180d438ef 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -1,6 +1,9 @@ --- type: reference description: "Automatic Let's Encrypt SSL certificates for GitLab Pages." +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Pages integration with Let's Encrypt diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md index bc51a9c90d2..e307b807aaa 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/ssl_tls_concepts.md @@ -1,5 +1,8 @@ --- type: concepts +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # SSL/TLS Certificates diff --git a/doc/user/project/pages/getting_started/fork_sample_project.md b/doc/user/project/pages/getting_started/fork_sample_project.md index ef30606a9ab..00cea846f91 100644 --- a/doc/user/project/pages/getting_started/fork_sample_project.md +++ b/doc/user/project/pages/getting_started/fork_sample_project.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # New Pages website from a forked sample diff --git a/doc/user/project/pages/getting_started/new_or_existing_website.md b/doc/user/project/pages/getting_started/new_or_existing_website.md index 027a238bf02..9a00b724753 100644 --- a/doc/user/project/pages/getting_started/new_or_existing_website.md +++ b/doc/user/project/pages/getting_started/new_or_existing_website.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Start a new Pages website from scratch or deploy an existing website diff --git a/doc/user/project/pages/getting_started/pages_bundled_template.md b/doc/user/project/pages/getting_started/pages_bundled_template.md index 9c097c9acdb..745c50e8d65 100644 --- a/doc/user/project/pages/getting_started/pages_bundled_template.md +++ b/doc/user/project/pages/getting_started/pages_bundled_template.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # New Pages website from a bundled template diff --git a/doc/user/project/pages/getting_started_part_four.md b/doc/user/project/pages/getting_started_part_four.md index 645cc1c5795..4e95b5d5a69 100644 --- a/doc/user/project/pages/getting_started_part_four.md +++ b/doc/user/project/pages/getting_started_part_four.md @@ -1,6 +1,9 @@ --- last_updated: 2020-01-06 type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Creating and Tweaking GitLab CI/CD for GitLab Pages @@ -37,8 +40,8 @@ anything for them to work. Explaining [every detail of GitLab CI/CD](../../../ci/yaml/README.md) and GitLab Runner is out of the scope of this guide, but we'll need to understand just a few things to be able to write our own -`.gitlab-ci.yml` or tweak an existing one. It's an -[Yaml](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html) file, +`.gitlab-ci.yml` or tweak an existing one. It's a +[YAML](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html) file, with its own syntax. You can always check your CI syntax with the [GitLab CI/CD Lint Tool](https://gitlab.com/ci/lint). @@ -60,7 +63,7 @@ jekyll build ### Script -To transpose this script to Yaml, it would be like this: +To transpose this script to YAML, it would be like this: ```yaml script: @@ -364,7 +367,7 @@ from Jekyll `_config.yml` file, otherwise Jekyll will understand it as a regular directory to build together with the site: -```yml +```yaml exclude: - vendor ``` diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index b876e547ba5..c0bdd4b7f0b 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -1,6 +1,9 @@ --- last_updated: 2018-06-04 type: concepts, reference +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Pages domain names, URLs, and baseurls diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 02bb8e13f4a..e81c9699153 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -2,6 +2,9 @@ description: 'Learn how to use GitLab Pages to deploy a static website at no additional cost.' last_updated: 2019-06-04 type: index, reference +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Pages diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index f77c572664a..e36dfd89ab3 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,6 +1,9 @@ --- type: reference last_updated: 2020-01-06 +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Exploring GitLab Pages @@ -86,11 +89,9 @@ When using Pages under the general domain of a GitLab instance (`*.example.io`), you _cannot_ use HTTPS with sub-subdomains. That means that if your username/groupname contains a dot, for example `foo.bar`, the domain `https://foo.bar.example.io` will _not_ work. This is a limitation of the -[HTTP Over TLS protocol][rfc]. HTTP pages will continue to work provided you +[HTTP Over TLS protocol](https://tools.ietf.org/html/rfc2818#section-3.1). HTTP pages will continue to work provided you don't redirect HTTP to HTTPS. -[rfc]: https://tools.ietf.org/html/rfc2818#section-3.1 "HTTP Over TLS RFC" - GitLab Pages [does **not** support group websites for subgroups](../../group/subgroups/index.md#limitations). You can only create the highest-level group website. @@ -169,13 +170,10 @@ pages: - pages ``` -See an example that has different files in the [`master` branch][jekyll-master] -and the source files for Jekyll are in a [`pages` branch][jekyll-pages] which +See an example that has different files in the [`master` branch](https://gitlab.com/pages/jekyll-branched/tree/master) +and the source files for Jekyll are in a [`pages` branch](https://gitlab.com/pages/jekyll-branched/tree/pages) which also includes `.gitlab-ci.yml`. -[jekyll-master]: https://gitlab.com/pages/jekyll-branched/tree/master -[jekyll-pages]: https://gitlab.com/pages/jekyll-branched/tree/pages - ### Serving compressed assets Most modern browsers support downloading files in a compressed format. This diff --git a/doc/user/project/pages/pages_access_control.md b/doc/user/project/pages/pages_access_control.md index 1d8119cfe87..7fe4c4c051d 100644 --- a/doc/user/project/pages/pages_access_control.md +++ b/doc/user/project/pages/pages_access_control.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # GitLab Pages Access Control @@ -11,6 +14,9 @@ You can enable Pages access control on your project, so that only [members of your project](../../permissions.md#project-members-permissions) (at least Guest) can access your website: +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a demonstration, see [Pages access controls](https://www.youtube.com/watch?v=tSPAr5mQYc8). + 1. Navigate to your project's **Settings > General** and expand **Visibility, project features, permissions**. 1. Toggle the **Pages** button to enable the access control. diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 6f68a2c9907..b134d283ba9 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -4,7 +4,7 @@ type: reference, howto # Protected Tags -> [Introduced][ce-10356] in GitLab 9.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10356) in GitLab 9.1. Protected Tags allow control over who has permission to create tags as well as preventing accidental update or deletion once created. Each rule allows you to match either an individual tag name, or use wildcards to control multiple tags at once. @@ -67,5 +67,3 @@ questions that you know someone might ask. Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> - -[ce-10356]: https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10356 "Protected Tags" diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index e8d94a05e7e..eab88d59867 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -33,10 +33,10 @@ git push -o <push_option> You can use push options to skip a CI/CD pipeline, or pass environment variables. -| Push option | Description | -| ------------------------------ | ------------------------------------------------------------------------------------------- | -| `ci.skip` | Do not create a CI pipeline for the latest push. | -| `ci.variable="<name>=<value>"` | Provide [environment variables](../../ci/variables/README.md) to be used in a CI pipeline, if one is created due to the push. | +| Push option | Description | Introduced in version | +| ------------------------------ | ------------------------------------------------------------------------------------------- |---------------------- | +| `ci.skip` | Do not create a CI pipeline for the latest push. | [11.7](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15643) | +| `ci.variable="<name>=<value>"` | Provide [environment variables](../../ci/variables/README.md) to be used in a CI pipeline, if one is created due to the push. | [12.6](https://gitlab.com/gitlab-org/gitlab/issues/27983) | An example of using `ci.skip`: diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 7937b7193d2..e2d0b616e4b 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -42,7 +42,7 @@ The following quick actions are applicable to descriptions, discussions and thre | `/remove_milestone` | ✓ | ✓ | | Remove milestone | | `/label ~label1 ~label2` | ✓ | ✓ | ✓ | Add label(s). Label names can also start without `~` but mixed syntax is not supported | | `/relabel ~label1 ~label2` | ✓ | ✓ | ✓ | Replace existing label(s) with those specified | -| `/unlabel ~label1 ~label2` | ✓ | ✓ | ✓ | Remove all or specific label(s) | +| `/unlabel ~label1 ~label2` or `/remove_label ~label1 ~label2` | ✓ | ✓ | ✓ | Remove all or specific label(s) | | `/copy_metadata <#issue>` | ✓ | ✓ | | Copy labels and milestone from another issue in the project | | `/copy_metadata <!merge_request>` | ✓ | ✓ | | Copy labels and milestone from another merge request in the project | | `/estimate <<W>w <DD>d <hh>h <mm>m>` | ✓ | ✓ | | Set time estimate. For example, `/estimate 1w 3d 2h 14m` | diff --git a/doc/user/project/releases/img/edit_release_page_v13_0.png b/doc/user/project/releases/img/edit_release_page_v13_0.png Binary files differnew file mode 100644 index 00000000000..1b4343019af --- /dev/null +++ b/doc/user/project/releases/img/edit_release_page_v13_0.png diff --git a/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png b/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png Binary files differnew file mode 100644 index 00000000000..453c7ca93cc --- /dev/null +++ b/doc/user/project/releases/img/release_milestone_dropdown_v13_0.png diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index ca28a79abf4..bdb99d16625 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -1,5 +1,8 @@ --- type: reference, howto +stage: Release +group: Release Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers --- # Releases @@ -102,12 +105,15 @@ The physical location of the asset can change at any time and the direct link wi ### Releases associated with milestones -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/29020) in GitLab 12.5. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/29020) in GitLab 12.5. +> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/39467) to edit milestones in the UI in GitLab 13.0. Releases can optionally be associated with one or more [project milestones](../milestones/index.md#project-milestones-and-group-milestones) by including a `milestones` array in your requests to the -[Releases API](../../../api/releases/index.md#create-a-release). +[Releases API](../../../api/releases/index.md#create-a-release) or by using the dropdown in the [Edit Release](#editing-a-release) page. + + Releases display this association with the **Milestone** indicator in the top section of the Release block on the **Project overview > Releases** page, along @@ -190,22 +196,13 @@ the edit button (pencil icon) in the top-right corner of the release you want to This will bring you to the **Edit Release** page, from which you can change some of the release's details. - + -Currently, it is only possible to edit the release title, notes, and asset -links. To change other release information, such as its tag, associated -milestones, or release date, use the [Releases +Currently, it is only possible to edit the release title, notes, associated milestones, and asset +links. To change other release information, such as its tag, or release date, use the [Releases API](../../../api/releases/index.md#update-a-release). Editing this information through the **Edit Release** page is planned for a future version of GitLab. -Please note that the ability to edit asset links is currently behind a feature -flag which is disabled by default. For self-managed instances, it can be enabled -through the Rails console by a GitLab administrator with the following command: - -```ruby -Feature.enable(:release_asset_link_editing) -``` - ## Notification for Releases > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/26001) in GitLab 12.4. @@ -258,6 +255,9 @@ generate Release Evidence for an existing release. Because of this, [each releas can have multiple Release Evidence snapshots. You can view the Release Evidence and its details on the Release page. +NOTE: **Note:** +When the issue tracker is disabled, release evidence [is not collected](https://gitlab.com/gitlab-org/gitlab/-/issues/208397). + Release Evidence is stored as a JSON object, so you can compare evidence by using commonly-available tools. @@ -360,6 +360,39 @@ terminal. Read the [GitLab Releaser documentation](https://gitlab.com/gitlab-org/gitlab-releaser/-/tree/master/docs/index.md) for details. +## Set a deploy freeze + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29382) in GitLab 13.0. + +With a deploy freeze, you can prevent an unintended production release during a +period of time you specify, whether a company event or public holiday. You can +now rely on the enforcement of policies that are typically outside the scope of +GitLab to reduce uncertainty and risk when automating deployments. + +Deploy freeze periods are set at the Project level, and may be created and +managed using the [Freeze Periods API](../../../api/freeze_periods.md). +Each Freeze Period has a `freeze_start` and a `freeze_end`, which are defined +as [crontab](https://crontab.guru/) entries. If a project contains multiple +freeze periods, all will apply, and should they overlap, the freeze covers the +complete overlapped period. + +During pipeline processing, GitLab CI creates an environment variable named +`$CI_DEPLOY_FREEZE` if the currently executing job is within a +Freeze Period. + +To take advantage of this variable, create a `rules` entry in your +`gitlab-ci.yaml` to prevent the deployment job from executing. + +For example: + +```yaml +deploy_to_production: + stage: deploy + script: deploy_to_prod.sh + rules: + - if: $CI_DEPLOY_FREEZE == null +``` + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/repository/file_finder.md b/doc/user/project/repository/file_finder.md index 91e6d2912d1..ac10071e578 100644 --- a/doc/user/project/repository/file_finder.md +++ b/doc/user/project/repository/file_finder.md @@ -4,7 +4,7 @@ disqus_identifier: 'https://docs.gitlab.com/ee/workflow/file_finder.html' # File finder -> [Introduced][gh-9889] in GitLab 8.4. +> [Introduced](https://github.com/gitlabhq/gitlabhq/pull/9889) in GitLab 8.4. The file finder feature allows you to search for a file in a repository using the GitLab UI. @@ -12,7 +12,7 @@ GitLab UI. You can find the **Find File** button when in the **Files** section of a project. - + For those who prefer to keep their fingers on the keyboard, there is a [shortcut button](../../shortcuts.md) as well, which you can invoke from _anywhere_ @@ -23,23 +23,20 @@ Press `t` to launch the File search function when in **Issues**, Start typing what you are searching for and watch the magic happen. With the up/down arrows, you go up and down the results, with `Esc` you close the search -and go back to **Files**. +and go back to **Files** ## How it works The File finder feature is powered by the [Fuzzy filter](https://github.com/jeancroy/fuzz-aldrin-plus) library. -It implements a fuzzy search with highlight, and tries to provide intuitive +It implements a fuzzy search with the highlight and tries to provide intuitive results by recognizing patterns that people use while searching. -For example, consider the [GitLab CE repository][ce] and that we want to open +For example, consider the [GitLab FOSS repository](https://gitlab.com/gitlab-org/gitlab-foss/tree/master) and that we want to open the `app/controllers/admin/deploy_keys_controller.rb` file. -Using fuzzy search, we start by typing letters that get us closer to the file. +Using a fuzzy search, we start by typing letters that get us closer to the file. **Tip:** To narrow down your search, include `/` in your search terms. - - -[gh-9889]: https://github.com/gitlabhq/gitlabhq/pull/9889 "File finder pull request" -[ce]: https://gitlab.com/gitlab-org/gitlab-foss/tree/master "GitLab CE repository" + diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index 126144de703..a49701017f3 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -72,5 +72,3 @@ changes are added to the repository and branch you're merging into. ## Removing a fork relationship You can unlink your fork from its upstream project in the [advanced settings](../settings/index.md#removing-a-fork-relationship). - -[gitlab flow]: https://about.gitlab.com/blog/2014/09/29/gitlab-flow/ "GitLab Flow blog post" diff --git a/doc/user/project/repository/img/file_finder_find_button.png b/doc/user/project/repository/img/file_finder_find_button.png Binary files differdeleted file mode 100644 index 0c2d7d7bc73..00000000000 --- a/doc/user/project/repository/img/file_finder_find_button.png +++ /dev/null diff --git a/doc/user/project/repository/img/file_finder_find_button_v12_10.png b/doc/user/project/repository/img/file_finder_find_button_v12_10.png Binary files differnew file mode 100644 index 00000000000..e93db946005 --- /dev/null +++ b/doc/user/project/repository/img/file_finder_find_button_v12_10.png diff --git a/doc/user/project/repository/img/file_finder_find_file.png b/doc/user/project/repository/img/file_finder_find_file.png Binary files differdeleted file mode 100644 index c2212c7cd9e..00000000000 --- a/doc/user/project/repository/img/file_finder_find_file.png +++ /dev/null diff --git a/doc/user/project/repository/img/file_finder_find_file_v12_10.png b/doc/user/project/repository/img/file_finder_find_file_v12_10.png Binary files differnew file mode 100644 index 00000000000..1404ccc6d0b --- /dev/null +++ b/doc/user/project/repository/img/file_finder_find_file_v12_10.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index f9c953db3e3..055443daa1f 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -162,7 +162,7 @@ Via command line, you can commit multiple times before pushing. you will trigger a pipeline per push, not per commit. - **Skip pipelines:** You can add to you commit message the keyword - [`[ci skip]`](../../../ci/yaml/README.md#skipping-jobs) + [`[ci skip]`](../../../ci/yaml/README.md#skip-pipeline) and GitLab CI/CD will skip that pipeline. - **Cross-link issues and merge requests:** [Cross-linking](../issues/crosslinking_issues.md#from-commit-messages) @@ -199,7 +199,7 @@ of commits to the fewest, and displayed on a nice graph: ## Repository graph -The repository graph displays visually the Git flow strategy used in that repository: +The repository graph displays the history of the repository network visually, including branches and merges. This can help you visualize the Git flow strategy used in the repository:  @@ -215,7 +215,7 @@ minutes.  Not all files are detected, among others; documentation, -vendored code, and most markup languages are excluded. This behaviour can be +vendored code, and most markup languages are excluded. This behavior can be adjusted by overriding the default. For example, to enable `.proto` files to be detected, add the following to `.gitattributes` in the root of your repository. diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index 8064eacf404..fdbea385998 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -55,6 +55,7 @@ For an existing project, you can set up push mirroring as follows: 1. Select **Push** from the **Mirror direction** dropdown. 1. Select an authentication method from the **Authentication method** dropdown, if necessary. 1. Check the **Only mirror protected branches** box, if necessary. +1. Check the **Keep divergent refs** box, if desired. 1. Click the **Mirror repository** button to save the configuration.  @@ -88,6 +89,27 @@ You can choose to only push your protected branches from GitLab to your remote r To use this option, check the **Only mirror protected branches** box when creating a repository mirror. +### Keep divergent refs **(CORE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208828) in GitLab 13.0. + +By default, if any ref on the remote mirror has diverged from the local +repository, the *entire push* will fail, and nothing will be updated. + +For example, if a repository has `master`, `develop`, and `stable` branches that +have been mirrored to a remote, and then a new commit is added to `develop` on +the mirror, the next push attempt will fail, leaving `master` and `stable` +out-of-date despite not having diverged. No change on any branch can be mirrored +until the divergence is resolved. + +With the **Keep divergent refs** option enabled, the `develop` branch is +skipped, allowing `master` and `stable` to be updated. The mirror status will +reflect that `develop` has diverged and was skipped, and be marked as a failed +update. + +NOTE: **Note:** +After the mirror is created, this option can currently only be modified via the [API](../../../api/remote_mirrors.md). + ## Setting up a push mirror from GitLab to GitHub **(CORE)** To set up a mirror from GitLab to GitHub, you need to follow these steps: diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index 19238839a5e..20143af0b33 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -2,21 +2,21 @@ type: concepts, howto --- -# Signing commits with x509 +# Signing commits and tags with X.509 -[x509](https://en.wikipedia.org/wiki/X.509) is a standard format for public key +[X.509](https://en.wikipedia.org/wiki/X.509) is a standard format for public key certificates issued by a public or private Public Key Infrastructure (PKI). -Personal x509 certificates are used for authentication or signing purposes +Personal X.509 certificates are used for authentication or signing purposes such as SMIME, but Git also supports signing of commits and tags -with x509 certificates in a similar way as with [GPG](../gpg_signed_commits/index.md). -The main difference is the trust anchor which is the PKI for x509 certificates +with X.509 certificates in a similar way as with [GPG](../gpg_signed_commits/index.md). +The main difference is the trust anchor which is the PKI for X.509 certificates instead of a web of trust with GPG. -## How GitLab handles x509 +## How GitLab handles X.509 GitLab uses its own certificate store and therefore defines the trust chain. -For a commit to be *verified* by GitLab: +For a commit or tag to be *verified* by GitLab: - The signing certificate email must match a verified email address used by the committer in GitLab. - The Certificate Authority has to be trusted by the GitLab instance, see also @@ -25,9 +25,14 @@ For a commit to be *verified* by GitLab: which is usually up to three years. - The signing time is equal or later then commit time. -NOTE: **Note:** There is no certificate revocation list check in place at the moment. +NOTE: **Note:** Certificate revocation lists are checked on a daily basis via background worker. -## Obtaining an x509 key pair +NOTE: **Note:** Self signed certificates without `authorityKeyIdentifier`, +`subjectKeyIdentifier`, and `crlDistributionPoints` are not supported. We +recommend using certificates from a PKI that are in line with +[RFC 5280](https://tools.ietf.org/html/rfc5280). + +## Obtaining an X.509 key pair If your organization has Public Key Infrastructure (PKI), that PKI will provide an S/MIME key. @@ -37,12 +42,12 @@ own self-signed one, or purchase one. MozillaZine keeps a nice collection of [S/MIME-capable signing authorities](http://kb.mozillazine.org/Getting_an_SMIME_certificate) and some of them generate keys for free. -## Associating your x509 certificate with Git +## Associating your X.509 certificate with Git -To take advantage of X509 signing, you will need Git 2.19.0 or later. You can +To take advantage of X.509 signing, you will need Git 2.19.0 or later. You can check your Git version with: -```sh +```shell git --version ``` @@ -52,7 +57,7 @@ If you have the correct version, you can proceed to configure Git. Configure Git to use your key for signing: -```sh +```shell signingkey = $( gpgsm --list-secret-keys | egrep '(key usage|ID)' | grep -B 1 digitalSignature | awk '/ID/ {print $2}' ) git config --global user.signingkey $signingkey git config --global gpg.format x509 @@ -64,21 +69,21 @@ Install [smimesign](https://github.com/github/smimesign) by downloading the installer or via `brew install smimesign` on MacOS. Get the ID of your certificate with `smimesign --list-keys` and set your -signingkey `git config --global user.signingkey ID`, then configure x509: +signingkey `git config --global user.signingkey ID`, then configure X.509: -```sh +```shell git config --global gpg.x509.program smimesign git config --global gpg.format x509 ``` ## Signing commits -After you have [associated your x509 certificate with Git](#associating-your-x509-certificate-with-git) you +After you have [associated your X.509 certificate with Git](#associating-your-x509-certificate-with-git) you can start signing your commits: 1. Commit like you used to, the only difference is the addition of the `-S` flag: - ```sh + ```shell git commit -S -m "feat: x509 signed commits" ``` @@ -87,7 +92,7 @@ can start signing your commits: If you don't want to type the `-S` flag every time you commit, you can tell Git to sign your commits automatically: -```sh +```shell git config --global commit.gpgsign true ``` @@ -95,6 +100,34 @@ git config --global commit.gpgsign true To verify that a commit is signed, you can use the `--show-signature` flag: -```sh +```shell git log --show-signature ``` + +## Signing tags + +After you have [associated your X.509 certificate with Git](#associating-your-x509-certificate-with-git) you +can start signing your tags: + +1. Tag like you used to, the only difference is the addition of the `-s` flag: + + ```shell + git tag -s v1.1.1 -m "My signed tag" + ``` + +1. Push to GitLab and check that your tags [are verified](#verifying-tags). + +If you don't want to type the `-s` flag every time you tag, you can tell Git +to sign your tags automatically: + +```shell +git config --global tag.gpgsign true +``` + +## Verifying tags + +To verify that a tag is signed, you can use the `--verify` flag: + +```shell +git tag --verify v1.1.1 +``` diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index 8f4ec7bbbed..50343e52a68 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -10,6 +10,9 @@ Requirements allow you to create criteria to check your products against. They can be based on users, stakeholders, system, software, or anything else you find important to capture. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [GitLab 12.10 Introduces Requirements Management](https://www.youtube.com/watch?v=uSS7oUNSEoU). +  ## Create a requirement diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index f18a202c63b..d021f259015 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -1,4 +1,4 @@ -# Service Desk **(PREMIUM)** +# Service Desk **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/149) in [GitLab Premium 9.1](https://about.gitlab.com/releases/2017/04/22/gitlab-9-1-released/#service-desk-eep). @@ -95,18 +95,18 @@ directory in your repository. Commit and push to your default branch. The **Thank you email** is the email sent to a user after they submit an issue. The file name of the template has to be `thank_you.md`. -You can use `%{ISSUE_ID}` placeholder which will be replaced by an issue iid in the email and -`%{ISSUE_PATH}` placeholder which will be replaced by project path and the issue iid. +You can use `%{ISSUE_ID}` placeholder which will be replaced by an issue IID in the email and +`%{ISSUE_PATH}` placeholder which will be replaced by project path and the issue IID. As the service desk issues are created as confidential (only project members can see them) -the response email doesn't provide the issue link. +the response email does not provide the issue link. #### New note email The **New note email** is the email sent to a user when the issue they submitted has a new comment. The file name of the template has to be `new_note.md`. -You can use `%{ISSUE_ID}` placeholder which will be replaced by an issue iid +You can use `%{ISSUE_ID}` placeholder which will be replaced by an issue IID in the email, `%{ISSUE_PATH}` placeholder which will be replaced by - project path and the issue iid and `%{NOTE_TEXT}` placeholder which will be replaced by the note text. + project path and the issue IID and `%{NOTE_TEXT}` placeholder which will be replaced by the note text. ### Using custom email display name @@ -115,11 +115,67 @@ in the email, `%{ISSUE_PATH}` placeholder which will be replaced by You can customize the email display name. Emails sent from Service Desk will have this name in the `From` header. The default display name is `GitLab Support Bot`. +### Using custom email address + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/2201) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.0. + +NOTE: **Note:** +This feature is disabled by default. For steps to enable it, see [Enable custom email address](#enable-custom-email-address). + +If the `service_desk_email` feature flag is enabled in your configuration, +then it's possible to create Service Desk issues by sending emails to the +custom Service Desk email address, which should have the following format: +`project_contact+%{key}@example.com`. + +The `%{key}` part is used to find the project where the issue should be created. The +`%{key}` part combines the path to the project and configurable project name suffix: +`<project_full_path>-<project_name_suffix>`. + +You can set the project name suffix in your project's Service Desk settings. +It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). + + + +For example, suppose you add the following to your configuration: + +```yaml +service_desk_email: + enabled: true + address: "project_contact+%{key}@example.com" + user: "project_support@example.com" + password: "[REDACTED]" + host: "imap.gmail.com" + port: 993 + ssl: true + start_tls: false + log_path: "log/mailroom.log" + mailbox: "inbox" + idle_timeout: 60 + expunge_deleted: true +``` + +In this case, suppose the `mygroup/myproject` project Service Desk settings has the project name +suffix set to `support`, and a user sends an email to `project_contact+mygroup-myproject-support@example.com`. +As a result, a new Service Desk issue is created from this email in the `mygroup/myproject` project. + +#### Enable custom email address + +This feature comes with the `service_desk_email` feature flag disabled by default. +To turn on the feature, ask a GitLab administrator with Rails console access to run the following +command: + +```ruby +Feature.enable(service_desk_email) +``` + +The configuration options are the same as for configuring +[incoming email](../../administration/incoming_email.md#set-it-up). + ## Using Service Desk ### As an end user (issue creator) -To create a Service Desk issue, an end user doesn't need to know anything about +To create a Service Desk issue, an end user does not need to know anything about the GitLab instance. They just send an email to the address they are given, and receive an email back confirming receipt: @@ -136,7 +192,7 @@ And any responses they send will be displayed in the issue itself. ### As a responder to the issue -For responders to the issue, everything works as usual. They'll see a familiar looking +For responders to the issue, everything works as usual. They will see a familiar looking issue tracker, where they can see issues created via customer support requests and filter and interact with them just like other GitLab issues. diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 7f241a74820..e9521a0567e 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -12,6 +12,7 @@ See also: - [Project import/export API](../../../api/project_import_export.md) - [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md) **(CORE ONLY)** +- [Group import/export](../../group/settings/import_export.md) - [Group import/export API](../../../api/group_import_export.md) To set up a project import/export: @@ -24,6 +25,8 @@ To set up a project import/export: Note the following: +- Imports from a newer version of GitLab are not supported. + The Importing GitLab version must be greater than or equal to the Exporting GitLab version. - Imports will fail unless the import and export GitLab instances are compatible as described in the [Version history](#version-history). - Exports are stored in a temporary [shared directory](../../../development/shared_files.md) @@ -41,11 +44,24 @@ Note the following: ## Version history -The following table lists updates to Import/Export: +Starting with GitLab 13.0, GitLab can import bundles that were exported from a different GitLab deployment. +This ability is limited to two previous GitLab [minor](../../../policy/maintenance.md#versioning) +releases, which is similar to our process for [Security Releases](../../../policy/maintenance.md#security-releases). + +For example: + +| Current version | Can import bundles exported from | +|-----------------|----------------------------------| +| 13.0 | 13.0, 12.10, 12.9 | +| 13.1 | 13.1, 13.0, 12.10 | + +### 12.x + +Prior to 13.0 this was a defined compatibility table: | Exporting GitLab version | Importing GitLab version | | -------------------------- | -------------------------- | -| 11.7 to current | 11.7 to current | +| 11.7 to 13.0 | 11.7 to 13.0 | | 11.1 to 11.6 | 11.1 to 11.6 | | 10.8 to 11.0 | 10.8 to 11.0 | | 10.4 to 10.7 | 10.4 to 10.7 | @@ -66,6 +82,13 @@ Projects can be exported and imported only between versions of GitLab with match For example, 8.10.3 and 8.11 have the same Import/Export version (0.1.3) and the exports between them will be compatible. +## Between CE and EE + +You can export projects from the [Community Edition to the Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/) and vice versa. +This assumes [version history](#version-history) requirements are met. + +If you're exporting a project from the Enterprise Edition to the Community Edition, you may lose data that is retained only in the Enterprise Edition. For more information, see [downgrading from EE to CE](../../../README.md). + ## Exported contents The following items will be exported: diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index ce94a8f5521..0c98772237b 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -20,6 +20,16 @@ Adjust your project's name, description, avatar, [default branch](../repository/ The project description also partially supports [standard Markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. +#### Compliance framework **(ULTIMATE)** + +You can select a framework label to identify that your project has certain compliance requirements or needs additional oversight. Available labels include: + +- GDPR - General Data Protection Regulation +- HIPAA - Health Insurance Portability and Accountability Act +- PCI-DSS - Payment Card Industry-Data Security Standard +- SOC 2 - Service Organization Control 2 +- SOX - Sarbanes-Oxley + ### Sharing and permissions For your repository, you can set up features such as public access, repository features, @@ -46,18 +56,19 @@ Use the switches to enable or disable the following features: | **Forks** | ✓ | Enables [forking](../index.md#fork-a-project) functionality | | **Pipelines** | ✓ | Enables [CI/CD](../../../ci/README.md) functionality | | **Container Registry** | | Activates a [registry](../../packages/container_registry/) for your docker images | -| **Git Large File Storage** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-lfs) | +| **Git Large File Storage** | | Enables the use of [large files](../../../topics/git/lfs/index.md#git-large-file-storage-lfs) | | **Packages** | | Supports configuration of a [package registry](../../../administration/packages/index.md#gitlab-package-registry-administration-premium-only) functionality | | **Wiki** | ✓ | Enables a separate system for [documentation](../wiki/) | | **Snippets** | ✓ | Enables [sharing of code and text](../../snippets.md) | | **Pages** | ✓ | Allows you to [publish static websites](../pages/) | +| **Metrics Dashboard** | ✓ | Control access to [metrics dashboard](../integrations/prometheus.md) Some features depend on others: - If you disable the **Issues** option, GitLab also removes the following features: - **Issue Boards** - - [**Service Desk**](#service-desk-premium) **(PREMIUM)** + - [**Service Desk**](#service-desk-starter) **(STARTER)** NOTE: **Note:** When the **Issues** option is disabled, you can still access **Milestones** @@ -70,13 +81,15 @@ Some features depend on others: - If you disable **Repository** functionality, GitLab also disables the following features for your project: - - **Merge Requests** - **Pipelines** - **Container Registry** - **Git Large File Storage** - **Packages** +- Metrics dashboard access requires reading both project environments and deployments. + Users with access to the metrics dashboard can also access environments and deployments. + #### Disabling email notifications Project owners can disable all [email notifications](../../profile/notifications.md#gitlab-notification-emails) @@ -96,7 +109,7 @@ Set up your project's merge request settings:  -### Service Desk **(PREMIUM)** +### Service Desk **(STARTER)** Enable [Service Desk](../service_desk.md) for your project to offer customer support. @@ -247,7 +260,7 @@ To do so: 1. Confirm the action by typing the project's path as instructed. NOTE: **Note:** -Only project maintainers have the [permissions](../../permissions.md#project-members-permissions) +Only project owners have the [permissions](../../permissions.md#project-members-permissions) to remove a fork relationship. ## Operations settings diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md new file mode 100644 index 00000000000..303a6f6d3be --- /dev/null +++ b/doc/user/project/settings/project_access_tokens.md @@ -0,0 +1,55 @@ +# Project access tokens **(CORE ONLY)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2587) in GitLab 13.0. + +Project access tokens are scoped to a project and can be used to authenticate with the [GitLab API](../../../api/README.md#personalproject-access-tokens). + +You can also use project access tokens with Git to authenticate over HTTP or SSH. + +Project access tokens expire on the date you define, at midnight UTC. + +For examples of how you can use a project access token to authenticate with the API, see the following section from our [API Docs](../../../api/README.md#personalproject-access-tokens). + +## Creating a project access token + +1. Log in to GitLab. +1. Navigate to the project you would like to create an access token for. +1. In the **{settings}** **Settings** menu choose **Access Tokens**. +1. Choose a name and optional expiry date for the token. +1. Choose the [desired scopes](#limiting-scopes-of-a-project-access-token). +1. Click the **Create project access token** button. +1. Save the project access token somewhere safe. Once you leave or refresh + the page, you won't be able to access it again. + +## Project bot users + +For each project access token created, a bot user will also be created and added to the project with +["Maintainer" level permissions](../../permissions.md#project-members-permissions). API calls made with a +project access token will be associated to the corresponding bot user. + +These users will appear in **{settings}** **Settings > Members** but can not be modified. +Furthermore, the bot user can not be added to any other project. + +When the project access token is [revoked](#revoking-a-project-access-token) the bot user will be deleted and all +records will be moved to a system-wide user with the username "Ghost User". For more information, +see [Associated Records](../../profile/account/delete_account.md#associated-records). + +## Revoking a project access token + +At any time, you can revoke any project access token by clicking the +respective **Revoke** button in **{settings}** **Settings > Access Tokens**. + +## Limiting scopes of a project access token + +Project access tokens can be created with one or more scopes that allow various +actions that a given token can perform. The available scopes are depicted in +the following table. + +| Scope | Description | +| ------------------ | ----------- | +| `api` | Grants complete read/write access to the scoped project API. | +| `read_api` | Grants read access to the scoped project API. | +| `read_registry` | Allows read-access (pull) to [container registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | +| `write_registry` | Allows write-access (push) to [container registry](../../packages/container_registry/index.md). | +| `read_repository` | Allows read-only access (pull) to the repository. | +| `write_repository` | Allows read-write access (pull, push) to the repository. | diff --git a/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png b/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png Binary files differdeleted file mode 100644 index 130dff9f30d..00000000000 --- a/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png +++ /dev/null diff --git a/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_0.png b/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_0.png Binary files differnew file mode 100644 index 00000000000..a2f5248bf39 --- /dev/null +++ b/doc/user/project/static_site_editor/img/wysiwyg_editor_v13_0.png diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index f186ff9919e..e2e498b605a 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -5,7 +5,14 @@ description: "The static site editor enables users to edit content on static web # Static Site Editor -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. +> - WYSIWYG editor [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214559) in GitLab 13.0. + +DANGER: **Danger:** +In GitLab 13.0, we [introduced breaking changes](https://gitlab.com/gitlab-org/gitlab/-/issues/213282) +to the URL structure of the Static Site Editor. Follow the instructions in this +[snippet](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman/snippets/1976539) +to update your project with the latest changes. Static Site Editor enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture, or @@ -44,7 +51,7 @@ When clicking it, GitLab will open up an editor window from which the content can be directly edited. When you're ready, you can submit your changes in a click of a button: - + When an editor submits their changes, in the background, GitLab automatically creates a new branch, commits their changes, and opens a merge request. The @@ -79,7 +86,8 @@ company and a new feature has been added to the company product. 1. You are assigned the task of updating the documentation. 1. You visit a page and see content that needs to be edited. 1. Click the **Edit this page** button on the production site. -1. The file is opened in the Static Site Editor. +1. The file is opened in the Static Site Editor in **WYSIWYG** mode. If you wish to edit the raw Markdown + instead, you can toggle the **Markdown** mode in the bottom-right corner. 1. You edit the file right there and click **Submit changes**. 1. A new merge request is automatically created and you assign it to your colleague for review. diff --git a/doc/user/project/status_page/index.md b/doc/user/project/status_page/index.md index d292ca94ba9..8ebfb638894 100644 --- a/doc/user/project/status_page/index.md +++ b/doc/user/project/status_page/index.md @@ -1,6 +1,12 @@ -# GitLab Status Page +--- +stage: Monitor +group: Health +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2479) in GitLab 12.10. +# GitLab Status Page **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2479) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. GitLab Status Page allows you to create and deploy a static website to communicate efficiently to users during an incident. @@ -35,15 +41,15 @@ To use GitLab Status Page you first need to set up your account details for your ### Status Page project -To deploy the status page to AWS S3 you need to add the Status Page project & configure the necessary CI variables. +To deploy the Status Page to AWS S3 you need to add the Status Page project & configure the necessary CI variables. 1. Fork the [Status Page](https://gitlab.com/gitlab-org/status-page) project. This can also be done via [Repository Mirroring](https://gitlab.com/gitlab-org/status-page#repository-mirroring) which will ensure you get the up-to-date Status Page features. 1. Add the following variables in **Settings > CI/CD > Variables**. (To get these variables from Amazon, use your Amazon Console): - - `S3_BUCKET_NAME` - name of the Amazon S3 bucket + - `S3_BUCKET_NAME` - name of the Amazon S3 bucket (If a bucket with the provided name doesn't exist, the first pipeline run will create one and configure it for [static website hosting](https://docs.aws.amazon.com/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html)) - `AWS_DEFAULT_REGION` - the AWS region - `AWS_ACCESS_KEY_ID` - the AWS access key ID - `AWS_SECRET_ACCESS_KEY` - the AWS secret -1. Run the pipeline to deploy the status page to S3. +1. Run the pipeline to deploy the Status Page to S3. ### Syncing incidents to the Status Page @@ -55,7 +61,7 @@ Once the CI/CD variables are set, you'll need to set up the Project you want to ## Status Page UI -The Status page landing page shows you an overview of the recent incidents. Clicking on an incident will take you to the incident's detail page. +The Status Page landing page shows you an overview of the recent incidents. Clicking on an incident will take you to the incident's detail page.  @@ -64,8 +70,8 @@ The Status page landing page shows you an overview of the recent incidents. Clic The incident detail page shows detailed information about a particular incident. For example: - Status on the incident, including when the incident was last updated. -- The incident title. -- The description of the incident. +- The incident title, including any emojis. +- The description of the incident, including emojis and static images. - A chronological ordered list of updates to the incident.  @@ -76,7 +82,9 @@ The incident detail page shows detailed information about a particular incident. To publish an Incident, you first need to create an issue in the Project you enabled the Status Page settings in. -Once this issue is created, a background worker will publish the issue onto the status page using the credentials you provided during setup. +Once this issue is created, a background worker will publish the issue onto the Status Page using the credentials you provided during setup. +Since all incidents are published publicly, user and group mentions are anonymized with `Incident Responder`, +and titles of non-public [GitLab references](../../markdown.md#special-gitlab-references) are removed. NOTE: **Note:** Confidential issues are not published. If a published issue is made confidential it will be unpublished. @@ -99,4 +107,4 @@ Anyone with access to view the Issue can add an Emoji Award to a comment, so you ### Changing the Incident status -To change the incident status from `open` to `closed`, close the incident issue within GitLab. This will then be updated shortly on the Status page website. +To change the incident status from `open` to `closed`, close the incident issue within GitLab. This will then be updated shortly on the Status Page website. diff --git a/doc/user/project/web_ide/img/admin_clientside_evaluation.png b/doc/user/project/web_ide/img/admin_clientside_evaluation.png Binary files differdeleted file mode 100644 index a930490398b..00000000000 --- a/doc/user/project/web_ide/img/admin_clientside_evaluation.png +++ /dev/null diff --git a/doc/user/project/web_ide/img/admin_live_preview_v13_0.png b/doc/user/project/web_ide/img/admin_live_preview_v13_0.png Binary files differnew file mode 100644 index 00000000000..90129d240bc --- /dev/null +++ b/doc/user/project/web_ide/img/admin_live_preview_v13_0.png diff --git a/doc/user/project/web_ide/img/dark_theme_v13.0.png b/doc/user/project/web_ide/img/dark_theme_v13.0.png Binary files differnew file mode 100644 index 00000000000..6034bc52c05 --- /dev/null +++ b/doc/user/project/web_ide/img/dark_theme_v13.0.png diff --git a/doc/user/project/web_ide/img/clientside_evaluation.png b/doc/user/project/web_ide/img/live_preview_v13_0.png Binary files differindex bd04d3d644b..bd04d3d644b 100644 --- a/doc/user/project/web_ide/img/clientside_evaluation.png +++ b/doc/user/project/web_ide/img/live_preview_v13_0.png diff --git a/doc/user/project/web_ide/img/solarized_light_theme_v13.0.png b/doc/user/project/web_ide/img/solarized_light_theme_v13.0.png Binary files differnew file mode 100644 index 00000000000..f3c4aa142a4 --- /dev/null +++ b/doc/user/project/web_ide/img/solarized_light_theme_v13.0.png diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index c98448ff904..d4daca0e1e4 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -1,6 +1,6 @@ # Web IDE -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4539) in [GitLab Ultimate][ee] 10.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4539) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. > - [Brought to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/issues/44157) in 10.7. The Web IDE editor makes it faster and easier to contribute changes to your @@ -15,7 +15,7 @@ and from merge requests. ## File finder -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18323) in [GitLab Core][ce] 10.8. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18323) in [GitLab Core](https://about.gitlab.com/pricing/) 10.8. The file finder allows you to quickly open files in the current branch by searching. The file finder is launched using the keyboard shortcut `Command-p`, @@ -43,6 +43,20 @@ you can find a more complete list of supported languages in the NOTE: **Note:** Single file editing is based on the [Ace Editor](https://ace.c9.io). +### Themes + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2389) in GitLab 13.0. + +All the themes GitLab supports for syntax highlighting are added to the Web IDE's code editor. +You can pick a theme from your [profile preferences](../../profile/preferences.md). + +The themes are available only in the Web IDE file editor, except for the [dark theme](https://gitlab.com/gitlab-org/gitlab/-/issues/209808), +which applies to the entire Web IDE screen. + +| Solarized Light Theme | Dark Theme | +|---------------------------------------------------------------|-----------------------------------------| +|  |  | + ## Commit changes > - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/4539) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4 and [brought to GitLab Core](https://gitlab.com/gitlab-org/gitlab-foss/issues/44157) in 10.7. @@ -73,7 +87,7 @@ shows you a preview of the merge request diff if you commit your changes. ## View CI job logs -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19279) in [GitLab Core][ce] 11.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19279) in [GitLab Core](https://about.gitlab.com/pricing/) 11.0. You can use the Web IDE to quickly fix failing tests by opening the branch or merge request in the Web IDE and opening the logs of the failed @@ -86,7 +100,7 @@ left. ## Switching merge requests -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19318) in [GitLab Core][ce] 11.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19318) in [GitLab Core](https://about.gitlab.com/pricing/) 11.0. To switch between your authored and assigned merge requests, click the dropdown in the top of the sidebar to open a list of merge requests. You will @@ -95,34 +109,35 @@ request. ## Switching branches -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20850) in [GitLab Core][ce] 11.2. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20850) in [GitLab Core](https://about.gitlab.com/pricing/) 11.2. To switch between branches of the current project repository, click the dropdown in the top of the sidebar to open a list of branches. You will need to commit or discard all your changes before switching to a different branch. -## Client Side Evaluation +## Live Preview -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19764) in [GitLab Core][ce] 11.2. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/19764) in [GitLab Core](https://about.gitlab.com/pricing/) 11.2. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/213853) from _Client Side Evaluation_ to _Live Preview_ in GitLab 13.0. You can use the Web IDE to preview JavaScript projects right in the browser. This feature uses CodeSandbox to compile and bundle the JavaScript used to preview the web application. - + Additionally, for public projects an **Open in CodeSandbox** button is available to transfer the contents of the project into a public CodeSandbox project to quickly share your project with others. -### Enabling Client Side Evaluation +### Enabling Live Preview -The Client Side Evaluation feature needs to be enabled in the GitLab instances -admin settings. Client Side Evaluation is enabled for all projects on +The Live Preview feature needs to be enabled in the GitLab instances +admin settings. Live Preview is enabled for all projects on GitLab.com - + Once you have done that, you can preview projects with a `package.json` file and a `main` entry point inside the Web IDE. An example `package.json` is shown @@ -302,6 +317,3 @@ active terminal at a time. - If the terminal displays **Connection Failure**, then the terminal could not connect to the runner. Please try to stop and restart the terminal. If the problem persists, double check your runner configuration. - -[ce]: https://about.gitlab.com/pricing/ -[ee]: https://about.gitlab.com/pricing/ diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 88b729272ec..fa3ad4536ef 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -126,7 +126,7 @@ found. The list is ordered alphabetically.  If you have many pages, not all will be listed in the sidebar. Click on -**More pages** to see all of them. +**View All Pages** to see all of them. ## Viewing the history of a wiki page @@ -189,7 +189,24 @@ instructions. On the project's Wiki page, there is a right side navigation that renders the full Wiki pages list by default, with hierarchy. -If the Wiki repository contains a `_sidebar` page, the file of this page replaces the default side navigation. -This custom file serves to render it's custom content, fully replacing the standard sidebar. +To customize the sidebar, you can create a file named `_sidebar` to fully replace the default navigation. + +CAUTION: **Warning:** +Unless you link the `_sidebar` file from your custom nav, to edit it you'll have to access it directly +from the browser's address bar by typing: `https://gitlab.com/<namespace>/<project_name>/-/wikis/_sidebar` (for self-managed GitLab instances, replace `gitlab.com` with your instance's URL). + +Example for `_sidebar` (using Markdown format): + +```markdown +### [Home](home) + +- [Hello World](hello) +- [Foo](foo) +- [Bar](bar) + +--- + +- [Sidebar](_sidebar) +``` Support for displaying a generated TOC with a custom side navigation is planned. diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index f38cf8f139e..fac1702f2ac 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -1,15 +1,17 @@ # Advanced Global Search **(STARTER ONLY)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/109) in GitLab [Starter](https://about.gitlab.com/pricing/) 8.4. -> - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** -Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). +Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. +[Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). Leverage Elasticsearch for faster, more advanced code search across your entire GitLab instance. +This is the user documentation. To install and configure Elasticsearch, +visit the [administrator documentation](../../integration/elasticsearch.md). + ## Overview The Advanced Global Search in GitLab is a powerful search service that saves diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index ebb957ad8e2..5113578af9e 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -3,14 +3,16 @@ > **Notes:** > > - Introduced in [GitLab Enterprise Starter](https://about.gitlab.com/pricing/) 9.2 -> - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** -Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). +Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. +[Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). Use advanced queries for more targeted search results. +This is the user documentation. To install and configure Elasticsearch, +visit the [administrator documentation](../../integration/elasticsearch.md). + ## Overview The Advanced Syntax Search is a subset of the diff --git a/doc/user/search/img/filter_approved_by_merge_requests_v13_0.png b/doc/user/search/img/filter_approved_by_merge_requests_v13_0.png Binary files differnew file mode 100644 index 00000000000..ef4c65aea4b --- /dev/null +++ b/doc/user/search/img/filter_approved_by_merge_requests_v13_0.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index ed2a4b36372..f5efa3519d9 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -41,7 +41,10 @@ groups: - [Label](../project/labels.md) - My-reaction - Confidential - - Epic ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/195704) in GitLab 12.9) + - Epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in GitLab 12.9), + including [child epic](../group/epics/index.md#multi-level-child-epics-ultimate) + ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in + [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0) - Search for this text 1. Select or type the operator to use for filtering the attribute. The following operators are available: @@ -94,10 +97,19 @@ You can filter the **Issues** list to individual instances by their ID. For exam > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/9468) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.9. To filter merge requests by an individual approver, you can type (or select from -the dropdown) `approver` and select the user. +the dropdown) **Approver** and select the user.  +### Filtering merge requests by "approved by" **(STARTER)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30335) in [GitLab Starter](https://about.gitlab.com/pricing/) 13.0. + +To filter merge requests already approved by a specific individual, you can type (or select from +the dropdown) **Approved-By** and select the user. + + + ## Search history You can view recent searches by clicking on the little arrow-clock icon, which is to the left of the search input. Click the search entry to run that search again. This feature is available for issues and merge requests. Searches are stored locally in your browser. diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 0372bb42038..96c8dba11e5 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -2,7 +2,7 @@ With GitLab Snippets you can store and share bits of code and text with other users. - + Snippets can be maintained using [snippets API](../api/snippets.md). @@ -49,6 +49,67 @@ CAUTION: **Warning:** Make sure to add the file name to get code highlighting and to avoid this [copy-pasting bug](https://gitlab.com/gitlab-org/gitlab/-/issues/22870). +## Versioned Snippets + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/239) in GitLab 13.0. + +Starting in 13.0, snippets (both personal and project snippets) +have version control enabled by default. + +This means that all snippets get their own underlying repository initialized with +a `master` branch at the moment the snippet is created. Whenever a change to the snippet is saved, a +new commit to the master branch is recorded. Commit messages are automatically +generated. The snippet's repository has only one branch (master) by default, deleting +it or creating other branches is not supported. + +Existing snippets will be automatically migrated in 13.0. Their current +content will be saved as the initial commit to the snippets' repository. + +### File names + +Snippets support syntax highlighting based on the filename and +extension provided for them. While it is possible to submit a snippet +without specifying a filename and extension, it needs a valid name so the +content can be created as a file in the snippet's repository. + +In case the user does not attribute a filename and extension to a snippet, +GitLab automatically adds a filename in the format `snippetfile<x>.txt` +where `<x>` represents a number added to the file, starting with 1. This +number increases incrementally when more snippets without an attributed +filename are added. + +When upgrading from an earlier version of GitLab to 13.0, existing snippets +without a supported filename will be renamed to a compatible format. For +example, if the snippet's filename is `http://a-weird-filename.me` it will +be changed to `http-a-weird-filename-me` to be included in the snippet's +repository. As snippets are stored by ID, changing their filenames will not break +direct or embedded links to the snippet. + +### Cloning snippets + +Snippets can be cloned as a regular Git repository using SSH or HTTPS. Click the **Clone** +button above the snippet content to copy the URL of your choice. + + + +This allows you to have a local copy of the snippet's repository and make +changes as needed. You can commit those changes and push them to the remote +master branch. + +### Limitations + +- Binary files are not supported. +- Creating or deleting branches is not supported. Only a default *master*. +branch is used. +- Git tags are not supported in snippet repositories. +- Snippets' repositories are limited to one file. Attempting to push more +than one file will result in an error. +- Revisions are not *yet* visible to the user on the GitLab UI, but +it's planned to be added in future iterations. See the [revisions tab issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39271) +for updates. +- The [maximum size for a snippet](../administration/snippets/index.md#snippets-content-size-limit) +is 50 MB, by default. + ## Discover snippets There are two main ways of how you can discover snippets in GitLab. @@ -87,7 +148,7 @@ snippet was created using the GitLab web interface the original line ending is W > Introduced in GitLab 10.8. Public snippets can not only be shared, but also embedded on any website. This -allows to reuse a GitLab snippet in multiple places and any change to the source +allows us to reuse a GitLab snippet in multiple places and any change to the source is automatically reflected in the embedded snippet. To embed a snippet, first make sure that: @@ -111,6 +172,6 @@ Here's how an embedded snippet looks like: <script src="https://gitlab.com/gitlab-org/gitlab-foss/snippets/1717978.js"></script> -Embedded snippets are displayed with a header that shows the file name if defined, +Embedded snippets are displayed with a header that shows the file name is defined, the snippet size, a link to GitLab, and the actual snippet content. Actions in the header allow users to see the snippet in raw format and download it. diff --git a/doc/user/todos.md b/doc/user/todos.md index d19f1b8f14b..84a7b6afdac 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -38,7 +38,7 @@ A To Do appears on your To-Do List when: - Epic **(ULTIMATE)** - You are `@mentioned` in a comment on a: - Commit - - Design **(PREMIUM)** + - Design - A job in the CI pipeline running for your merge request failed, but this job is not allowed to fail - An open merge request becomes unmergeable due to conflict, and you are either: @@ -137,7 +137,7 @@ There are four kinds of filters you can use on your To-Do List. | Project | Filter by project | | Group | Filter by group | | Author | Filter by the author that triggered the To Do | -| Type | Filter by issue, merge request, or epic **(ULTIMATE)** | +| Type | Filter by issue, merge request, design, or epic **(ULTIMATE)** | | Action | Filter by the action that triggered the To Do | You can also filter by more than one of these at the same time. The possible Actions are |
