diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-20 18:38:24 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-04-20 18:38:24 +0000 |
| commit | 983a0bba5d2a042c4a3bbb22432ec192c7501d82 (patch) | |
| tree | b153cd387c14ba23bd5a07514c7c01fddf6a78a0 /doc | |
| parent | a2bddee2cdb38673df0e004d5b32d9f77797de64 (diff) | |
| download | gitlab-ce-983a0bba5d2a042c4a3bbb22432ec192c7501d82.tar.gz | |
Add latest changes from gitlab-org/gitlab@12-10-stable-ee
Diffstat (limited to 'doc')
125 files changed, 2457 insertions, 312 deletions
diff --git a/doc/README.md b/doc/README.md index 6b863436ce2..8437e7f798e 100644 --- a/doc/README.md +++ b/doc/README.md @@ -156,6 +156,7 @@ The following documentation relates to the DevOps **Create** stage: | [Search through GitLab](user/search/index.md) | Search for issues, merge requests, projects, groups, and todos. | | [Snippets](user/snippets.md) | Snippets allow you to create little bits of code. | | [Web IDE](user/project/web_ide/index.md) | Edit files within GitLab's user interface. | +| [Static Site Editor](user/project/static_site_editor/index.md) | Edit content on static websites. | | [Wikis](user/project/wiki/index.md) | Enhance your repository documentation with built-in wikis. | <div align="right"> @@ -367,7 +368,7 @@ The following documentation relates to the DevOps **Secure** stage: | [Group Security Dashboard](user/application_security/security_dashboard/index.md#group-security-dashboard) **(ULTIMATE)** | View vulnerabilities in all the projects in a group and its subgroups. | | [Instance Security Dashboard](user/application_security/security_dashboard/index.md#instance-security-dashboard) **(ULTIMATE)** | View vulnerabilities in all the projects you're interested in. | | [License Compliance](user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project's dependencies for their licenses. | -| [Pipeline Security Dashboard](user/application_security/security_dashboard/index.md#pipeline-security-dashboard) **(ULTIMATE)** | View the security reports for your project's pipelines. | +| [Pipeline Security](user/application_security/security_dashboard/index.md#pipeline-security) **(ULTIMATE)** | View the security reports for your project's pipelines. | | [Project Security Dashboard](user/application_security/security_dashboard/index.md#project-security-dashboard) **(ULTIMATE)** | View the latest security reports for your project. | | [Static Application Security Testing (SAST)](user/application_security/sast/index.md) **(ULTIMATE)** | Analyze source code for known vulnerabilities. | diff --git a/doc/administration/file_hooks.md b/doc/administration/file_hooks.md index 7d935af8e37..21ade36a2a5 100644 --- a/doc/administration/file_hooks.md +++ b/doc/administration/file_hooks.md @@ -23,22 +23,22 @@ see the [system hooks] documentation. ## Setup -The file hooks must be placed directly into the `plugins` directory, subdirectories +The file hooks must be placed directly into the `file_hooks` directory, subdirectories will be ignored. There is an -[`example` directory inside `plugins`](https://gitlab.com/gitlab-org/gitlab/tree/master/plugins/examples) +[`example` directory inside `file_hooks`](https://gitlab.com/gitlab-org/gitlab/tree/master/file_hooks/examples) where you can find some basic examples. Follow the steps below to set up a custom hook: 1. On the GitLab server, navigate to the plugin directory. For an installation from source the path is usually - `/home/git/gitlab/plugins/`. For Omnibus installs the path is - usually `/opt/gitlab/embedded/service/gitlab-rails/plugins`. + `/home/git/gitlab/file_hooks/`. For Omnibus installs the path is + usually `/opt/gitlab/embedded/service/gitlab-rails/file_hooks`. For [highly available] configurations, your hook file should exist on each application server. -1. Inside the `plugins` directory, create a file with a name of your choice, +1. Inside the `file_hooks` directory, create a file with a name of your choice, without spaces or special characters. 1. Make the hook file executable and make sure it's owned by the Git user. 1. Write the code to make the file hook function as expected. That can be @@ -106,9 +106,9 @@ bundle exec rake file_hooks:validate RAILS_ENV=production Example of output: ```plaintext -Validating file hooks from /plugins directory -* /home/git/gitlab/plugins/save_to_file.clj succeed (zero exit code) -* /home/git/gitlab/plugins/save_to_file.rb failure (non-zero exit code) +Validating file hooks from /file_hooks directory +* /home/git/gitlab/file_hooks/save_to_file.clj succeed (zero exit code) +* /home/git/gitlab/file_hooks/save_to_file.rb failure (non-zero exit code) ``` [system hooks]: ../system_hooks/system_hooks.md diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index 0a5c39665f4..6f417f955ac 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -248,6 +248,10 @@ The data can be removed with the following command: sudo rm -rf /var/opt/gitlab/geo-postgresql ``` +If you have any `geo_secondary[]` configuration options enabled in your `gitlab.rb` +file, these can be safely commented out or removed, and then [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) +for the changes to take effect. + ## Promoting secondary Geo replica in multi-secondary configurations If you have more than one **secondary** node and you need to promote one of them, we suggest you follow diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index a1f511fe2a5..3431df3ed1f 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -145,6 +145,7 @@ successfully, you must replicate their data using some other means. | [Maven Repository](../../../user/packages/maven_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | | | [Conan Repository](../../../user/packages/conan_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | | | [NuGet Repository](../../../user/packages/nuget_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | | +| [PyPi Repository](../../../user/packages/pypi_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2554) | No | | | [External merge request diffs](../../merge_request_diffs.md) | [No](https://gitlab.com/gitlab-org/gitlab/issues/33817) | No | | | Content in object storage | **Yes** | No | | diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index fd14c100ffb..e305718580e 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -45,13 +45,17 @@ Skip to the [Configure secondary application node](#configure-secondary-applicat The [geo_primary_role](https://docs.gitlab.com/omnibus/roles/#gitlab-geo-roles) configures the **primary** node's database to be replicated by making changes to `pg_hba.conf` and `postgresql.conf`. Make the following configuration changes -manually to your external database configuration: +manually to your external database configuration and ensure that you restart PostgreSQL +afterwards for the changes to take effect: ```plaintext ## ## Geo Primary Role ## - pg_hba.conf ## +host all all <trusted primary IP>/32 md5 +host replication gitlab_replicator <trusted primary IP>/32 md5 +host all all <trusted secondary IP>/32 md5 host replication gitlab_replicator <trusted secondary IP>/32 md5 ``` @@ -60,7 +64,6 @@ host replication gitlab_replicator <trusted secondary IP>/32 md5 ## Geo Primary Role ## - postgresql.conf ## -sql_replication_user = gitlab_replicator wal_level = hot_standby max_wal_senders = 10 wal_keep_segments = 50 @@ -72,8 +75,19 @@ hot_standby = on ### Manually configure the replica database -Make the following configuration changes manually to your `postgresql.conf` -of external replica database: +Make the following configuration changes manually to your `pg_hba.conf` and `postgresql.conf` +of your external replica database and ensure that you restart PostgreSQL afterwards +for the changes to take effect: + +```plaintext +## +## Geo Secondary Role +## - pg_hba.conf +## +host all all <trusted secondary IP>/32 md5 +host replication gitlab_replicator <trusted secondary IP>/32 md5 +host all all <trusted primary IP>/24 md5 +``` ```plaintext ## @@ -145,6 +159,19 @@ outbound rules do not apply to RDS PostgreSQL databases. Therefore, you need to rule to the read-replica's security group allowing any TCP traffic from the tracking database on port 5432. +1. Ensure that your secondary node can communicate with your tracking database by + manually changing the `pg_hba.conf` that is associated with your tracking database. + Remember to restart PostgreSQL afterwards for the changes to take effect: + + ```plaintext + ## + ## Geo Tracking Database Role + ## - pg_hba.conf + ## + host all all <trusted tracking IP>/32 md5 + host all all <trusted secondary IP>/32 md5 + ``` + 1. SSH into a GitLab **secondary** server and login as root: ```shell diff --git a/doc/administration/geo/replication/img/geo_node_dashboard.png b/doc/administration/geo/replication/img/geo_node_dashboard.png Binary files differindex 0d3dc5895af..53ca4767c5b 100644 --- a/doc/administration/geo/replication/img/geo_node_dashboard.png +++ b/doc/administration/geo/replication/img/geo_node_dashboard.png diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index d1d0c358dc6..19a69dc348c 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -300,6 +300,60 @@ application server, or a Gitaly node. edit `/etc/gitlab/gitlab.rb`, remember to run `sudo gitlab-ctl reconfigure` again before trying the `sql-ping` command. +#### Automatic failover + +When automatic failover is enabled, Praefect will do automatic detection of the health of internal Gitaly nodes. If the +primary has a certain amount of health checks fail, it will decide to promote one of the secondaries to be primary, and +demote the primary to be a secondary. + +1. To enable automatic failover, edit `/etc/gitlab/gitlab.rb`: + + ```ruby + # failover_enabled turns on automatic failover + praefect['failover_enabled'] = true + praefect['virtual_storages'] = { + 'praefect' => { + 'gitaly-1' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN', + 'primary' => true + }, + 'gitaly-2' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + }, + 'gitaly-3' => { + 'address' => 'tcp://GITALY_HOST:8075', + 'token' => 'PRAEFECT_INTERNAL_TOKEN' + } + } + } + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + +Below is the picture when Praefect starts up with the config.toml above: + +```mermaid +graph TD + A[Praefect] -->|Mutator RPC| B(internal_storage_0) + B --> |Replication|C[internal_storage_1] +``` + +Let's say suddenly `internal_storage_0` goes down. Praefect will detect this and +automatically switch over to `internal_storage_1`, and `internal_storage_0` will serve as a secondary: + +```mermaid +graph TD + A[Praefect] -->|Mutator RPC| B(internal_storage_1) + B --> |Replication|C[internal_storage_0] +``` + +NOTE: **Note:**: Currently this feature is supported for setups that only have 1 Praefect instance. Praefect instances running, +for example behind a load balancer, `failover_enabled` should be disabled. The reason is The reason is because there +is no coordination that currently happens across different Praefect instances, so there could be a situation where +two Praefect instances think two different Gitaly nodes are the primary. + ### Gitaly NOTE: **Note:** Complete these steps for **each** Gitaly node. @@ -697,6 +751,31 @@ during a failover. Follow issue It is likely that we will implement support for Consul, and a cloud native strategy in the future. +## Identifying Impact of a Primary Node Failure + +When a primary Gitaly node fails, there is a chance of dataloss. Dataloss can occur if there were outstanding replication jobs the secondaries did not manage to process before the failure. The Praefect `dataloss` subcommand helps identify these cases by counting the number of dead replication jobs for each repository within a given timeframe. + +```shell +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dataloss -from <rfc3339-time> -to <rfc3339-time> +``` + +If the timeframe is not specified, dead replication jobs from the last six hours are counted: + +```shell +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dataloss + +Failed replication jobs between [2020-01-02 00:00:00 +0000 UTC, 2020-01-02 06:00:00 +0000 UTC): +example/repository-1: 1 jobs +example/repository-2: 4 jobs +example/repository-3: 2 jobs +``` + +To specify a timeframe in UTC, run: + +```shell +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dataloss -from 2020-01-02T00:00:00+00:00 -to 2020-01-02T00:02:00+00:00 +``` + ## Backend Node Recovery When a Praefect backend node fails and is no longer able to diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index a1e5482e5dc..1e19e7e6c01 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -8,10 +8,15 @@ type: reference The following are the requirements for providing your own Redis instance: -- GitLab 12.0 and later requires Redis version 3.2 or higher. Version 3.2 or higher is recommend as this is - what ships with the GitLab Omnibus package. Older Redis versions do not - support an optional count argument to SPOP which is now required for - [Merge Trains](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). +- Redis version 5.0 or higher is recommended, as this is what ships with + Omnibus GitLab packages starting with GitLab 12.7. +- Support for Redis 3.2 is deprecated with GitLab 12.10 and will be completely + removed in GitLab 13.0. +- GitLab 12.0 and later requires Redis version 3.2 or higher. Older Redis + versions do not support an optional count argument to SPOP which is now + required for [Merge Trains](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md). +- In addition, if Redis 4 or later is available, GitLab makes use of certain + commands like `UNLINK` and `USAGE` which were introduced only in Redis 4. - Standalone Redis or Redis high availability with Sentinel are supported. Redis Cluster is not supported. - Managed Redis from cloud providers such as AWS ElastiCache will work. If these diff --git a/doc/administration/logs.md b/doc/administration/logs.md index c43406fb647..e4a3514a539 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -30,17 +30,21 @@ Line breaks have been added to this example for legibility: "controller":"Projects::IssuesController", "action":"show", "status":200, - "duration":229.03, - "view":174.07, - "db":13.24, "time":"2017-08-08T20:15:54.821Z", "params":[{"key":"param_key","value":"param_value"}], "remote_ip":"18.245.0.1", "user_id":1, "username":"admin", - "gitaly_calls":76, - "gitaly_duration":7.41, - "queue_duration": 112.47 + "queue_duration_s":0.0, + "gitaly_calls":16, + "gitaly_duration_s":0.16, + "redis_calls":115, + "redis_duration_s":0.13, + "correlation_id":"O1SdybnnIq7", + "cpu_s":17.50, + "db_duration_s":0.08, + "view_duration_s":2.39, + "duration_s":20.54 } ``` @@ -48,12 +52,15 @@ This example was a GET request for a specific issue. Each line also contains performance data, with times in milliseconds: -1. `duration`: total time taken to retrieve the request -1. `queue_duration`: total time that the request was queued inside GitLab Workhorse -1. `view`: total time taken inside the Rails views -1. `db`: total time to retrieve data from the database +1. `duration_s`: total time taken to retrieve the request +1. `queue_duration_s`: total time that the request was queued inside GitLab Workhorse +1. `view_duration_s`: total time taken inside the Rails views +1. `db_duration_s`: total time to retrieve data from PostgreSQL +1. `redis_duration_s`: total time to retrieve data from Redis +1. `cpu_s`: total time spent on CPU +1. `gitaly_duration_s`: total time taken by Gitaly calls 1. `gitaly_calls`: total number of calls made to Gitaly -1. `gitaly_duration`: total time taken by Gitaly calls +1. `redis_calls`: total number of calls made to Redis User clone and fetch activity using HTTP transport appears in this log as `action: git_upload_pack`. @@ -73,9 +80,6 @@ NOTE: **Note:** Starting with GitLab 12.5, if an error occurs, an "controller": "Admin::DashboardController", "action": "index", "status": 500, - "duration": 2584.11, - "view": 0, - "db": 9.21, "time": "2019-11-14T13:12:46.156Z", "params": [], "remote_ip": "127.0.0.1", @@ -84,7 +88,16 @@ NOTE: **Note:** Starting with GitLab 12.5, if an error occurs, an "ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:70.0) Gecko/20100101 Firefox/70.0", "queue_duration": 274.35, "correlation_id": "KjDVUhNvvV3", - "cpu_s": 2.837645135999999, + "queue_duration_s":0.0, + "gitaly_calls":16, + "gitaly_duration_s":0.16, + "redis_calls":115, + "redis_duration_s":0.13, + "correlation_id":"O1SdybnnIq7", + "cpu_s":17.50, + "db_duration_s":0.08, + "view_duration_s":2.39, + "duration_s":20.54 "exception.class": "NameError", "exception.message": "undefined local variable or method `adsf' for #<Admin::DashboardController:0x00007ff3c9648588>", "exception.backtrace": [ diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index d14726d33de..f826741d66f 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -8,6 +8,7 @@ The Packages feature allows GitLab to act as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | +| [PyPi Repository](../../user/packages/pypi_repository/index.md) | The GitLab PyPi Repository enables every project in GitLab to have its own space to store [PyPi](https://pypi.org/) packages. | 12.10+ | | [NuGet Repository](../../user/packages/nuget_repository/index.md) | The GitLab NuGet Repository enables every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.8+ | | [Conan Repository](../../user/packages/conan_repository/index.md) | The GitLab Conan Repository enables every project in GitLab to have its own space to store [Conan](https://conan.io/) packages. | 12.4+ | | [Maven Repository](../../user/packages/maven_repository/index.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | diff --git a/doc/administration/raketasks/praefect.md b/doc/administration/raketasks/praefect.md new file mode 100644 index 00000000000..4f9cf0e10ba --- /dev/null +++ b/doc/administration/raketasks/praefect.md @@ -0,0 +1,22 @@ +# Praefect Rake Tasks + +> [Introduced]( https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28369) in GitLab 12.10. + +## Replica checksums + +Prints out checksums of the repository of a given project_id on the primary as well as secondary internal Gitaly nodes. + +NOTE: **Note:** +This only is relevant and works for projects that have been created on a praefect storage. See the [Praefect Documentation](../gitaly/praefect.md) for configuring Praefect. + +**Omnibus Installation** + +```shell +sudo gitlab-rake "gitlab:praefect:replicas[project_id]" +``` + +**Source Installation** + +```shell +sudo -u git -H bundle exec rake "gitlab:praefect:replicas[project_id]" RAILS_ENV=production +``` diff --git a/doc/administration/troubleshooting/img/AzureAD-scim_attribute_mapping.png b/doc/administration/troubleshooting/img/AzureAD-scim_attribute_mapping.png Binary files differindex 807936423e5..5ad82003eed 100644 --- a/doc/administration/troubleshooting/img/AzureAD-scim_attribute_mapping.png +++ b/doc/administration/troubleshooting/img/AzureAD-scim_attribute_mapping.png diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index aebbc95750d..8b3d8462cc9 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -130,6 +130,7 @@ The following API resources are available outside of project and group contexts | [License](license.md) **(CORE ONLY)** | `/license` | | [Markdown](markdown.md) | `/markdown` | | [Merge requests](merge_requests.md) | `/merge_requests` (also available for groups and projects) | +| [Metrics dashboard annotations](metrics_dashboard_annotations.md) | `/environments/:id/metrics_dashboard/annotations` | | [Namespaces](namespaces.md) | `/namespaces` | | [Notification settings](notification_settings.md) | `/notification_settings` (also available for groups and projects) | | [Pages domains](pages_domains.md) | `/pages/domains` (also available for projects) | diff --git a/doc/api/discussions.md b/doc/api/discussions.md index f1e8965b336..572e174201d 100644 --- a/doc/api/discussions.md +++ b/doc/api/discussions.md @@ -739,7 +739,7 @@ GET /projects/:id/merge_requests/:merge_request_iid/discussions ] ``` -Diff comments contain also position: +Diff comments also contain position: ```json [ @@ -774,7 +774,11 @@ Diff comments contain also position: "new_path": "package.json", "position_type": "text", "old_line": 27, - "new_line": 27 + "new_line": 27, + "line_range": { + "start_line_code": "588440f66559714280628a4f9799f0c4eb880a4a_10_10", + "end_line_code": "588440f66559714280628a4f9799f0c4eb880a4a_11_11" + } }, "resolved": false, "resolvable": true, @@ -820,25 +824,28 @@ POST /projects/:id/merge_requests/:merge_request_iid/discussions Parameters: -| Attribute | Type | Required | Description | -| ------------------------- | -------------- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of a merge request | -| `body` | string | yes | The content of the thread | -| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | -| `position` | hash | no | Position when creating a diff note | -| `position[base_sha]` | string | yes | Base commit SHA in the source branch | -| `position[start_sha]` | string | yes | SHA referencing commit in target branch | -| `position[head_sha]` | string | yes | SHA referencing HEAD of this merge request | -| `position[position_type]` | string | yes | Type of the position reference', allowed values: 'text' or 'image' | -| `position[new_path]` | string | no | File path after change | -| `position[new_line]` | integer | no | Line number after change (for 'text' diff notes) | -| `position[old_path]` | string | no | File path before change | -| `position[old_line]` | integer | no | Line number before change (for 'text' diff notes) | -| `position[width]` | integer | no | Width of the image (for 'image' diff notes) | -| `position[height]` | integer | no | Height of the image (for 'image' diff notes) | -| `position[x]` | integer | no | X coordinate (for 'image' diff notes) | -| `position[y]` | integer | no | Y coordinate (for 'image' diff notes) | +| Attribute | Type | Required | Description | +| --------------------------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a merge request | +| `body` | string | yes | The content of the thread | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | +| `position` | hash | no | Position when creating a diff note | +| `position[base_sha]` | string | yes | Base commit SHA in the source branch | +| `position[start_sha]` | string | yes | SHA referencing commit in target branch | +| `position[head_sha]` | string | yes | SHA referencing HEAD of this merge request | +| `position[position_type]` | string | yes | Type of the position reference', allowed values: 'text' or 'image' | +| `position[new_path]` | string | no | File path after change | +| `position[new_line]` | integer | no | Line number after change (for 'text' diff notes) | +| `position[old_path]` | string | no | File path before change | +| `position[old_line]` | integer | no | Line number before change (for 'text' diff notes) | +| `position[line_range]` | hash | no | Line range for a multi-line diff note | +| `position[line_range][start_line_code]` | string | yes | Line code for the start line | +| `position[line_range][end_line_code]` | string | yes | Line code for the end line | +| `position[width]` | integer | no | Width of the image (for 'image' diff notes) | +| `position[height]` | integer | no | Height of the image (for 'image' diff notes) | +| `position[x]` | integer | no | X coordinate (for 'image' diff notes) | +| `position[y]` | integer | no | Y coordinate (for 'image' diff notes) | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/merge_requests/11/discussions?body=comment diff --git a/doc/api/graphql/reference/gitlab_schema.graphql b/doc/api/graphql/reference/gitlab_schema.graphql index 91d6717de7e..9cdc83d584c 100644 --- a/doc/api/graphql/reference/gitlab_schema.graphql +++ b/doc/api/graphql/reference/gitlab_schema.graphql @@ -3009,6 +3009,11 @@ input EpicTreeNodeFieldsInputType { id: ID! """ + ID of the new parent epic + """ + newParentId: ID + + """ The type of the switch, after or before allowed """ relativePosition: MoveType! @@ -5335,11 +5340,108 @@ type Metadata { type MetricsDashboard { """ + Annotations added to the dashboard. Will always return `null` if `metrics_dashboard_annotations` feature flag is disabled + """ + annotations( + """ + Returns the elements in the list that come after the specified cursor. + """ + after: String + + """ + Returns the elements in the list that come before the specified cursor. + """ + before: String + + """ + Returns the first _n_ elements from the list. + """ + first: Int + + """ + Timestamp marking date and time from which annotations need to be fetched + """ + from: Time! + + """ + Returns the last _n_ elements from the list. + """ + last: Int + + """ + Timestamp marking date and time to which annotations need to be fetched + """ + to: Time + ): MetricsDashboardAnnotationConnection + + """ Path to a file with the dashboard definition """ path: String } +type MetricsDashboardAnnotation { + """ + Description of the annotation + """ + description: String + + """ + Timestamp marking end of annotated time span + """ + endingAt: String + + """ + ID of the annotation + """ + id: ID! + + """ + ID of a dashboard panel to which the annotation should be scoped + """ + panelId: String + + """ + Timestamp marking start of annotated time span + """ + startingAt: String +} + +""" +The connection type for MetricsDashboardAnnotation. +""" +type MetricsDashboardAnnotationConnection { + """ + A list of edges. + """ + edges: [MetricsDashboardAnnotationEdge] + + """ + A list of nodes. + """ + nodes: [MetricsDashboardAnnotation] + + """ + Information to aid in pagination. + """ + pageInfo: PageInfo! +} + +""" +An edge in a connection. +""" +type MetricsDashboardAnnotationEdge { + """ + A cursor for use in pagination. + """ + cursor: String! + + """ + The item at the end of the edge. + """ + node: MetricsDashboardAnnotation +} + """ Represents a milestone. """ @@ -6740,7 +6842,7 @@ type Project { suggestionCommitMessage: String """ - List of project tags + List of project topics (not Git tags) """ tagList: String @@ -9561,6 +9663,11 @@ type Vulnerability { location: JSON """ + The project on which the vulnerability was found + """ + project: Project + + """ Type of the security report that found the vulnerability (SAST, DEPENDENCY_SCANNING, CONTAINER_SCANNING, DAST) """ reportType: VulnerabilityReportType diff --git a/doc/api/graphql/reference/gitlab_schema.json b/doc/api/graphql/reference/gitlab_schema.json index fb642b1222f..26a2acdb4c6 100644 --- a/doc/api/graphql/reference/gitlab_schema.json +++ b/doc/api/graphql/reference/gitlab_schema.json @@ -8716,6 +8716,16 @@ } }, "defaultValue": null + }, + { + "name": "newParentId", + "description": "ID of the new parent epic", + "type": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + }, + "defaultValue": null } ], "interfaces": null, @@ -15280,6 +15290,83 @@ "description": null, "fields": [ { + "name": "annotations", + "description": "Annotations added to the dashboard. Will always return `null` if `metrics_dashboard_annotations` feature flag is disabled", + "args": [ + { + "name": "from", + "description": "Timestamp marking date and time from which annotations need to be fetched", + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + } + }, + "defaultValue": null + }, + { + "name": "to", + "description": "Timestamp marking date and time to which annotations need to be fetched", + "type": { + "kind": "SCALAR", + "name": "Time", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "after", + "description": "Returns the elements in the list that come after the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "before", + "description": "Returns the elements in the list that come before the specified cursor.", + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "first", + "description": "Returns the first _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + }, + { + "name": "last", + "description": "Returns the last _n_ elements from the list.", + "type": { + "kind": "SCALAR", + "name": "Int", + "ofType": null + }, + "defaultValue": null + } + ], + "type": { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotationConnection", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "path", "description": "Path to a file with the dashboard definition", "args": [ @@ -15303,6 +15390,205 @@ }, { "kind": "OBJECT", + "name": "MetricsDashboardAnnotation", + "description": null, + "fields": [ + { + "name": "description", + "description": "Description of the annotation", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "endingAt", + "description": "Timestamp marking end of annotated time span", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "id", + "description": "ID of the annotation", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "ID", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "panelId", + "description": "ID of a dashboard panel to which the annotation should be scoped", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "startingAt", + "description": "Timestamp marking start of annotated time span", + "args": [ + + ], + "type": { + "kind": "SCALAR", + "name": "String", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotationConnection", + "description": "The connection type for MetricsDashboardAnnotation.", + "fields": [ + { + "name": "edges", + "description": "A list of edges.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotationEdge", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "nodes", + "description": "A list of nodes.", + "args": [ + + ], + "type": { + "kind": "LIST", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotation", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "pageInfo", + "description": "Information to aid in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "OBJECT", + "name": "PageInfo", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotationEdge", + "description": "An edge in a connection.", + "fields": [ + { + "name": "cursor", + "description": "A cursor for use in pagination.", + "args": [ + + ], + "type": { + "kind": "NON_NULL", + "name": null, + "ofType": { + "kind": "SCALAR", + "name": "String", + "ofType": null + } + }, + "isDeprecated": false, + "deprecationReason": null + }, + { + "name": "node", + "description": "The item at the end of the edge.", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "MetricsDashboardAnnotation", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + } + ], + "inputFields": null, + "interfaces": [ + + ], + "enumValues": null, + "possibleTypes": null + }, + { + "kind": "OBJECT", "name": "Milestone", "description": "Represents a milestone.", "fields": [ @@ -20096,7 +20382,7 @@ }, { "name": "tagList", - "description": "List of project tags", + "description": "List of project topics (not Git tags)", "args": [ ], @@ -28864,6 +29150,20 @@ "deprecationReason": null }, { + "name": "project", + "description": "The project on which the vulnerability was found", + "args": [ + + ], + "type": { + "kind": "OBJECT", + "name": "Project", + "ofType": null + }, + "isDeprecated": false, + "deprecationReason": null + }, + { "name": "reportType", "description": "Type of the security report that found the vulnerability (SAST, DEPENDENCY_SCANNING, CONTAINER_SCANNING, DAST)", "args": [ diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index d1ea825bef3..4d3d77ba35f 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -833,6 +833,16 @@ Autogenerated return type of MergeRequestSetWip | --- | ---- | ---------- | | `path` | String | Path to a file with the dashboard definition | +## MetricsDashboardAnnotation + +| Name | Type | Description | +| --- | ---- | ---------- | +| `description` | String | Description of the annotation | +| `endingAt` | String | Timestamp marking end of annotated time span | +| `id` | ID! | ID of the annotation | +| `panelId` | String | ID of a dashboard panel to which the annotation should be scoped | +| `startingAt` | String | Timestamp marking start of annotated time span | + ## Milestone Represents a milestone. @@ -985,7 +995,7 @@ Information about pagination in a connection. | `starCount` | Int! | Number of times the project has been starred | | `statistics` | ProjectStatistics | Statistics of the project | | `suggestionCommitMessage` | String | The commit message used to apply merge request suggestions | -| `tagList` | String | List of project tags | +| `tagList` | String | List of project topics (not Git tags) | | `userPermissions` | ProjectPermissions! | Permissions for the current user on the resource | | `visibility` | String | Visibility of the project | | `vulnerabilitySeveritiesCount` | VulnerabilitySeveritiesCount | Counts for each severity of vulnerability of the project. Available only when feature flag `first_class_vulnerabilities` is enabled | @@ -1502,6 +1512,7 @@ Represents a vulnerability. | `description` | String | Description of the vulnerability | | `id` | ID! | GraphQL ID of the vulnerability | | `location` | JSON | The JSON location metadata for the vulnerability. Its format depends on the type of the security scan that found the vulnerability | +| `project` | Project | The project on which the vulnerability was found | | `reportType` | VulnerabilityReportType | Type of the security report that found the vulnerability (SAST, DEPENDENCY_SCANNING, CONTAINER_SCANNING, DAST) | | `severity` | VulnerabilitySeverity | Severity of the vulnerability (INFO, UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL) | | `state` | VulnerabilityState | State of the vulnerability (DETECTED, DISMISSED, RESOLVED, CONFIRMED) | diff --git a/doc/api/groups.md b/doc/api/groups.md index 1809ddfa47f..34c01766d06 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -862,49 +862,71 @@ Lists LDAP group links. GET /groups/:id/ldap_group_links ``` -Parameters: - -- `id` (required) - The ID of a group +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | -### Add LDAP group link **(STARTER)** +### Add LDAP group link with CN or filter **(STARTER)** -Adds an LDAP group link. +Adds an LDAP group link using a CN or filter. Adding a group link by filter is only supported in the Premium tier and above. ```plaintext POST /groups/:id/ldap_group_links ``` -Parameters: +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `cn` | string | no | The CN of an LDAP group | +| `filter` | string | no | The LDAP filter for the group | +| `group_access` | integer | yes | Minimum access level for members of the LDAP group | +| `provider` | string | yes | LDAP provider for the LDAP group link | -- `id` (required) - The ID of a group -- `cn` (required) - The CN of a LDAP group -- `group_access` (required) - Minimum access level for members of the LDAP group -- `provider` (required) - LDAP provider for the LDAP group +NOTE: **Note:** +To define the LDAP group link, provide either a `cn` or a `filter`, but not both. ### Delete LDAP group link **(STARTER)** -Deletes an LDAP group link. +Deletes an LDAP group link. Deprecated. Will be removed in a future release. ```plaintext DELETE /groups/:id/ldap_group_links/:cn ``` -Parameters: - -- `id` (required) - The ID of a group -- `cn` (required) - The CN of a LDAP group +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `cn` | string | yes | The CN of an LDAP group | -Deletes a LDAP group link for a specific LDAP provider +Deletes an LDAP group link for a specific LDAP provider. Deprecated. Will be removed in a future release. ```plaintext DELETE /groups/:id/ldap_group_links/:provider/:cn ``` -Parameters: +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `cn` | string | yes | The CN of an LDAP group | +| `provider` | string | yes | LDAP provider for the LDAP group link | + +### Delete LDAP group link with CN or filter **(STARTER)** + +Deletes an LDAP group link using a CN or filter. Deleting by filter is only supported in the Premium tier and above. + +```plaintext +DELETE /groups/:id/ldap_group_links +``` -- `id` (required) - The ID of a group -- `cn` (required) - The CN of a LDAP group -- `provider` (required) - Name of a LDAP provider +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `cn` | string | no | The CN of an LDAP group | +| `filter` | string | no | The LDAP filter for the group | +| `provider` | string | yes | LDAP provider for the LDAP group link | + +NOTE: **Note:** +To delete the LDAP group link, provide either a `cn` or a `filter`, but not both. ## Namespaces in groups diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index adb5b00085e..b0ffeae2b55 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -668,6 +668,7 @@ dependent on the `merge_status`. It'll return `false` unless `merge_status` is }, "diverged_commits_count": 2, "rebase_in_progress": false, + "first_contribution": false, "task_completion_status":{ "count":0, "completed_count":0 diff --git a/doc/api/metrics_dashboard_annotations.md b/doc/api/metrics_dashboard_annotations.md new file mode 100644 index 00000000000..d8a018fe6c3 --- /dev/null +++ b/doc/api/metrics_dashboard_annotations.md @@ -0,0 +1,51 @@ +# Dashboard annotations API + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29089) in GitLab 12.10 behind a disabled feature flag. + +Metrics dashboard annotations allow you to indicate events on your graphs at a single point in time or over a timespan. + +## Enable the metrics dashboard annotations API + +The `:metrics_dashboard_annotations` feature flag is disabled by default. +To turn on this API, ask a GitLab administrator with Rails console +access to run the following command: + +```ruby +Feature.enable(:metrics_dashboard_annotations) +``` + +## Create a new annotation + +```plaintext +POST /environments/:id/metrics_dashboard/annotations/ +``` + +Parameters: + +| Attribute | Type | Required | Description | +|:---------------|:---------------|:---------|:-----------------------------------------------------------------------------| +| `dashboard_path` | string | yes | ID of the dashboard which needs to be annotated. | +| `starting_at` | string | yes | Date time string, ISO 8601 formatted, such as `2016-03-11T03:45:40Z`. Timestamp marking start point of annotation. | +| `ending_at` | string | no | Date time string, ISO 8601 formatted, such as `2016-03-11T03:45:40Z`. Timestamp marking end point of annotation. When not supplied annotation will be displayed as single event at start point. | +| `description` | string | yes | Description of the annotation. | + +```shell +curl --header 'Private-Token: <your_access_token>' https://gitlab.example.com/api/v4/environments/1/metrics_dashboard/annotations \ + --data-urlencode "dashboard_path=.gitlab/dashboards/custom_metrics.yml" \ + --data-urlencode "starting_at=2016-03-11T03:45:40Z" \ + --data-urlencode "description=annotation description" +``` + +Example Response: + +```json +{ + "id": 4, + "starting_at": "2016-04-08T03:45:40.000Z", + "ending_at": null, + "dashboard_path": ".gitlab/dashboards/custom_metrics.yml", + "description": "annotation description", + "environment_id": 1, + "cluster_id": null +} +``` diff --git a/doc/api/packages.md b/doc/api/packages.md index 31fc2863708..8671de006d2 100644 --- a/doc/api/packages.md +++ b/doc/api/packages.md @@ -20,7 +20,7 @@ GET /projects/:id/packages | `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) | | `order_by`| string | no | The field to use as order. One of `created_at` (default), `name`, `version`, or `type`. | | `sort` | string | no | The direction of the order, either `asc` (default) for ascending order or `desc` for descending order. | -| `package_type` | string | no | Filter the returned packages by type. One of `conan`, `maven`, `npm` or `nuget`. (_Introduced in GitLab 12.9_) +| `package_type` | string | no | Filter the returned packages by type. One of `conan`, `maven`, `npm`, `pypi` or `nuget`. (_Introduced in GitLab 12.9_) | `package_name` | string | no | Filter the project packages with a fuzzy search by name. (_Introduced in GitLab 12.9_) ```shell @@ -67,7 +67,7 @@ GET /groups/:id/packages | `exclude_subgroups` | boolean | false | If the parameter is included as true, packages from projects from subgroups are not listed. Default is `false`. | | `order_by`| string | no | The field to use as order. One of `created_at` (default), `name`, `version`, `type`, or `project_path`. | | `sort` | string | no | The direction of the order, either `asc` (default) for ascending order or `desc` for descending order. | -| `package_type` | string | no | Filter the returned packages by type. One of `conan`, `maven`, `npm` or `nuget`. (_Introduced in GitLab 12.9_) | +| `package_type` | string | no | Filter the returned packages by type. One of `conan`, `maven`, `npm`, `pypi` or `nuget`. (_Introduced in GitLab 12.9_) | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/:id/packages?exclude_subgroups=true diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index 78022e8e754..21a90670aa6 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -173,7 +173,8 @@ requests.post(url, headers=headers, data=data, files=files) "path_with_namespace": "root/api-project", "created_at": "2018-02-13T09:05:58.023Z", "import_status": "scheduled", - "correlation_id": "mezklWso3Za" + "correlation_id": "mezklWso3Za", + "failed_relations": [] } ``` @@ -202,6 +203,15 @@ Status can be one of: - `finished` If the status is `failed`, it will include the import error message under `import_error`. +If the status is `failed`, `started` or `finished`, the `failed_relations` array might +be populated with any occurrences of relations that failed to import either due to +unrecoverable errors or because retries were exhausted (a typical example are query timeouts.) + +NOTE: **Note:** +An element's `id` field in `failed_relations` references the failure record, not the relation. + +NOTE: **Note:** +The `failed_relations` array is currently capped to 100 items. ```json { @@ -213,6 +223,16 @@ If the status is `failed`, it will include the import error message under `impor "path_with_namespace": "gitlab-org/gitlab-test", "created_at": "2017-08-29T04:36:44.383Z", "import_status": "started", - "correlation_id": "mezklWso3Za" + "correlation_id": "mezklWso3Za", + "failed_relations": [ + { + "id": 42, + "created_at": "2020-04-02T14:48:59.526Z", + "exception_class": "RuntimeError", + "exception_message": "A failure occurred", + "source": "custom error context", + "relation_name": "merge_requests" + } + ] } ``` diff --git a/doc/api/settings.md b/doc/api/settings.md index 5fe068cf085..cf48048c830 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -68,7 +68,8 @@ Example response: "allow_local_requests_from_system_hooks": false, "asset_proxy_enabled": true, "asset_proxy_url": "https://assets.example.com", - "asset_proxy_whitelist": ["example.com", "*.example.com", "your-instance.com"] + "asset_proxy_whitelist": ["example.com", "*.example.com", "your-instance.com"], + "npm_package_requests_forwarding": true } ``` @@ -154,7 +155,8 @@ Example response: "geo_node_allowed_ips": "0.0.0.0/0, ::/0", "allow_local_requests_from_hooks_and_services": true, "allow_local_requests_from_web_hooks_and_services": true, - "allow_local_requests_from_system_hooks": false + "allow_local_requests_from_system_hooks": false, + "npm_package_requests_forwarding": true } ``` @@ -285,6 +287,7 @@ are listed in the descriptions of the relevant settings. | `mirror_capacity_threshold` | integer | no | **(PREMIUM)** Minimum capacity to be available before scheduling more mirrors preemptively | | `mirror_max_capacity` | integer | no | **(PREMIUM)** Maximum number of mirrors that can be synchronizing at the same time. | | `mirror_max_delay` | integer | no | **(PREMIUM)** Maximum time (in minutes) between updates that a mirror can have when scheduled to synchronize. | +| `npm_package_requests_forwarding` | boolean | no | **(PREMIUM)** Use npmjs.org as a default remote repository when the package is not found in the GitLab NPM Registry | | `outbound_local_requests_whitelist` | array of strings | no | Define a list of trusted domains or ip addresses to which local requests are allowed when local requests for hooks and services are disabled. | `pages_domain_verification_enabled` | boolean | no | Require users to prove ownership of custom domains. Domain verification is an essential security measure for public GitLab sites. Users are required to demonstrate they control a domain before it is enabled. | | `password_authentication_enabled_for_git` | boolean | no | Enable authentication for Git over HTTP(S) via a GitLab account password. Default is `true`. | diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 0ae7a10a290..427f61deb29 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -118,7 +118,7 @@ not without its own challenges: instance of Docker engine so they won't conflict with each other. But this also means that jobs can be slower because there's no caching of layers. - By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is - the recommended storage driver. See [Using the overlayfs driver](#using-the-overlayfs-driver) + the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver) for details. - Since the `docker:19.03.8-dind` container and the Runner container don't share their root filesystem, the job's working directory can be used as a mount point for @@ -448,7 +448,7 @@ The steps in the `script` section for the `build` stage can be summed up to: 1. The last two commands push the tagged Docker images to the container registry so that they may also be used as cache for subsequent builds. -## Using the OverlayFS driver +## Use the OverlayFS driver NOTE: **Note:** The shared Runners on GitLab.com use the `overlay2` driver by default. @@ -480,18 +480,22 @@ which can be avoided if a different driver is used, for example `overlay2`. overlay ``` -### Use driver per project +### Use the OverlayFS driver per project -You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: +You can enable the driver for each project individually by using the `DOCKER_DRIVER` +environment [variable](../yaml/README.md#variables) in `.gitlab-ci.yml`: ```yaml variables: DOCKER_DRIVER: overlay2 ``` -### Use driver for every project +### Use the OverlayFS driver for every project -To enable the driver for every project, you can set the environment variable for every build by adding `environment` in the `[[runners]]` section of `config.toml`: +If you use your own [GitLab Runners](https://docs.gitlab.com/runner/), you +can enable the driver for every project by setting the `DOCKER_DRIVER` +environment variable in the +[`[[runners]]` section of `config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section): ```toml environment = ["DOCKER_DRIVER=overlay2"] @@ -499,11 +503,9 @@ environment = ["DOCKER_DRIVER=overlay2"] If you're running multiple Runners you will have to modify all configuration files. -> **Notes:** -> -> - More information about the Runner configuration is available in the [Runner documentation](https://docs.gitlab.com/runner/configuration/). -> - For more information about using OverlayFS with Docker, you can read -> [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). +NOTE: **Note:** +Read more about the [Runner configuration](https://docs.gitlab.com/runner/configuration/) +and [using the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). ## Using the GitLab Container Registry diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 64367727371..1b3c4c887f4 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -20,24 +20,25 @@ Examples are available in several forms. As a collection of: The following table lists examples with step-by-step tutorials that are contained in this section. -| Use case | Resource | -|:----------------------------|:---------------------------------------------------------------------------------------------------------------------------| -| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | -| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | -| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | -| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | -| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | -| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Use case | Resource | +|:------------------------------|:---------------------------------------------------------------------------------------------------------------------------| +| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | +| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | +| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | +| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | +| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | +| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | +| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | ### Contributing examples diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png Binary files differnew file mode 100644 index 00000000000..65c8546deb2 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png Binary files differnew file mode 100644 index 00000000000..2399d8f8879 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md new file mode 100644 index 00000000000..2986afe8284 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -0,0 +1,215 @@ +--- +type: tutorial +--- + +# Authenticating and Reading Secrets With Hashicorp Vault + +This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. + +## Requirements + +This tutorial assumes you are familiar with GitLab CI/CD and Vault. + +To follow along, you will need: + +- An account on GitLab. +- A running Vault server and the access required to configure authentication and create roles and policies. + +NOTE: **Note:** +You will need to replace the `vault.example.com` URL below with the URL of your Vault server and `gitlab.example.com` with the URL of your GitLab instance. + +## How it works + +Each job has JSON Web Token (JWT) provided as environment variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt/#jwt-authentication) method. + +The JWT's payload looks like this: + +```json +{ + "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558", # Unique identifier for this token + "iss": "gitlab.example.com", # Issuer, the domain of your GitLab instance + "iat": 1585710286, # Issued at + "nbf": 1585798372, # Not valid before + "exp": 1585713886, # Expire at + "sub": "job_1212", # Subject (job id) + "namespace_id": "1", + "namespace_path": "mygroup", + "project_id": "22", + "project_path": "mygroup/myproject", + "user_id": "42", + "user_login": "myuser", + "user_email": "myuser@example.com" + "pipeline_id": "1212", + "job_id": "1212", + "ref": "auto-deploy-2020-04-01", # Git ref for this job + "ref_type": "branch", # Git ref type, branch or tag + "ref_protected": "true" # true if this git ref is protected, false otherwise +} +``` + +The JWT is encoded by using RS256 and signed with your GitLab instance's OpenID Connect private key. The expire time for the token will be set to job's timeout, if specifed, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. + +You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication. + +When configuring roles in Vault, you can use [bound_claims](https://www.vaultproject.io/docs/auth/jwt/#bound-claims) to match against the JWT's claims and restrict which secrets each CI job has access to. + +To communicate with Vault, you can use either its CLI client or perform API requests (using `curl` or another client). + +## Example + +CAUTION: **Caution**: +JWTs are credentials, which can grant access to resources. Be careful where you paste them! + +Let's say you have the passwords for your staging and production databases stored in a Vault server that is running on `http://vault.example.com:8200`. Your staging password is `pa$$w0rd` and your production password is `real-pa$$w0rd`. + +```shell +$ vault kv get -field=password secret/myproject/staging/db +pa$$w0rd + +$ vault kv get -field=password secret/myproject/production/db +real-pa$$w0rd +``` + +To configure your Vault server, start by enabling the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt/) method: + +```shell +$ vault auth enable jwt +Success! Enabled jwt auth method at: jwt/ +``` + +Then create policies that allow you to read these secrets (one for each secret): + +```shell +$ vault policy write myproject-staging - <<EOF +# Policy name: myproject-staging +# +# Read-only permission on 'secret/data/myproject/staging/*' path +path "secret/data/myproject/staging/*" { + capabilities = [ "read" ] +} +EOF +Success! Uploaded policy: myproject-staging + +$ vault policy write myproject-production - <<EOF +# Policy name: myproject-production +# +# Read-only permission on 'secret/data/myproject/production/*' path +path "secret/data/myproject/production/*" { + capabilities = [ "read" ] +} +EOF +Success! Uploaded policy: myproject-production +``` + +You'll also need roles that will link the JWT with these policies. + +One for staging named `myproject-staging`: + +```shell +$ vault write auth/jwt/role/myproject-staging - <<EOF +{ + "role_type": "jwt", + "policies": ["myproject-staging"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims": { + "project_id": "22", + "ref": "master", + "ref_type": "branch" + } +} +EOF +``` + +And one for production named `myproject-production`: + +```shell +$ vault write auth/jwt/role/myproject-production - <<EOF +{ + "role_type": "jwt", + "policies": ["myproject-production"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims_type": "glob", + "bound_claims": { + "project_id": "22", + "ref_protected": "true", + "ref_type": "branch", + "ref": "auto-deploy-*" + } +} +EOF +``` + +This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims will be allowed to authenticate. + +Combined with GitLab's [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. + +[token_explicit_max_ttl](https://www.vaultproject.io/api/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds. + +[user_claim](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. + +[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values will be interpreted as globs, with `*` matching any number of characters. + +For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). + +CAUTION: **Caution**: +Always restrict your roles to project or namespace by using one of the provided claims (e.g. `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. + +Now, configure the JWT Authentication method: + +```shell +$ vault write auth/jwt/config \ + jwks_url="https://gitlab.example.com/-/jwks" \ + bound_issuer="gitlab.example.com" +``` + +[bound_issuer](https://www.vaultproject.io/api/auth/jwt#inlinecode-bound_issuer) specifies that only a JWT with the issuer (that is, the `iss` claim) set to `gitlab.example.com` can use this method to authenticate, and that the JWKS endpoint (`https://gitlab.example.com/-/jwks`) should be used to validate the token. + +For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api/auth/jwt#configure). + +The following job, when run for the `master` branch, will be able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: + +```yaml +read_secrets: + script: + # Check job's ref name + - echo $CI_COMMIT_REF_NAME + # and is this ref protected + - echo $CI_COMMIT_REF_PROTECTED + # Vault's address can be provided here or as CI variable + - export VAULT_ADDR=http://vault.example.com:8200 + # Authenticate and get token. Token expiry time and other properties can be configured + # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 + - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-staging jwt=$CI_JOB_JWT)" + # Now use the VAULT_TOKEN to read the secret and store it in an environment variable + - export PASSWORD="$(vault kv get -field=password secret/myproject/staging/db)" + # Use the secret + - echo $PASSWORD + # This will fail because the role myproject-staging can not read secrets from secret/myproject/production/* + - export PASSWORD="$(vault kv get -field=password secret/myproject/production/db)" +``` + + + +The following job will be able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: + +```yaml +read_secrets: + script: + # Check job's ref name + - echo $CI_COMMIT_REF_NAME + # and is this ref protected + - echo $CI_COMMIT_REF_PROTECTED + # Vault's address can be provided here or as CI variable + - export VAULT_ADDR=http://vault.example.com:8200 + # Authenticate and get token. Token expiry time and other properties can be configured + # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 + - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-production jwt=$CI_JOB_JWT)" + # Now use the VAULT_TOKEN to read the secret and store it in environment variable + - export PASSWORD="$(vault kv get -field=password secret/myproject/production/db)" + # Use the secret + - echo $PASSWORD +``` + + diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index a97d4b865c8..c79db6dfbea 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -88,7 +88,7 @@ ruby: stage: test script: - bundle install - - rspec spec/lib/ --format RspecJunitFormatter --out rspec.xml + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml artifacts: paths: - rspec.xml diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png Binary files differindex de5897c271b..7b903716a3d 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index b6706c2a272..641192afea1 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -168,6 +168,27 @@ A Merge Train pipeline cannot be retried because the merge request is dropped fr In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which will then initiate a new pipeline. +### Unable to add to merge train with message "The pipeline for this merge request failed." + +Sometimes the **Start/Add to Merge Train** button is not available and the merge request says, +"The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure." + +This issue occurs when [**Pipelines must succeed**](../../../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +is enabled in **Settings > General > Merge requests**. This option requires that you +run a new successful pipeline before you can re-add a merge request to a merge train. + +Merge trains ensure that each pipeline has succeeded before a merge happens, so +you can clear the **Pipelines must succeed** check box and keep +**Merge pipelines will try to validate the post-merge result prior to merging** (merge trains) enabled. + +If you want to keep the **Pipelines must succeed** option enabled along with Merge +Trains, you can create a new pipeline for merged results when this error occurs by +going to the **Pipelines** tab and clicking **Run pipeline**. Then click +**Start/Add to merge train when pipeline succeeds**. + +See [the related issue](https://gitlab.com/gitlab-org/gitlab/issues/35135) +for more information. + ### Merge Trains feature flag **(PREMIUM ONLY)** To enable and disable the Merge Trains feature, use the `:disable_merge_trains` feature flag. diff --git a/doc/ci/pipelines/img/job_group_v12_10.png b/doc/ci/pipelines/img/job_group_v12_10.png Binary files differindex 0ce121ab572..27e6bfbfc0f 100644 --- a/doc/ci/pipelines/img/job_group_v12_10.png +++ b/doc/ci/pipelines/img/job_group_v12_10.png diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index bddf64f397e..7ac9ba6a7dd 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -160,7 +160,7 @@ This also determines the visibility of these related features: - Job output logs - Job artifacts -- The [pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security-dashboard) **(ULTIMATE)** +- The [pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security) **(ULTIMATE)** If **Public pipelines** is enabled (default): diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 860eab469dd..c3bdd524bff 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -188,9 +188,6 @@ With Visual Reviews, you can provide a feedback form to your Review Apps so that reviewers can post comments directly from the app back to the merge request that spawned the Review App. -NOTE: **Note:** Visual Reviews currently only work for public projects. Support for private -and internal projects [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/42750). - ### Configuring Visual Reviews Ensure that the `anonymous_visual_review_feedback` feature flag is enabled. @@ -218,6 +215,7 @@ looks like: data-merge-request-id='1' data-mr-url='https://gitlab.example.com' data-project-path='sarah/review-app-tester' + data-require-auth='true' id='review-app-toolbar-script' src='https://gitlab.example.com/assets/webpack/visual_review_toolbar.js'> </script> @@ -235,6 +233,7 @@ to replace those values at runtime when each review app is created: - `data-mr-url` is the URL of the GitLab instance and will be the same for all review apps. - `data-project-path` is the project's path, which can be found by `CI_PROJECT_PATH`. +- `data-require-auth` is optional for public projects but required for [private and internal ones](#visual-reviews-in-private-or-internal-projects). If this is set to `true`, the user will be required to enter their [personal access token](../../user/profile/personal_access_tokens.md) instead of their name and email. - `id` is always `review-app-toolbar-script`, you don't need to change that. - `src` is the source of the review toolbar script, which resides in the respective GitLab instance and will be the same for all review apps. @@ -272,6 +271,15 @@ can supply the ID by either: - Dynamically adding the `data-merge-request-id` value during the build of the app. - Supplying it manually through the visual review form in the app. +### Visual Reviews in private or internal projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/42750#note_317271120) in GitLab 12.10. + +To enable visual reviews for private and internal projects, set the +[`data-require-auth` variable](#configuring-visual-reviews) to `true`. When enabled, +the user must enter a [personal access token](../../user/profile/personal_access_tokens.md) +with `read_api` scope before submitting feedback. + ### Using Visual Reviews After Visual Reviews has been [enabled](#configuring-visual-reviews) for the @@ -285,7 +293,7 @@ To use the feedback form: 1. Make a comment on the visual review. You can make use of all the [Markdown annotations](../../user/markdown.md) that are also available in merge request comments. -1. Submit your feedback anonymously or add your name. +1. If `data-require-auth` is `true`, you must enter your [personal access token](../../user/profile/personal_access_tokens.md). Otherwise, you must enter your name, and optionally, your email. 1. Finally, click **Send feedback**. After you make and submit a comment in the visual review box, it will appear diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index e619a32b90f..99fbc2134a4 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -81,6 +81,8 @@ echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" ``` +There are [some predefined variables](#custom-variables-validated-by-gitlab) of this type, which may be further validated. They will appear when you add or update a variable. + ##### File type The example above can now be simplified by creating a "File" type variable, and using @@ -116,6 +118,21 @@ If the value does not meet the requirements above, then the CI variable will fai In order to save, either alter the value to meet the masking requirements or disable **Masked** for the variable. +#### Custom variables validated by GitLab + +Some variables are listed in the UI so you can choose them more quickly. +GitLab validates the values of these variables to ensure they are in the correct format. + +| Variable | Allowed Values | Introduced in | +|-------------------------|----------------------------------------------------|---------------| +| `AWS_ACCESS_KEY_ID` | 20 characters: letters, digits | 12.10 | +| `AWS_DEFAULT_REGION` | Any | 12.10 | +| `AWS_SECRET_ACCESS_KEY` | 40 characters: letters, digits, special characters | 12.10 | + +NOTE: **Note:** +When you store credentials, there are security implications. If you are using AWS keys, +for example, follow their [best practices](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). + ## Getting started To get started with environment variables in the scope of GitLab @@ -482,11 +499,16 @@ value you set for this specific pipeline: ## Environment variables expressions -> Introduced in GitLab 10.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyexcept-advanced) +> - [Expanded](https://gitlab.com/gitlab-org/gitlab/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules) -It is possible to use variables expressions with only / except policies in -`.gitlab-ci.yml`. By using this approach you can limit what jobs are going to -be created within a pipeline after pushing a code to GitLab. +Variable expressions can be used to limit what jobs are going to be created +within a pipeline after pushing changes to GitLab. + +In `.gitlab-ci.yml`, they work with both + +- [`rules`](../yaml/README.md#rules), which is the recommended approach, and +- [`only` and `except`](../yaml/README.md#onlyexcept-basic), which are candidates for deprecation. This is particularly useful in combination with variables and triggered pipeline variables. @@ -573,8 +595,8 @@ Below you can find supported syntax reference: Examples: - - `$VARIABLE =~ /^content.*/` - - `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) + - `=~`: True if pattern is matched. Ex: `$VARIABLE =~ /^content.*/` + - `!~`: True if pattern is not matched. Ex: `$VARIABLE_1 !~ /^content.*/` ([Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/61900) in GitLab 11.11) Variable pattern matching with regular expressions uses the [RE2 regular expression syntax](https://github.com/google/re2/wiki/Syntax). @@ -596,8 +618,48 @@ Below you can find supported syntax reference: It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise supported syntax may be used in a conjunctive or disjunctive statement. - Precedence of operators follows standard Ruby 2.5 operation - [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). + Precedence of operators follows the + [Ruby 2.5 standard](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html), + so `&&` is evaluated before `||`. + +### Storing regular expressions in variables + +It is possible to store a regular expression in a variable, to be used for pattern matching: + +```yaml +variables: + STAGINGRELS: '/staging0|staging1/' + +deploy_staging: + script: do.sh deploy staging + environment: staging + rules: + - if: '$RELEASE =~ $STAGINGRELS' +``` + +NOTE: **Note:** +The available regular expression syntax is limited. See [related issue](https://gitlab.com/gitlab-org/gitlab/issues/35438) +for more details. + +If needed, you can use a test pipeline to determine whether a regular expression will +work in a variable. The example below tests the `^mast.*` regular expression directly, +as well as from within a variable: + +```yaml +variables: + MYSTRING: 'master' + MYREGEX: '/^mast.*/' + +testdirect: + script: /bin/true + rules: + - if: '$MYSTRING =~ /^mast.*/' + +testvariable: + script: /bin/true + rules: + - if: '$MYSTRING =~ $MYREGEX' +``` ## Debug logging diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 8b8eeb83d16..f53fd371c10 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -62,6 +62,7 @@ future GitLab releases.** | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | `CI_JOB_TOKEN` | 9.0 | 1.2 | Token used for authenticating with the [GitLab Container Registry](../../user/packages/container_registry/index.md) and downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories) | +| `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../examples/authenticating-with-hashicorp-vault). | | `CI_JOB_URL` | 11.1 | 0.5 | Job details URL | | `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_CHANGED_PAGE_PATHS` | 12.9 | all | Comma-separated list of paths of changed pages in a deployed [Review App](../review_apps/index.md) for a [Merge Request](../merge_request_pipelines/index.md). A [Route Map](../review_apps/index.md#route-maps) must be configured. | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index e79d44cb057..31459735101 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -3056,6 +3056,9 @@ There can be multiple `resource_group`s defined per environment. A good use case is when deploying to physical devices. You may have more than one physical device, and each one can be deployed to, but there can be only one deployment per device at any given time. +NOTE: **Note:** +This key can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces, but it cannot start or end with `/`. + ### `include` > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. @@ -3777,11 +3780,12 @@ script: You can set the number for attempts the running job will try to execute each of the following stages: -| Variable | Description | -|-------------------------------- |-------------| -| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a job | -| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a job | -| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a job | +| Variable | Description | +|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a job | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a job | +| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a job | +| **EXECUTOR_JOB_SECTION_ATTEMPTS** | [Since GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450), the number of attempts to run a section in a job after a [`No Such Container`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) error ([Docker executor](https://docs.gitlab.com/runner/executors/docker.html) only). | The default is one single attempt. diff --git a/doc/development/README.md b/doc/development/README.md index 16858b0c58e..b041e48f0c2 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -171,6 +171,10 @@ Complementary reads: - [Testing standards and style guidelines](testing_guide/index.md) - [Frontend testing standards and style guidelines](testing_guide/frontend_testing.md) +## Refactoring guides + +- [Refactoring guidelines](refactoring_guide/index.md) + ## Documentation guides - [Writing documentation](documentation/index.md) diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md index 37d8a677389..78e35023766 100644 --- a/doc/development/api_styleguide.md +++ b/doc/development/api_styleguide.md @@ -15,7 +15,7 @@ Always use an [Entity] to present the endpoint's payload. API endpoints must come with [documentation](documentation/styleguide.md#api), unless it is internal or behind a feature flag. The docs should be in the same merge request, or, if strictly necessary, -in a follow-up with the same milestone as the original merge request. +in a follow-up with the same milestone as the original merge request. ## Methods and parameters description @@ -125,6 +125,58 @@ different components are making use of. [validation, and coercion of the parameters]: https://github.com/ruby-grape/grape#parameter-validation-and-coercion [installing GitLab under a relative URL]: https://docs.gitlab.com/ee/install/relative_url.html +## Avoiding N+1 problems + +In order to avoid N+1 problems that are common when returning collections +of records in an API endpoint, we should use eager loading. + +A standard way to do this within the API is for models to implement a +scope called `with_api_entity_associations` that will preload the +associations and data returned in the API. An example of this scope can +be seen in +[the `Issue` model](https://gitlab.com/gitlab-org/gitlab/blob/2fedc47b97837ea08c3016cf2fb773a0300a4a25/app%2Fmodels%2Fissue.rb#L62). + +In situations where the same model has multiple entities in the API +(for instance, `UserBasic`, `User` and `UserPublic`) you should use your +discretion with applying this scope. It may be that you optimize for the +most basic entity, with successive entities building upon that scope. + +The `with_api_entity_associations` scope will also [automatically preload +data](https://gitlab.com/gitlab-org/gitlab/blob/19f74903240e209736c7668132e6a5a735954e7c/app%2Fmodels%2Ftodo.rb#L34) +for `Todo` _targets_ when returned in the Todos API. + +For more context and discussion about preloading see +[this merge request](https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/25711) +which introduced the scope. + +### Verifying with tests + +When an API endpoint returns collections, always add a test to verify +that the API endpoint does not have an N+1 problem, now and in the future. +We can do this using [`ActiveRecord::QueryRecorder`](query_recorder.md). + +Example: + +```ruby +def make_api_request + get api('/foo', personal_access_token: pat) +end + +it 'avoids N+1 queries', :request_store do + # Firstly, record how many PostgreSQL queries the endpoint will make + # when it returns a single record + create_record + + control = ActiveRecord::QueryRecorder.new { make_api_request } + + # Now create a second record and ensure that the API does not execute + # any more queries than before + create_record + + expect { make_api_request }.not_to exceed_query_limit(control) +end +``` + ## Testing When writing tests for new API endpoints, consider using a schema [fixture](./testing_guide/best_practices.md#fixtures) located in `/spec/fixtures/api/schemas`. You can `expect` a response to match a given schema: @@ -132,3 +184,5 @@ When writing tests for new API endpoints, consider using a schema [fixture](./te ```ruby expect(response).to match_response_schema('merge_requests') ``` + +Also see [verifying N+1 performance](#verifying-with-tests) in tests. diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 436cb43199e..bbd5ca3c494 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -314,7 +314,7 @@ You can use it either for personal or business websites, such as portfolios, doc - Layer: Core Service (Processor) - GitLab.com: [Runner](../user/gitlab_com/index.md#shared-runners) -GitLab Runner runs tests and sends the results to GitLab. +GitLab Runner runs jobs and sends the results to GitLab. GitLab CI/CD is the open-source continuous integration service included with GitLab that coordinates the testing. The old name of this project was `GitLab CI Multi Runner` but please use `GitLab Runner` (without CI) from now on. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index c8705a174af..13ff35ed65c 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -113,7 +113,7 @@ Stage labels respects the `devops::<stage_key>` naming convention. <https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml> with `_` replaced with a space. -For instance, the "Manage" stage is represented by the ~"devops::manage" label in +For instance, the "Manage" stage is represented by the `~"devops::manage"` label in the `gitlab-org` group since its key under `stages` is `manage`. The current stage labels can be found by [searching the labels list for `devops::`](https://gitlab.com/groups/gitlab-org/-/labels?search=devops::). @@ -156,10 +156,10 @@ As a team needs some way to collect the work their members are planning to be as Normally there is a 1:1 relationship between Stage labels and Group labels. In the spirit of "Everyone can contribute", any issue can be picked up by any group, depending on current priorities. When picking up an issue belonging to a different -group, it should be relabelled. For example, if an issue labelled ~"devops::create" -and ~"group::knowledge" is picked up by someone in the Access group of the Plan stage, -the issue should be relabelled as ~"group::access" while keeping the original -~"devops::create" unchanged. +group, it should be relabelled. For example, if an issue labelled `~"devops::create"` +and `~"group::knowledge"` is picked up by someone in the Access group of the Plan stage, +the issue should be relabelled as `~"group::access"` while keeping the original +`~"devops::create"` unchanged. We also use stage and group labels to help quantify our [throughput](https://about.gitlab.com/handbook/engineering/management/throughput/). Please read [Stage and Group labels in Throughput](https://about.gitlab.com/handbook/engineering/management/throughput/#stage-and-group-labels-in-throughput) for more information on how the labels are used in this context. diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index 975b524c1df..c1dfb220df8 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -379,6 +379,24 @@ are: To reduce unnecessary differences between two distribution methods, Omnibus and CNG **should always use the same Go version**. +## Secure Team standards and style guidelines + +The following are some style guidelines that are specific to the Secure Team. + +### Code style and format + +Use `goimports -local gitlab.com/gitlab-org` before committing. +[goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) +is a tool that automatically formats Go source code using +[Gofmt](https://golang.org/cmd/gofmt/), in addition to formatting import lines, +adding missing ones and removing unreferenced ones. +By using the `-local gitlab.com/gitlab-org` option, `goimports` will group locally referenced +packages separately from external ones. See +[the imports section](https://github.com/golang/go/wiki/CodeReviewComments#imports) +of the Code Review Comments page on the Go wiki for more details. +Most editors/IDEs will allow you to run commands before/after saving a file, you can set it +up to run `goimports -local gitlab.com/gitlab-org` so that it's applied to every file when saving. + --- [Return to Development documentation](../README.md). diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 91ca6120db9..7ddcd426fd7 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -306,6 +306,65 @@ This makes use of [`Intl.DateTimeFormat`](https://developer.mozilla.org/en-US/do ## Best practices +### Keep translations dynamic + +There are cases when it makes sense to keep translations together within an array or a hash. + +Examples: + +- Mappings for a dropdown list +- Error messages + +To store these kinds of data, using a constant seems like the best choice, however this won't work for translations. + +Bad, avoid it: + +```ruby +class MyPresenter + MY_LIST = { + key_1: _('item 1'), + key_2: _('item 2'), + key_3: _('item 3') + } +end +``` + +The translation method (`_`) will be called when the class is loaded for the first time and translates the text to the default locale. Regardless of what's the user's locale, these values will not be translated again. + +Similar thing happens when using class methods with memoization. + +Bad, avoid it: + +```ruby +class MyModel + def self.list + @list ||= { + key_1: _('item 1'), + key_2: _('item 2'), + key_3: _('item 3') + } + end +end +``` + +This method will memoize the translations using the locale of the user, who first "called" this method. + +To avoid these problems, keep the translations dynamic. + +Good: + +```ruby +class MyPresenter + def self.my_list + { + key_1: _('item 1'), + key_2: _('item 2'), + key_3: _('item 3') + }.freeze + end +end +``` + ### Splitting sentences Please never split a sentence as that would assume the sentence grammar and diff --git a/doc/development/img/architecture_simplified.png b/doc/development/img/architecture_simplified.png Binary files differindex 4899993310f..46ae2b3c055 100644 --- a/doc/development/img/architecture_simplified.png +++ b/doc/development/img/architecture_simplified.png diff --git a/doc/development/logging.md b/doc/development/logging.md index f10737da766..ef2d2d7022d 100644 --- a/doc/development/logging.md +++ b/doc/development/logging.md @@ -164,6 +164,14 @@ Resources: - [Elasticsearch mapping - avoiding type gotchas](https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping.html#_avoiding_type_gotchas) - [Elasticsearch mapping types]( https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) +#### Logging durations + +Similar to timezones, choosing the right time unit to log can impose avoidable overhead. So, whenever +challenged to choose between seconds, milliseconds or any other unit, lean towards _seconds_ as float. + +In order to make it easier to track timings in the logs, make sure the log key has `_s` as +suffix and `duration` within its name (e.g., `view_duration_s`). + ## Multi-destination Logging GitLab is transitioning from unstructured/plaintext logs to structured/JSON logs. During this transition period some logs will be recorded in multiple formats through multi-destination logging. diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index aecfe3af9c4..9ba6dfc110a 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -151,15 +151,15 @@ request, be sure to start the `dont-interrupt-me` job before pushing. ## PostgreSQL versions testing -We follow [the PostgreSQL versions Omnibus support policy](https://gitlab.com/groups/gitlab-org/-/epics/2184#proposal): - -| | 12.10 (April 2020) | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5 | 13.6 (November 2020) | 14.0 (May 2021?) | -| ------ | ------------------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | -------------------- | -| PG9.6 | nightly | - | - | - | - | - | - | - | -| PG10 | `master` | - | - | - | - | - | - | - | -| PG11 | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | nightly | - | -| PG12 | - | - | - | - | `master` | `master` | MRs/`master` | `master` | -| PG13 | - | - | - | - | - | - | - | MRs/`master` | +We follow the [PostgreSQL versions shipped with Omnibus GitLab](https://docs.gitlab.com/omnibus/package-information/postgresql_versions.html): + +| | 12.10 (April 2020) | 13.0 (May 2020) | 13.1 (June 2020) | 13.2 (July 2020) | 13.3 (August 2020) | 13.4, 13.5 | 13.6 (November 2020) | 14.0 (May 2021?) | +| ------ | ------------------ | --------------- | ---------------- | ---------------- | ------------------ | ------------ | -------------------- | ---------------- | +| PG9.6 | nightly | - | - | - | - | - | - | - | +| PG10 | `master` | - | - | - | - | - | - | - | +| PG11 | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | MRs/`master` | nightly | - | +| PG12 | - | - | - | - | `master` | `master` | MRs/`master` | `master` | +| PG13 | - | - | - | - | - | - | - | MRs/`master` | ## Directed acyclic graph diff --git a/doc/development/refactoring_guide/index.md b/doc/development/refactoring_guide/index.md new file mode 100644 index 00000000000..4bd9d0e9c11 --- /dev/null +++ b/doc/development/refactoring_guide/index.md @@ -0,0 +1,77 @@ +# Refactoring guide + +This document is a collection of techniques and best practices to consider while performing a refactor. + +## Pinning tests + +Pinning tests help you ensure that you don't unintentionally change the output or behavior of the entity you're refactoring. This even includes preserving any existing *buggy* behavior, since consumers may rely on those bugs implicitly. + +### Example steps + +1. Identify all the possible inputs to the refactor subject (e.g. anything that's injected into the template or used in a conditional). +1. For each possible input, identify the significant possible values. +1. Create a test to save a full detailed snapshot for each helpful combination values per input. This should guarantee that we have "pinned down" the current behavior. The snapshot could be literally a screenshot, a dump of HTML, or even an ordered list of debugging statements. +1. Run all the pinning tests against the code, before you start refactoring (Oracle) +1. Perform the refactor (or checkout the commit with the work done) +1. Run again all the pinning test against the post refactor code (Pin) +1. Compare the Oracle with the Pin. If the Pin is different, you know the refactoring doesn't preserve existing behavior. +1. Repeat the previous three steps as necessary until the refactoring is complete. + +### Example commit history + +Leaving in the commits for adding and removing pins helps others checkout and verify the result of the test. + +```bash +AAAAAA Add pinning tests to funky_foo +BBBBBB Refactor funky_foo into nice_foo +CCCCCC Remove pinning tests for funky_foo +``` + +Then you can leave a reviewer instructions on how to run the pinning test in your MR. Example: + +> First revert the commit which removes the pin. +> +> ```bash +> git revert --no-commit $(git log -1 --grep="Remove pinning test for funky_foo" --pretty=format:"%H") +> ``` +> +> Then run the test +> +> ```bash +> yarn run jest path/to/funky_foo_pin_spec.js +> ``` + +### Try to keep pins green + +It's hard for a refactor to be 100% pure. This means that a pin which captures absolutely everything is bound to fail with +some trivial and expected differences. Try to keep the pins green by cleaning the pin with the expected changes. This helps +others quickly verify that a refactor was safe. + +[Example](https://gitlab.com/gitlab-org/gitlab/-/commit/7b73da4078a60cf18f5c10c712c66c302174f506?merge_request_iid=29528#a061e6835fd577ccf6802c8a476f4e9d47466d16_0_23): + +```javascript +// funky_foo_pin_spec.js + +const cleanForSnapshot = el => { + Array.from(rootEl.querySelectorAll('[data-deprecated-attribute]')).forEach(el => { + el.removeAttribute('data-deprecated-attribute'); + }); +}; + +// ... + +expect(cleanForSnapshot(wrapper.element)).toMatchSnapshot(); +``` + +### Resources + +[Unofficial wiki explanation](http://wiki.c2.com/?PinningTests) + +### Examples + +- [Pinning test in a haml to vue refactor](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/27691#pinning-tests) +- [Pinning test in isolating a bug](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/32198#note_212736225) +- [Pinning test in refactoring dropdown](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28173) +- [Pinning test in refactoring vulnerability_details.vue](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25830/commits) +- [Pinning test in refactoring notes_award_list.vue](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29528#pinning-test) +- [Video of pair programming session on pinning tests](https://youtu.be/LrakPcspBK4) diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 429ec262250..2a3fcf122a6 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -206,12 +206,19 @@ Following you'll find some general common practices you will find as part of our When it comes to querying DOM elements in your tests, it is best to uniquely target the element, without adding additional attributes specifically for testing purposes. Sometimes this cannot be done feasibly. In these cases, adding test attributes to simplify the selectors might be the best option. -Preferentially, in component testing with `@vue/test-utils`, you should query for child components using the component itself. Otherwise, try to use an existing attribute like `name` or a Vue `ref` (if using `@vue/test-utils`): +Preferentially, in component testing with `@vue/test-utils`, you should query for child components using the component itself. This helps enforce that specific behavior can be covered by that component's individual unit tests. Otherwise, try to use: + +- A behavioral attribute like `name` (also verifies that `name` was setup properly) +- A `data-testid` attribute ([recommended by maintainers of `@vue/test-utils`](https://github.com/vuejs/vue-test-utils/issues/1498#issuecomment-610133465)) +- a Vue `ref` (if using `@vue/test-utils`) + +Examples: ```javascript it('exists', () => { wrapper.find(FooComponent); wrapper.find('input[name=foo]'); + wrapper.find('[data-testid="foo"]'); wrapper.find({ ref: 'foo'}); wrapper.find('.js-foo'); }); diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index d510dff82dd..48de5e274b0 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -4,8 +4,6 @@ type: howto # Installing GitLab HA on Amazon Web Services (AWS) -DANGER: **Danger:** This guide is under review and the steps below will be revised and updated in due time. For more detail, please see [this epic](https://gitlab.com/groups/gitlab-org/-/epics/912). - This page offers a walkthrough of a common HA (Highly Available) configuration for GitLab on AWS. You should customize it to accommodate your needs. diff --git a/doc/install/installation.md b/doc/install/installation.md index f6eeec11539..34228b2811e 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -134,7 +134,7 @@ Make sure you have the right version of Git installed: # Install Git sudo apt-get install -y git-core -# Make sure Git is version 2.26.0 or higher (minimal supported version is 2.22.0) +# Make sure Git is version 2.26.1 or higher (minimal supported version is 2.22.0) git --version ``` @@ -171,9 +171,9 @@ sudo make install # Download and compile from source cd /tmp -curl --remote-name --location --progress https://www.kernel.org/pub/software/scm/git/git-2.26.0.tar.gz -echo 'aa168c2318e7187cd295a645f7370cc6d71a324aafc932f80f00c780b6a26bed git-2.26.0.tar.gz' | shasum -a256 -c - && tar -xzf git-2.26.0.tar.gz -cd git-2.26.0/ +curl --remote-name --location --progress https://www.kernel.org/pub/software/scm/git/git-2.26.1.tar.gz +echo 'aa168c2318e7187cd295a645f7370cc6d71a324aafc932f80f00c780b6a26bed git-2.26.1.tar.gz' | shasum -a256 -c - && tar -xzf git-2.26.1.tar.gz +cd git-2.26.1/ ./configure --with-libpcre make prefix=/usr/local all diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index e98df17d944..24e4ed43543 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -216,6 +216,10 @@ For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`. ### Backup filename +CAUTION: **Warning:** +If you use a custom backup filename, you will not be able to +[limit the lifetime of the backups](#limit-backup-lifetime-for-local-files-prune-old-backups). + By default a backup file is created according to the specification in [the Backup timestamp](#backup-timestamp) section above. You can however override the `[TIMESTAMP]` part of the filename by setting the `BACKUP` environment variable. For example: ```shell @@ -585,50 +589,79 @@ For installations from source: ### Configuring cron to make daily backups -NOTE: **Note:** +CAUTION: **Warning:** The following cron jobs do not [backup your GitLab configuration files](#storing-configuration-files) or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). +You can schedule a cron job that backs up your repositories and GitLab metadata. + For Omnibus GitLab packages: -1. Edit `/etc/gitlab/gitlab.rb`: +1. Edit the crontab for the `root` user: - ```ruby - ## Limit backup lifetime to 7 days - 604800 seconds - gitlab_rails['backup_keep_time'] = 604800 + ```shell + sudo su - + crontab -e ``` -1. [Reconfigure GitLab] for the changes to take effect. +1. There, add the following line to schedule the backup for everyday at 2 AM: -Note that the `backup_keep_time` configuration option only manages local -files. GitLab does not automatically prune old files stored in a third-party -object storage (e.g., AWS S3) because the user may not have permission to list -and delete files. We recommend that you configure the appropriate retention -policy for your object storage. For example, you can configure [the S3 backup -policy as described here](https://stackoverflow.com/questions/37553070/gitlab-omnibus-delete-backup-from-amazon-s3). + ```plaintext + 0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1 + ``` -To schedule a cron job that backs up your repositories and GitLab metadata, use the root user: + NOTE: **Note** + For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`. -```shell -sudo su - -crontab -e -``` +For installations from source: -There, add the following line to schedule the backup for everyday at 2 AM: +1. Edit the crontab for the `git` user: -```plaintext -0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1 -``` + ```shell + sudo -u git crontab -e + ``` -NOTE: **Note** -For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`. +1. Add the following lines at the bottom: + + ```plaintext + # Create a full backup of the GitLab repositories and SQL database every day at 2am + 0 2 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 + ``` + +NOTE: **Note:** +The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. +This is recommended to reduce cron spam. -You may also want to set a limited lifetime for backups to prevent regular +### Limit backup lifetime for local files (prune old backups) + +CAUTION: **Warning:** +This will not work if you have used a [custom filename](#backup-filename) +for your backups. + +NOTE: **Note:** +This configuration option only manages local files. GitLab does not automatically +prune old files stored in a third-party [object storage](#uploading-backups-to-a-remote-cloud-storage) +because the user may not have permission to list and delete files. It is +recommended that you configure the appropriate retention policy for your object +storage (for example, [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html)). + +You may want to set a limited lifetime for backups to prevent regular backups using all your disk space. +For Omnibus GitLab packages: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + ## Limit backup lifetime to 7 days - 604800 seconds + gitlab_rails['backup_keep_time'] = 604800 + ``` + +1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + For installations from source: -1. Edit `home/git/gitlab/config/gitlab.yml`: +1. Edit `/home/git/gitlab/config/gitlab.yml`: ```yaml backup: @@ -638,20 +671,6 @@ For installations from source: 1. [Restart GitLab] for the changes to take effect. -```shell -sudo -u git crontab -e # Edit the crontab for the git user -``` - -Add the following lines at the bottom: - -```plaintext -# Create a full backup of the GitLab repositories and SQL database every day at 4am -0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 -``` - -The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. -This is recommended to reduce cron spam. - ## Restore GitLab provides a simple command line interface to restore your whole installation, diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md index 036dcf80416..d58eb130522 100644 --- a/doc/security/rate_limits.md +++ b/doc/security/rate_limits.md @@ -22,8 +22,9 @@ similarly mitigated by a rate limit. ## Admin Area settings +- [Issues rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md). - [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md). -- [Rate limits on raw endpoints](../user/admin_area/settings/rate_limits_on_raw_endpoints.md) +- [Raw endpoints rate limits](../user/admin_area/settings/rate_limits_on_raw_endpoints.md). - [Protected paths](../user/admin_area/settings/protected_paths.md). ## Rack Attack initializer diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index 63a10fdf4be..c81310edc44 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -308,6 +308,11 @@ Sg0KU1hNMGExaE9SVGR2V2pKQlBUMWNiaUo5DQo=', </details> +You can view the exact JSON payload in the administration panel. To view the payload: + +1. Navigate to **Admin Area > Settings > Metrics and profiling** and expand **Seat Links**. +1. Click **Preview payload**. + #### Disable Seat Link > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212375) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.10. diff --git a/doc/topics/airgap/index.md b/doc/topics/airgap/index.md index 44589c7e5f8..e712e3bb6b5 100644 --- a/doc/topics/airgap/index.md +++ b/doc/topics/airgap/index.md @@ -18,7 +18,7 @@ Follow these best practices to use GitLab's features in an offline environment: To use many GitLab features, including [security scans](../../user/application_security/index.md#working-in-an-offline-environment) -and [Auto Devops](../autodevops/), the GitLab Runner must be able to fetch the +and [Auto DevOps](../autodevops/), the GitLab Runner must be able to fetch the relevant Docker images. The process for making these images available without direct access to the public internet diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index d373c3212c3..7c587ad3444 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -218,6 +218,16 @@ include: See the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) for information on available jobs. +CAUTION: **Deprecation** +Auto DevOps templates using the [`only`](../../ci/yaml/README.md#onlyexcept-basic) or +[`except`](../../ci/yaml/README.md#onlyexcept-basic) syntax will switch +to the [`rules`](../../ci/yaml/README.md#rules) syntax, starting in +[GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213336). +If your `.gitlab-ci.yml` extends these Auto DevOps templates and override the `only` or +`except` keywords, you must migrate your templates to use the +[`rules`](../../ci/yaml/README.md#rules) syntax after the +base template is migrated to use the `rules` syntax. + ## PostgreSQL database support To support applications requiring a database, diff --git a/doc/topics/autodevops/img/guide_project_landing_page_v12_10.png b/doc/topics/autodevops/img/guide_project_landing_page_v12_10.png Binary files differindex ec646c50a20..54e7141dad2 100644 --- a/doc/topics/autodevops/img/guide_project_landing_page_v12_10.png +++ b/doc/topics/autodevops/img/guide_project_landing_page_v12_10.png diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 3fb4aa64f3f..53d5e664bc1 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -64,8 +64,6 @@ to deploy this project to. 1. On the **Add a Kubernetes cluster integration** page, click the **Create new cluster** tab, then click **Google GKE**. -  - 1. Connect with your Google account, and click **Allow** to allow access to your Google account. (This authorization request is only displayed the first time you connect GitLab with your Google account.) @@ -86,7 +84,8 @@ to deploy this project to. - **Number of nodes** - **Machine type** - For more information about [machine types](https://cloud.google.com/compute/docs/machine-types), see Google's documentation. - - **Enable Cloud Run for Anthos** - Select this checkbox to use the Cloud Run, + - **Enable Cloud Run for Anthos** - Select this checkbox to use the + [Cloud Run](../../user/project/clusters/add_gke_clusters.md#cloud-run-for-anthos), Istio, and HTTP Load Balancing add-ons for this cluster. - **GitLab-managed cluster** - Select this checkbox to [allow GitLab to manage namespace and service accounts](../..//user/project/clusters/index.md#gitlab-managed-clusters) for this cluster. @@ -184,7 +183,7 @@ The jobs are separated into stages: susceptible to vulnerabilities and is allowed to fail ([Auto Dependency Scanning](stages.md#auto-dependency-scanning-ultimate)) **(ULTIMATE)** - The `sast` job runs static analysis on the current code to check for potential - security issues and is allowed to fail([Auto SAST](stages.md#auto-sast-ultimate)) **(ULTIMATE)** + security issues and is allowed to fail ([Auto SAST](stages.md#auto-sast-ultimate)) **(ULTIMATE)** - The `license_management` job searches the application's dependencies to determine each of their licenses and is allowed to fail ([Auto License Compliance](stages.md#auto-license-compliance-ultimate)) **(ULTIMATE)** @@ -211,15 +210,17 @@ you to common environment tasks:  -- **{external-link}** **Open live environment** - Opens the URL of the application deployed in production -- **{chart}** **Monitoring** - Opens the metrics page where Prometheus collects data +- **Open live environment** (**{external-link}**) - Opens the URL of the application deployed in production +- **Monitoring** (**{chart}**) - Opens the metrics page where Prometheus collects data about the Kubernetes cluster and how the application affects it in terms of memory usage, CPU usage, and latency -- **{play}** **{angle-down}** **Deploy to** - Displays a list of environments you can deploy to -- **{terminal}** **Terminal** - Opens a [web terminal](../../ci/environments.md#web-terminals) +- **Deploy to** (**{play}** **{angle-down}**) - Displays a list of environments you can deploy to +- **Terminal** (**{terminal}**) - Opens a [web terminal](../../ci/environments.md#web-terminals) session inside the container where the application is running -- **{repeat}** **Re-deploy to environment** -- **{stop}** **Stop environment** +- **Re-deploy to environment** (**{repeat}**) - For more information, see + [Retrying and rolling back](../../ci/environments.md#retrying-and-rolling-back) +- **Stop environment** (**{stop}**) - For more information, see + [Stopping an environment](../../ci/environments.md#stopping-an-environment) GitLab displays the [Deploy Board](../../user/project/deploy_boards.md) below the environment's information, with squares representing pods in your @@ -279,13 +280,13 @@ To fix the broken test: 1. Change line 7 to say `You're on Rails! Powered by GitLab Auto DevOps.` 1. Click **Commit**. 1. In the left-hand column, under **Unstaged changes**, click the checkmark icon - to stage the changes. + (**{stage-all}**) to stage the changes. 1. Write a commit message, and click **Commit**. Return to the **Overview** page of your merge request, and you should not only see the test passing, but also the application deployed as a [review application](stages.md#auto-review-apps). You can visit it by clicking -the **View app** button to see your changes deployed. +the **View app** **{external-link}** button to see your changes deployed.  diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 72fa3870abd..66b76dcc05a 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -474,7 +474,7 @@ To enable ModSecurity with Auto Deploy, you need to create a `.gitlab/auto-deplo |Attribute | Description | Default | -----------|-------------|---------| -|`enabled` | Enables custom configuration for modsecurity, defaulting to the [Core Rule Set](https://coreruleset.org/) | `false` | +|`enabled` | Enables custom configuration for ModSecurity, defaulting to the [Core Rule Set](https://coreruleset.org/) | `false` | |`secRuleEngine` | Configures the [rules engine](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#secruleengine) | `DetectionOnly` | |`secRules` | Creates one or more additional [rule](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRule) | `nil` | diff --git a/doc/topics/web_application_firewall/quick_start_guide.md b/doc/topics/web_application_firewall/quick_start_guide.md index 6483a56e7f7..d55ab03a3f2 100644 --- a/doc/topics/web_application_firewall/quick_start_guide.md +++ b/doc/topics/web_application_firewall/quick_start_guide.md @@ -213,7 +213,7 @@ the WAF with OWASP CRS! ## Testing out the OWASP Core Rule Set Now let's send a potentially malicious request, as if we were a scanner, -checking for vulnerabilities within our application and examine the modsecurity logs: +checking for vulnerabilities within our application and examine the ModSecurity logs: ```shell $ curl --location --insecure fjdiaz-auto-devv-2.34.68.60.207.nip.io --header "User-Agent: absinthe" | grep 'Rails!' --after 2 --before 2 diff --git a/doc/user/admin_area/settings/img/rate_limit_on_issues_creation.png b/doc/user/admin_area/settings/img/rate_limit_on_issues_creation.png Binary files differnew file mode 100644 index 00000000000..5aa9b95f835 --- /dev/null +++ b/doc/user/admin_area/settings/img/rate_limit_on_issues_creation.png diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md new file mode 100644 index 00000000000..96a20681b2f --- /dev/null +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -0,0 +1,25 @@ +--- +type: reference +--- + +# Rate limits on issue creation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/55241) in GitLab 12.10. + +This setting allows you to rate limit the requests to the issue creation endpoint. +It defaults to 300 requests per minute. +You can change it in **Admin Area > Settings > Network > Performance Optimization**. + +For example, requests using the +[Projects::IssuesController#create](https://gitlab.com/gitlab-org/gitlab/raw/master/app/controllers/projects/issues_controller.rb) +action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. + + + +This limit is: + +- Applied independently per project and per user. +- Not applied per IP address. +- Active by default. To disable it, set the option to `0`. + +Requests over the rate limit are logged into the `auth.log` file. diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index f28bab6ad86..7869f7de1b6 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -222,6 +222,7 @@ but commented out to help encourage others to add to it in the future. --> |issues_with_associated_zoom_link|counts|| |issues_using_zoom_quick_actions|counts|| |issues_with_embedded_grafana_charts_approx|counts|| +|issues_with_health_status|counts|| |keys|counts|| |label_lists|counts|| |lfs_objects|counts|| diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 703b794981f..32f393d342b 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -280,6 +280,28 @@ administrator can open a Rails console and disable it with the following command Feature.disable(:cycle_analytics_scatterplot_median_enabled) ``` +## Type of work - Tasks by type chart + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32421) in GitLab 12.10. + +This chart shows a cumulative count of issues and merge requests per day. + +This chart uses the global page filters for displaying data based on the selected +group, projects, and timeframe. The chart defaults to showing counts for issues but can be +toggled to show data for merge requests and further refined for specific group-level labels. + +By default the top group-level labels (max. 10) are pre-selected, with the ability to +select up to a total of 15 labels. + +### Disabling chart + +This chart is enabled by default. If you have a self-managed instance, an +administrator can open a Rails console and disable it with the following command: + +```ruby +Feature.disable(:tasks_by_type_chart) +``` + ## Permissions The current permissions on the Project Value Stream Analytics dashboard are: diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 27b22fb925c..68ad2d427dd 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -187,6 +187,10 @@ using environment variables. ### Overriding the Container Scanning template +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + If you want to override the job definition (for example, change properties like `variables`), you need to declare a `container_scanning` job after the template inclusion and specify any additional keys under it. For example: @@ -212,11 +216,46 @@ If you want to whitelist specific vulnerabilities, you'll need to: ### Running Container Scanning in an offline environment -Container Scanning can be executed on an offline GitLab Ultimate installation by using the following process: +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access +to external resources through the internet, some adjustments are required for the Container Scanning job to +successfully run. For more information, see [Offline environments](../offline_deployments/index.md). + +#### Requirements for offline Container Scanning + +To use Container Scanning in an offline environment, you need: + +- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- To configure a local Docker Container Registry with copies of the Container Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [Container Scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). + +NOTE: **Note:** +GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +meaning the runner may try to pull remote images even if a local copy is available. Set GitLab +Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. + +#### Make GitLab Container Scanning analyzer images available inside your Docker registry -1. Host the following Docker images on a [local Docker container registry](../../packages/container_registry/index.md): - - [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db) - - GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar` +For Container Scanning, import and host the following images from `registry.gitlab.com` to your +offline [local Docker container registry](../../packages/container_registry/index.md): + +- [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db) +- GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar` + +The process for importing Docker images into a local offline Docker registry depends on +**your network security policy**. Please consult your IT staff to find an accepted and approved +process by which external resources can be imported or temporarily accessed. + +Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you are able to make periodic updates yourself. +You can read more specific steps on how to do this [below](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). + +For details on saving and transporting Docker images as a file, see Docker's documentation on +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). + +#### Set Container Scanning CI job variables to use local Container Scanner analyzers + +Container Scanning can be executed on an offline GitLab Ultimate installation using the following process: 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: @@ -234,7 +273,12 @@ Container Scanning can be executed on an offline GitLab Ultimate installation by self-signed certificate, then you must set `DOCKER_INSECURE: "true"` in the above `container_scanning` section of your `.gitlab-ci.yml`. -It may be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to automatically build a new version of the vulnerabilities database on a preset schedule. You can use the following `.gitlab-yml.ci` as a template: +#### Automating Container Scanning vulnerability database updates with a pipeline + +It can be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to +automatically build a new version of the vulnerabilities database on a preset schedule. Automating +this with a pipeline means you won't have to do it manually each time. You can use the following +`.gitlab-yml.ci` as a template: ```yaml image: docker:stable diff --git a/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png b/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png Binary files differindex c15a2da513c..9f277dcb578 100644 --- a/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png +++ b/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 6f51aaf4931..abf194aae48 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -376,6 +376,10 @@ configuration, the last mention of the variable will take precedence. ### Overriding the DAST template +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + If you want to override the job definition (for example, change properties like `variables` or `dependencies`), you need to declare a `dast` job after the template inclusion and specify any additional keys under it. For example: diff --git a/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png b/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png Binary files differnew file mode 100644 index 00000000000..2755b42f1e4 --- /dev/null +++ b/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index b9c3b6521d6..73d2cfeaf00 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -17,32 +17,25 @@ sidebar. This information is sometimes referred to as a Software Bill of Materia ## Viewing dependencies - + Dependencies are displayed with the following information: | Field | Description | | --------- | ----------- | -| Status | Displays whether or not the dependency has any known vulnerabilities | -| Component | The dependency's name | -| Version | The exact locked version of the dependency your project uses | +| Component | The dependency's name and version | | Packager | The packager used to install the depedency | | Location | A link to the packager-specific lockfile in your project that declared the dependency | | License | Links to dependency's software licenses | -Dependencies shown are initially sorted by their names. They can also be sorted -by the packager they were installed by, or by the severity of their known -vulnerabilities. - -There is a second list under the `Vulnerable components` tab displaying only -those dependencies with known vulnerabilities. If there are none, this tab is -disabled. +Dependencies shown are initially sorted by the severity of their known vulnerabilities, if any. They +can also be sorted by name or by the packager that installed them. ### Vulnerabilities -If a dependency has known vulnerabilities, they can be viewed by clicking on the -`Status` cell of that dependency. The severity and description of each -vulnerability will then be displayed below it. +If a dependency has known vulnerabilities, you can view them by clicking the arrow next to the +dependency's name or the badge that indicates how many known vulnerabilities exist. For each +vulnerability, its severity and description then appears below it. ## Licenses diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index ae006178945..cda621e61a6 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -115,6 +115,10 @@ configuration, the last mention of the variable will take precedence. ### Overriding the Dependency Scanning template +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + If you want to override the job definition (for example, change properties like `variables` or `dependencies`), you need to declare a `dependency_scanning` job after the template inclusion and specify any additional keys under it. For example: @@ -175,6 +179,8 @@ The following variables are used for configuring specific analyzers (used for a | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) | | `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)| | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repos](../index.md#using-private-maven-repos). | +| `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | +| `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | | `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| | `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | @@ -415,6 +421,181 @@ You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-product to find a vulnerability in the Gemnasium database. You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md). +## Running Dependency Scanning in an offline environment + +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access +to external resources through the internet, some adjustments are required for dependency scannings jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). + +### Requirements for offline Dependency Scanning + +Here are the requirements for using Dependency Scanning in an offline environment: + +- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-dependency-scanning) +- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. +- Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/) +- _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). +- _Only if scanning npm/yarn projects_: Host an offline copy of the [retire.js](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [js](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. + +NOTE: **Note:** +GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +meaning the runner will try to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend keeping the pull policy setting to `always` as it will better enable updated scanners to +be utilized within your CI/CD pipelines. + +### Make GitLab Dependency Scanning analyzer images available inside your Docker registry + +For Dependency Scanning, import docker images ([supported languages and frameworks](#supported-languages-and-package-managers)) +from `registry.gitlab.com` to your offline docker registry. The Dependency Scanning analyzer +docker images are: + +```plaintext +registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 +registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven:2 +registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2 +registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2 +registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 +``` + +The process for importing Docker images into a local offline Docker registry depends on +**your network security policy**. Please consult your IT staff to find an accepted and approved +process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you are able to make periodic updates yourself. + +For details on saving and transporting Docker images as a file, see Docker's documentation on +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). + +### Set Dependency Scanning CI config for "offline" use + +Below is a general `.gitlab-ci.yml` template to configure your environment for running Dependency +Scanning offline: + +```yaml +include: + - template: Dependency-Scanning.gitlab-ci.yml + +variables: + DS_DISABLE_DIND: "true" + DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers" +``` + +See explanations of the variables above in the [configuration section](#configuration). + +### Specific settings for languages and package managers + +For every language and package manager, add the following to the variables section of +`.gitlab-ci.yml`: + +```yaml +GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" +``` + +See the following sections for additional instructions on specific languages and package managers. + +#### JavaScript (npm and yarn) projects + +Add the following to the variables section of `.gitlab-ci.yml`: + +```yaml +RETIREJS_JS_ADVISORY_DB: "example.com/jsrepository.json" +RETIREJS_NODE_ADVISORY_DB: "example.com/npmrepository.json" +``` + +#### Ruby (gem) projects + +Add the following to the variables section of `.gitlab-ci.yml`: + +```yaml +BUNDLER_AUDIT_ADVISORY_DB_REF_NAME: "master" +BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git" +``` + +#### Java (Maven) projects + +When using a self-signed certificates, add the following to the variables section of`.gitlab-ci.yml`: + +```yaml +MAVEN_CLI_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"` +``` + +#### Java (Gradle) projects + +When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`: + +```yaml +gemnasium-maven-dependency_scanning: + variables: + before_script: + - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt +``` + +This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. + +#### Scala (sbt) projects + +When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`: + +```yaml +gemnasium-maven-dependency_scanning: + variables: + before_script: + - echo -n | openssl s_client -connect gitlab-airgap-test.us-west1-b.c.group-secure-a89fe7.internal:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt +``` + +This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. + +#### Python (pip) and Python (Pipfile) projects + +Add the following `pip.conf` to your repository to define your index URL and trust its self-signed +certificate: + +```toml +[global] +index-url = https://pypi.example.com +trusted-host = pypi.example.com +``` + +Add the following job section to `.gitlab-ci.yml`: + +```yaml +gemnasium-python-dependency_scanning: + before_script: + - mkdir ~/.config/pip + - cp pip.conf ~/.config/pip/pip.conf +``` + +#### Python (setuptools) + +When using self-signed certificates for your private PyPi repo no extra job configuration (aside +from the template `.gitlab-ci.yml` above) is needed. However, you must update your `setup.py` to +ensure that it can reach your private repo. Here is an example configuration: + +1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repo for each + dependency in the `install_requires` list: + + ```python + install_requires=['pyparsing>=2.0.3'], + dependency_links=['https://pypi.example.com/simple/pyparsing'], + ``` + +1. Fetch the certificate from your repository URL and add it to the project: + + ```bash + echo -n | openssl s_client -connect pypi.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > internal.crt + ``` + +1. Point `setup.py` at the newly downloaded certificate: + + ```python + import setuptools.ssl_support + setuptools.ssl_support.cert_paths = ['internal.crt'] + ``` + ## Troubleshooting ### Error response from daemon: error processing tar file: docker-tar: relocation error diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 011f95c7049..a6457d58fe2 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -143,6 +143,10 @@ the pipeline configuration, the last mention of the variable will take precedenc ### Overriding the SAST template +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + If you want to override the job definition (for example, change properties like `variables` or `dependencies`), you need to declare a `sast` job after the template inclusion and specify any additional keys under it. For example: diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png Binary files differnew file mode 100644 index 00000000000..07b41b471d4 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 1eef6b9b696..42b28b7b9f2 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -34,13 +34,13 @@ To use the instance, group, project, or pipeline security dashboard: 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used. If you're using the shared Runners on GitLab.com, this is already the case. -## Pipeline Security Dashboard +## Pipeline Security > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13496) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3. -At the pipeline level, the Security Dashboard displays the vulnerabilities present in the branch of the project the pipeline was run against. +At the pipeline level, the Security section displays the vulnerabilities present in the branch of the project the pipeline was run against. -Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security Dashboard. +Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security findings.  @@ -54,6 +54,18 @@ for your project from the last successful pipeline. Use it to find and fix vulne  +### Export vulnerabilities + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. + +You can export all your project's vulnerabilities as CSV by clicking on the export button located at top right of the Project Security Dashboard. This will initiate the process, and once complete, the CSV report will be downloaded. The report will contain all vulnerabilities in the project as filters won't apply. + +NOTE: **Note:** +It may take several minutes for the download to start if your project consists +of thousands of vulnerabilities. Do not close the page until the download finishes. + + + ## Group Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6709) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5. diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index cc7b5dcd5fb..47cbc0d4a1e 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -553,12 +553,12 @@ To enable Fluentd: 1. Navigate to **{cloud-gear}** **Operations > Kubernetes** and click **Applications**. You will be prompted to enter a host, port and protocol where the WAF logs will be sent to via syslog. -1. Provide the host domain name or URL in **SIEM URL or Host**. +1. Provide the host domain name or URL in **SIEM Hostname**. 1. Provide the host port number in **SIEM Port**. 1. Select a **SIEM Protocol**. 1. Check **Send ModSecurity Logs**. If you do not select this checkbox, the **Install** button is disabled. -1. Click **Install**. +1. Click **Save changes**.  @@ -592,6 +592,7 @@ Supported applications: - [Elastic Stack](#install-elastic-stack-using-gitlab-cicd) - [Crossplane](#install-crossplane-using-gitlab-cicd) - [Fluentd](#install-fluentd-using-gitlab-cicd) +- [Knative](#install-knative-using-gitlab-cicd) ### Usage @@ -652,7 +653,7 @@ ingress: Ingress will then be installed into the `gitlab-managed-apps` namespace of your cluster. -You can customize the installation of Ingress by defining +You can customize the installation of Ingress by defining a `.gitlab/managed-apps/ingress/values.yaml` file in your cluster management project. Refer to the [chart](https://github.com/helm/charts/tree/master/stable/nginx-ingress) @@ -689,7 +690,7 @@ certManager: installed: false ``` -You can customize the installation of cert-manager by defining +You can customize the installation of cert-manager by defining a `.gitlab/managed-apps/cert-manager/values.yaml` file in your cluster management project. Refer to the [chart](https://hub.helm.sh/charts/jetstack/cert-manager) for the @@ -1054,7 +1055,7 @@ In this alpha implementation of installing Elastic Stack through CI, reading the ### Install Crossplane using GitLab CI/CD -> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/merge_requests/68) in GitLab 12.9. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/35675) in GitLab 12.9. Crossplane is installed using GitLab CI/CD by defining configuration in `.gitlab/managed-apps/config.yaml`. @@ -1068,13 +1069,15 @@ Crossplane: Crossplane is installed into the `gitlab-managed-apps` namespace of your cluster. -You can check the default [values.yaml](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) we set for this chart. +You can check the default +[values.yaml](https://github.com/crossplane/crossplane/blob/master/cluster/charts/crossplane/values.yaml.tmpl) +we set for this chart. You can customize the installation of Crossplane by defining `.gitlab/managed-apps/crossplane/values.yaml` file in your cluster management project. Refer to the [chart](https://github.com/crossplane/crossplane/tree/master/cluster/charts/crossplane#configuration) for the -available configuration options. Note that this link points to the docs for the current development release, which may differ from the version you have installed. You can check out a specific version in the branch/tag switcher. +available configuration options. Note that this link points to the documentation for the current development release, which may differ from the version you have installed. ### Install Fluentd using GitLab CI/CD @@ -1100,6 +1103,48 @@ The configuration chart link points to the current development release, which may differ from the version you have installed. To ensure compatibility, switch to the specific branch or tag you are using. +### Install Knative using GitLab CI/CD + +To install Knative, define the `.gitlab/managed-apps/config.yaml` file +with: + +```yaml +knative: + installed: true +``` + +You can customize the installation of Knative by defining `.gitlab/managed-apps/knative/values.yaml` +file in your cluster management project. Refer to the [chart](https://gitlab.com/gitlab-org/charts/knative) +for the available configuration options. + +Here is an example configuration for Knative: + +```yaml +domain: 'my.wildcard.A.record.dns' +``` + +If you plan to use GitLab Serverless capabilities, be sure to set an A record wildcard domain on your custom configuration. + +#### Knative Metrics + +GitLab provides [Invocation Metrics](../project/clusters/serverless/index.md#invocation-metrics) for your functions. To collect these metrics, you must have: + +1. Knative and Prometheus managed applications installed on your cluster. +1. Manually applied the custom metrics on your cluster by running the following command: + + ```bash + kubectl apply -f https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/raw/02c8231e30ef5b6725e6ba368bc63863ceb3c07d/src/default-data/knative/istio-metrics.yaml + ``` + +#### Uninstall Knative + +To uninstall Knative, you must first manually remove any custom metrics you have added +by running the following command: + +```bash +kubectl delete -f https://gitlab.com/gitlab-org/cluster-integration/cluster-applications/-/raw/02c8231e30ef5b6725e6ba368bc63863ceb3c07d/src/default-data/knative/istio-metrics.yaml +``` + ## Upgrading applications > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24789) in GitLab 11.8. diff --git a/doc/user/clusters/img/fluentd_v12_10.png b/doc/user/clusters/img/fluentd_v12_10.png Binary files differindex 7593f99ab51..e8c5c832020 100644 --- a/doc/user/clusters/img/fluentd_v12_10.png +++ b/doc/user/clusters/img/fluentd_v12_10.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 9fcc9acf5ea..2e771a17163 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -154,6 +154,10 @@ directory of your project. ### Overriding the template +CAUTION: **Deprecation:** +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. + If you want to override the job definition (for example, change properties like `variables` or `dependencies`), you need to declare a `license_scanning` job after the template inclusion and specify any additional keys under it. For example: @@ -301,29 +305,69 @@ license_scanning: ## Running License Compliance in an offline environment -License Compliance can be executed on an offline GitLab Ultimate installation by using the following -process: +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access +to external resources through the internet, some adjustments are required for the License Compliance job to +successfully run. + +### Requirements for offline License Compliance + +To use License Compliance in an offline environment, you need: + +- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). +- Docker Container Registry with locally available copies of License Compliance [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. + +NOTE: **Note:** +GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), +meaning the runner will try to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend leaving the pull policy set to `always`, as it better enables updated scanners to be used +within your CI/CD pipelines. + +### Make GitLab License Compliance analyzer images available inside your Docker registry + +For License Compliance with all [supported languages and package managers](#supported-languages-and-package-managers), +import the following default License Compliance analyzer images from `registry.gitlab.com` to your +offline [local Docker container registry](../../packages/container_registry/index.md): + +```plaintext +registry.gitlab.com/gitlab-org/security-products/license-management:latest +``` + +The process for importing Docker images into a local offline Docker registry depends on +**your network security policy**. Please consult your IT staff to find an accepted and approved +process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../../application_security/index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you are able to make periodic updates yourself. + +For details on saving and transporting Docker images as a file, see Docker's documentation on +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). + +### Set License Compliance CI job variables to use local License Compliance analyzers -1. Host the License Compliance image - `registry.gitlab.com/gitlab-org/security-products/license-management:latest` in your local Docker - container registry. -1. Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer - to the License Compliance Docker image hosted on your local Docker container registry: +Override License Compliance environment variables to use to your local container registry +as the source for License Compliance analyzer images. - ```yaml - include: - - template: License-Scanning.gitlab-ci.yml +For example, this assumes a local Docker registry repository of `localhost:5000/analyzers`: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml - license_scanning: - image: registry.example.com/namespace/license-management:latest - ``` +license_scanning: + image: + name: localhost:5000/analyzers/license-management:latest +``` -1. Ensure the package registry is reachable from within the GitLab environment and that the package - manager is configured to use your preferred package registry. +The License Compliance job should now use local copies of the License Compliance analyzers to scan +your code and generate security reports, without requiring internet access. Additional [configuration](#using-private-maven-repos) may be needed for connecting to private Maven repositories. +Exact name matches are required for [project policies](#project-policies-for-license-compliance) +when running in an offline environment ([see related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/212388)). + ## Project policies for License Compliance > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5940) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. diff --git a/doc/user/group/img/group_activity_analytics_v12_10.png b/doc/user/group/img/group_activity_analytics_v12_10.png Binary files differnew file mode 100644 index 00000000000..e49594c155b --- /dev/null +++ b/doc/user/group/img/group_activity_analytics_v12_10.png diff --git a/doc/user/group/index.md b/doc/user/group/index.md index fdcc4105620..b819e3e971e 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -226,16 +226,42 @@ To change this setting for a specific group: To change this setting globally, see [Default project creation protection](../admin_area/settings/visibility_and_access_controls.md#default-project-creation-protection). -## Viewing group activity +## View group details + +A group's **Details** page includes tabs for: + +- Subgroups and projects. +- Shared projects. +- Archived projects. + +### Group activity analytics overview + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab [Starter](https://about.gitlab.com/pricing/) 12.10 as +a [beta feature](https://about.gitlab.com/handbook/product/#beta) + +The group details view also shows the number of the following items created in the last 90 days: **(STARTER)** + +- Merge requests. +- Issues. +- Members. + +These Group Activity Analytics can be enabled with the `group_activity_analytics` [feature flag](../../development/feature_flags/development.md#enabling-a-feature-flag-in-development). + + + +For details, see the section on how you can [View group activity](#view-group-activity). + +## View group activity A group's **Activity** page displays the most recent actions taken in a group, including: -- **Push events**: Recent pushes to branches -- **Merge events**: Recent merges -- **Issue events**: Issues opened or closed -- **Epic events**: Epics opened or closed -- **Comments**: Comments opened or closed -- **Team**: Team members who have joined or left the group +- **Push events**: Recent pushes to branches. +- **Merge events**: Recent merges. +- **Issue events**: Issues opened or closed. +- **Epic events**: Epics opened or closed. +- **Comments**: Comments opened or closed. +- **Team**: Team members who have joined or left the group. +- **Wiki**: Wikis created, deleted, or updated. The entire activity feed is also available in Atom format by clicking the **RSS** icon. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index e78a894d987..f49dd225146 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -242,7 +242,7 @@ For a demo of the Azure SAML setup including SCIM, see [SCIM Provisioning on Azu |--------------|----------------| | Identifier | Identifier (Entity ID) | | Assertion consumer service URL | Reply URL (Assertion Consumer Service URL) | -| Identity provider single sign on URL | Login URL | +| Identity provider single sign on URL | Sign on URL | | Certificate fingerprint | Thumbprint | We recommend: diff --git a/doc/user/incident_management/index.md b/doc/user/incident_management/index.md index 2b95128bffd..8505afb375e 100644 --- a/doc/user/incident_management/index.md +++ b/doc/user/incident_management/index.md @@ -45,7 +45,7 @@ The emails will be sent to [owners and maintainers](../permissions.md) of the pr Prometheus alerts can be set up in both: -- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics-ultimate) and +- [GitLab-managed Prometheus](../project/integrations/prometheus.md#setting-up-alerts-for-prometheus-metrics) and - [Self-managed Prometheus](../project/integrations/prometheus.md#external-prometheus-instances) installations. ### Alert endpoint diff --git a/doc/user/packages/img/group_packages_list_v12_10.png b/doc/user/packages/img/group_packages_list_v12_10.png Binary files differnew file mode 100644 index 00000000000..ba9f2892961 --- /dev/null +++ b/doc/user/packages/img/group_packages_list_v12_10.png diff --git a/doc/user/packages/img/package_activity_v12_10.png b/doc/user/packages/img/package_activity_v12_10.png Binary files differnew file mode 100644 index 00000000000..4fea9a7ca3f --- /dev/null +++ b/doc/user/packages/img/package_activity_v12_10.png diff --git a/doc/user/packages/img/package_detail_v12_10.png b/doc/user/packages/img/package_detail_v12_10.png Binary files differnew file mode 100644 index 00000000000..b2cd8e31955 --- /dev/null +++ b/doc/user/packages/img/package_detail_v12_10.png diff --git a/doc/user/packages/img/project_packages_list_v12_10.png b/doc/user/packages/img/project_packages_list_v12_10.png Binary files differnew file mode 100644 index 00000000000..2dfb92fa796 --- /dev/null +++ b/doc/user/packages/img/project_packages_list_v12_10.png diff --git a/doc/user/packages/index.md b/doc/user/packages/index.md index 78ddc06173c..8e98dd70346 100644 --- a/doc/user/packages/index.md +++ b/doc/user/packages/index.md @@ -14,6 +14,85 @@ The Packages feature allows GitLab to act as a repository for the following: | [Maven Repository](maven_repository/index.md) **(PREMIUM)** | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | | [NPM Registry](npm_registry/index.md) **(PREMIUM)** | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | | [NuGet Repository](nuget_repository/index.md) **(PREMIUM)** | The GitLab NuGet Repository will enable every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.8+ | +| [PyPi Repository](pypi_repository/index.md) **(PREMIUM)** | The GitLab PyPi Repository will enable every project in GitLab to have its own space to store [PyPi](https://pypi.org/) packages. | 12.10+ | + +## Enable the Package Registry for your project + +If you cannot find the **{package}** **Packages > List** entry under your +project's sidebar, it is not enabled in your GitLab instance. Ask your +administrator to enable GitLab Package Registry following the administration +documentation. + +Once enabled for your GitLab instance, to enable Package Registry for your +project: + +1. Go to your project's **Settings > General** page. +1. Expand the **Visibility, project features, permissions** section and enable the +**Packages** feature on your project. +1. Press **Save changes** for the changes to take effect. You should now be able to +see the **Packages > List** link in the sidebar. + +### View Packages for your project + +Navigating to your project's **{package}** **Packages > List** will show a list +of all packages that have been added to your project. + + + +On this page, you can: + +- View all the packages that have been uploaded to the project. +- Sort the packages list by created date, version or name. +- Filter the list by package name. +- Change tabs to display packages of a certain type. +- Remove a package (if you have suitable [permissions](../permissions.md)). +- Navigate to specific package detail page. + +### View Packages for your group + +You can view all packages belonging to a group by navigating to **{package}** +**Packages > List** from the group sidebar. + + + +On this page, you can: + +- View all the packages that have been uploaded to each of the groups projects. +- Sort the packages list by created date, version, name or project. +- Filter the list by package name. +- Change tabs to display packages of a certain type. +- Navigate to specific package detail page. + +### View additional package information + +Additional package information can be viewed by browsing to the package details +page from the either the project or group list. + + + +On this page you can: + +- See the extended package information, including metadata. This is unique to +each package type and will display different information for different types. +- View quick installation and registry setup instructions. These are a shortcut +for users who have already set up the Package Registry and just want quick +installation instructions. +- View the package activity, including when and how a package was published. +- View and download the contents of the package. Outside of installing a +package via a manager, you can also download the files individually. +- Delete the package (if you have suitable [permissions](../permissions.md)). + +### Build packages via GitLab CI/CD + +Some of the supported packages can be built via [GitLab CI/CD](./../../ci/README.md) +using the `CI_JOB_TOKEN`. If a package is built this way, then extended activity +information is displayed on the package details page: + + + +You can view which pipeline published the package, as well as the commit and +user who triggered it. To see if a package type supports being built via CI/CD, +check the specific documentation for your package type. ## Suggested contributions diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md new file mode 100644 index 00000000000..11d7b828813 --- /dev/null +++ b/doc/user/packages/pypi_repository/index.md @@ -0,0 +1,84 @@ +# GitLab PyPi Repository **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208747) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.10. + +With the GitLab PyPi Repository, every project can have its own space to store PyPi packages. + +The GitLab PyPi Repository works with: + +- [pip](https://pypi.org/project/pip/) +- [twine](https://pypi.org/project/twine/) + +## Setting up your development environment + +You will need a recent version of [pip](https://pypi.org/project/pip/) and [twine](https://pypi.org/project/twine/). + +## Enabling the PyPi Repository + +NOTE: **Note:** +This option is available only if your GitLab administrator has +[enabled support for the Package Registry](../../../administration/packages/index.md). **(PREMIUM ONLY)** + +After the PyPi Repository is enabled, it will be available for all new projects +by default. To enable it for existing projects, or if you want to disable it: + +1. Navigate to your project's **Settings > General > Permissions**. +1. Find the Packages feature and enable or disable it. +1. Click on **Save changes** for the changes to take effect. + +You should then be able to see the **Packages** section on the left sidebar. + +## Adding the GitLab PyPi Repository as a source + +You will need the following: + +- A personal access token. You can generate a [personal access token](../../../user/profile/personal_access_tokens.md) with the scope set to `api` for repository authentication. +- A suitable name for your source. +- Your project ID which can be found on the home page of your project. + +Edit your `~/.pypirc` file and add the following: + +```ini +[gitlab] +repository = https://gitlab.com/api/v4/projects/<project_id>/packages/pypi +username = __token__ +password = <your personal access token> +``` + +## Uploading packages + +When uploading packages, note that: + +- The maximum allowed size is 50 Megabytes. +- If you upload the same package with the same version multiple times, each consecutive upload + is saved as a separate file. When installing a package, GitLab will serve the most recent file. +- When uploading packages to GitLab, they will not be displayed in the packages UI of your project + immediately. It can take up to 10 minutes to process a package. + +### Upload packages with Twine + +This section assumes that your project is properly built and you already [created a PyPi package with setuptools](https://packaging.python.org/tutorials/packaging-projects/). +Upload your package using the following command: + +```shell +python -m twine upload --repository <source_name> dist/<package_file> +``` + +Where: + +- `<package_file>` is your package filename, ending in `.tar.gz` or `.whl`. +- `<source_name>` is the [source name used during setup](#adding-the-gitlab-pypi-repository-as-a-source). + +## Install packages + +Install the latest version of a package using the following command: + +```shell +pip install --index-url https://__token__:<personal_access_token>@gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name> +``` + +Where: + +- `<package_name>` is the package name. +- `<personal_access_token>` is your personal access token. +- `<project_id>` is your project id number. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 59867f492b8..9e0486e05c9 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -85,6 +85,10 @@ The following table depicts the various user permission levels in a project. | View project statistics | | ✓ | ✓ | ✓ | ✓ | | View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | | Create new merge request | | ✓ | ✓ | ✓ | ✓ | +| View requirements **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | +| View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | +| Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | | Pull from [Conan repository](packages/conan_repository/index.md), [Maven repository](packages/maven_repository/index.md), or [NPM registry](packages/npm_registry/index.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | | Publish to [Conan repository](packages/conan_repository/index.md), [Maven repository](packages/maven_repository/index.md), or [NPM registry](packages/npm_registry/index.md) **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | @@ -118,6 +122,8 @@ The following table depicts the various user permission levels in a project. | Create and edit wiki pages | | | ✓ | ✓ | ✓ | | Rewrite/remove Git tags | | | ✓ | ✓ | ✓ | | Manage Feature Flags **(PREMIUM)** | | | ✓ | ✓ | ✓ | +| Manage requirements **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | Use environment terminals | | | | ✓ | ✓ | | Run Web IDE's Interactive Web Terminals **(ULTIMATE ONLY)** | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | @@ -222,10 +228,12 @@ group. | Create/edit group epic **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | +| View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | | Create project in group | | | ✓ (3) | ✓ (3) | ✓ (3) | | Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | | Enable/disable a dependency proxy **(PREMIUM)** | | | ✓ | ✓ | ✓ | | Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | +| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | | View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | | Create subgroup | | | | ✓ (1) | ✓ | | Edit epic comments (posted by any user) **(ULTIMATE)** | | | | ✓ (2) | ✓ (2) | diff --git a/doc/user/project/deploy_tokens/img/deploy_tokens.png b/doc/user/project/deploy_tokens/img/deploy_tokens.png Binary files differindex afe1dfb922f..1981b0747bf 100644 --- a/doc/user/project/deploy_tokens/img/deploy_tokens.png +++ b/doc/user/project/deploy_tokens/img/deploy_tokens.png diff --git a/doc/user/project/img/issue_boards_blocked_icon_v12_8.png b/doc/user/project/img/issue_boards_blocked_icon_v12_8.png Binary files differindex 779f643ba56..1055fb48322 100644 --- a/doc/user/project/img/issue_boards_blocked_icon_v12_8.png +++ b/doc/user/project/img/issue_boards_blocked_icon_v12_8.png diff --git a/doc/user/project/img/status_page_incidents_v12_10.png b/doc/user/project/img/status_page_incidents_v12_10.png Binary files differindex ccc1eef3ea3..3540fbffcf8 100644 --- a/doc/user/project/img/status_page_incidents_v12_10.png +++ b/doc/user/project/img/status_page_incidents_v12_10.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png Binary files differindex 8983d685a24..4ab42485d0b 100644 --- a/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png +++ b/doc/user/project/import/img/jira/import_issues_from_jira_button_v12_10.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png Binary files differindex 0ac5e9bdb91..6278cb5f970 100644 --- a/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png +++ b/doc/user/project/import/img/jira/import_issues_from_jira_form_v12_10.png diff --git a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png b/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png Binary files differindex cf5f0dd59cd..bf9728e0311 100644 --- a/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png +++ b/doc/user/project/import/img/jira/import_issues_from_jira_projects_v12_10.png diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 99050f823c5..585c45e35df 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -102,6 +102,8 @@ When you create a project in GitLab, you'll have access to a large number of - [Code owners](code_owners.md): specify code owners for certain files **(STARTER)** - [License Compliance](../compliance/license_compliance/index.md): approve and blacklist licenses for projects. **(ULTIMATE)** - [Dependency List](../application_security/dependency_list/index.md): view project dependencies. **(ULTIMATE)** +- [Requirements](requirements/index.md): Requirements allow you to create criteria to check your products against. **(ULTIMATE)** +- [Static Site Editor](static_site_editor/index.md): quickly edit content on static websites without prior knowledge of the codebase or Git commands. ### Project integrations @@ -170,6 +172,13 @@ Read through the documentation on [CI/CD for external repositories](../../ci/ci_ Learn how to [add members to your projects](members/index.md). +## Project activity + +To view the activity of a project, navigate to **Project overview > Activity**. +From there, you can click on the tabs to see **All** the activity, or see it +filtered by **Push events**, **Merge events**, **Issue events**, **Comments**, +**Team**, and **Wiki**. + ### Leave a project **Leave project** will only display on the project's dashboard diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 88a9bd40f07..bbed14ea93f 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -289,6 +289,7 @@ The following tables outline the details of expected properties. | `title` | string | yes | Heading for the panel. | | `y_label` | string | no, but highly encouraged | Y-Axis label for the panel. | | `y_axis` | string | no | Y-Axis configuration for the panel. | +| `max_value` | number | no | Denominator value used for calculating [percentile based results](#percentile-based-results) | | `weight` | number | no, defaults to order in file | Order to appear within the grouping. Lower number means higher priority, which will be higher on the page. Numbers do not need to be consecutive. | | `metrics` | array | yes | The metrics which should be displayed in the panel. Any number of metrics can be displayed when `type` is `area-chart` or `line-chart`, whereas only 3 can be displayed when `type` is `anomaly-chart`. | @@ -304,7 +305,7 @@ The following tables outline the details of expected properties. | Property | Type | Required | Description | | ------ | ------ | ------ | ------ | -| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics-ultimate) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/60319)). | +| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab-foss/issues/60319)). | | `unit` | string | yes | Defines the unit of the query's return data. | | `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. | | `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | @@ -627,7 +628,21 @@ The options are: - [View logs](#view-logs-ultimate) - [Download CSV](#downloading-data-as-csv) - [Copy link to chart](#embedding-gitlab-managed-kubernetes-metrics) -- [Alerts](#setting-up-alerts-for-prometheus-metrics-ultimate) +- [Alerts](#setting-up-alerts-for-prometheus-metrics) + +### Dashboard Annotations + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/211330) in GitLab 12.10 (enabled by feature flag `metrics_dashboard_annotations`). + +You can use **Metrics Dashboard Annotations** to mark any important events on +every metrics dashboard by adding annotations to it. While viewing a dashboard, +annotation entries assigned to the selected time range will be automatically +fetched and displayed on every chart within that dashboard. On mouse hover, each +annotation presents additional details, including the exact time of an event and +its description. + +You can create annotations by making requests to the +[Metrics dashboard annotations API](../../../api/metrics_dashboard_annotations.md) ### View Logs **(ULTIMATE)** @@ -652,7 +667,7 @@ and end times to the URL, enabling you to share specific timeframes more easily. Data from Prometheus charts on the metrics dashboard can be downloaded as CSV. -### Setting up alerts for Prometheus metrics **(ULTIMATE)** +### Setting up alerts for Prometheus metrics #### Managed Prometheus instances @@ -805,7 +820,7 @@ It is also possible to embed either the default dashboard metrics or individual ### Embedding metrics based on alerts in incident issues -For [GitLab-managed alerting rules](#setting-up-alerts-for-prometheus-metrics-ultimate), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after. +For [GitLab-managed alerting rules](#setting-up-alerts-for-prometheus-metrics), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after. For [manually configured Prometheus instances](#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met: diff --git a/doc/user/project/issues/csv_export.md b/doc/user/project/issues/csv_export.md index 53af878cbd6..ec53b3dbbba 100644 --- a/doc/user/project/issues/csv_export.md +++ b/doc/user/project/issues/csv_export.md @@ -1,6 +1,7 @@ # Export Issues to CSV -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1126) in [GitLab Starter 9.0](https://about.gitlab.com/releases/2017/03/22/gitlab-9-0-released/#export-issues-ees-eep). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1126) in [GitLab Starter 9.0](https://about.gitlab.com/releases/2017/03/22/gitlab-9-0-released/#export-issues-ees-eep). +> - Moved to GitLab Core in GitLab 12.10. Issues can be exported as CSV from GitLab and are sent to your default notification email as an attachment. diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index ff481109e58..1078d0410ed 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -1,4 +1,4 @@ -# Design Management +# Design Management **(PREMIUM)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/660) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.2. diff --git a/doc/user/project/issues/img/epic_tree_health_status_v12_10.png b/doc/user/project/issues/img/epic_tree_health_status_v12_10.png Binary files differnew file mode 100644 index 00000000000..c6d0117b1c4 --- /dev/null +++ b/doc/user/project/issues/img/epic_tree_health_status_v12_10.png diff --git a/doc/user/project/issues/img/issue_activity_sort_order_v12_10.png b/doc/user/project/issues/img/issue_activity_sort_order_v12_10.png Binary files differnew file mode 100644 index 00000000000..3e19ee1ac66 --- /dev/null +++ b/doc/user/project/issues/img/issue_activity_sort_order_v12_10.png diff --git a/doc/user/project/issues/img/issue_health_status_dropdown_v12_10.png b/doc/user/project/issues/img/issue_health_status_dropdown_v12_10.png Binary files differindex c0f0a0dfe16..f8517de4e12 100644 --- a/doc/user/project/issues/img/issue_health_status_dropdown_v12_10.png +++ b/doc/user/project/issues/img/issue_health_status_dropdown_v12_10.png diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index d6576fc780d..06524f785ab 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -8,11 +8,38 @@ The image below illustrates what an issue may look like. Note that certain parts look slightly different or will be absent, depending on the version of GitLab being used and the permissions of the user viewing the issue. - - You can find all the information for that issue on one screen. -### Issue screen + + +- **1.** [New Issue, close issue (reopen issue, report issue)](#new-issue-close-issue-reopen-issue-report-issue) +- **2.** [To Do](#to-do) +- **3.** [Assignee](#assignee) + - **3.1.** [Multiple Assignees **(STARTER)**](#multiple-assignees-starter) +- **4.** [Epic **(ULTIMATE)**](#epic-ultimate) +- **5.** [Milestone](#milestone) +- **6.** [Time tracking](#time-tracking) +- **7.** [Due date](#due-date) +- **8.** [Labels](#labels) +- **9.** [Weight **(STARTER)**](#weight-starter) +- **10.** [Confidentiality](#confidentiality) +- **11.** [Lock issue](#lock-issue) +- **12.** [Participants](#participants) +- **13.** [Notifications](#notifications) +- **14.** [Reference](#reference) +- **15.** [Edit](#edit) +- **16.** [Description](#description) +- **17.** [Mentions](#mentions) +- **18.** [Related Issues **(STARTER)**](#related-issues-starter) +- **19.** [Related Merge Requests](#related-merge-requests) +- **20.** [Award emoji](#award-emoji) +- **21.** [Show all activity](#show-all-activity) +- **22.** [Create Merge Request](#create-merge-request) +- **23.** [Issue history](#issue-history) + - [Activity sort order](#activity-sort-order) +- **24.** [Comments](#comments) +- **25.** [Submit comment, start a thread, or comment and close](#submit-comment-start-a-thread-or-comment-and-close) +- **26.** [Zoom meetings](#zoom-meetings) An issue starts with its status (open or closed), followed by its author, and includes many other functionalities, numbered in the image above to @@ -22,7 +49,7 @@ Many of the elements of the issue screen refresh automatically, such as the titl description, when they are changed by another user. Comments and system notes also update automatically in response to various actions and content updates. -#### 1. New Issue, close issue (reopen issue, report issue) +### New Issue, close issue (reopen issue, report issue) Clicking on **New issue** will open a new window to create a new issue in the same project. Clicking on **Close issue** will close this issue, but it will not be deleted. If the @@ -39,22 +66,23 @@ after it is closed.  -#### 2. To Do +### To Do You can add issues to and remove issues from your [GitLab To-Do List](../../todos.md). -The button to do this has a different label depending on whether the issue is already on your To-Do List or not. If the issue is: +The button to do this has a different label depending on whether the issue is already on your To-Do +List or not. If the issue is: - Already on your To-Do List: The button is labeled **Mark as done**. Click the button to remove the issue from your To-Do List. -- Not on your To-Do List: The button is labelled **Add a To Do**. Click the button to add the issue to your To-Do List. +- Not on your To-Do List: The button is labeled **Add a To Do**. Click the button to add the issue to your To-Do List. -#### 3. Assignee +### Assignee An issue can be assigned to: - Yourself. - Another person. -- [Many people](#31-multiple-assignees-STARTER). **(STARTER)** +- [Many people](#multiple-assignees-STARTER). **(STARTER)** The assignee(s) can be changed as often as needed. The idea is that the assignees are responsible for that issue until it's reassigned to someone else to take it from there. @@ -64,7 +92,7 @@ TIP: **Tip:** If a user is not member of that project, it can only be assigned to them if they created the issue themselves. -##### 3.1. Multiple Assignees **(STARTER)** +#### Multiple Assignees **(STARTER)** Often multiple people work on the same issue together, which can be especially difficult to track in large teams where there is shared ownership of an issue. @@ -72,29 +100,29 @@ to track in large teams where there is shared ownership of an issue. In [GitLab Starter](https://about.gitlab.com/pricing/), you can [assign multiple people](multiple_assignees_for_issues.md) to an issue. -#### 4. Epic **(ULTIMATE)** +### Epic **(ULTIMATE)** You can assign issues to an [Epic](../../group/epics/index.md), which allows better management of groups of related issues. -#### 5. Milestone +### Milestone Select a [milestone](../milestones/index.md) to attribute that issue to. -#### 6. Time Tracking +### Time tracking Use [GitLab Quick Actions](../quick_actions.md) to [track estimates and time spent on issues](../time_tracking.md). You can add an [estimate of the time it will take](../time_tracking.md#estimates) to resolve the issue, and also add [the time spent](../time_tracking.md#time-spent) on the resolution of the issue. -#### 7. Due date +### Due date When you work on a tight schedule, it's important to have a way to set a deadline for implementations and for solving problems. This can be done in the [due date](due_dates.md) element. Due dates can be changed as many times as needed. -#### 8. Labels +### Labels Categorize issues by giving them [labels](../labels.md). They help to organize workflows, and they enable you to work with the [GitLab Issue Board](index.md#issue-boards). @@ -107,29 +135,29 @@ TIP: **Tip:** If a label doesn't exist yet, you can click **Edit**, and it opens a dropdown menu from which you can select **Create new label**. -#### 9. Weight **(STARTER)** +### Weight **(STARTER)** [Assign a weight](issue_weight.md) to an issue. Larger values are used to indicate more effort is required to complete the issue. Only positive values or zero are allowed. -#### 10. Confidentiality +### Confidentiality You can [set an issue to be confidential](confidential_issues.md). When set, unauthorized users will not be able to access the issue, and will not see it listed in project issue boards or the issue list. -#### 11. Lock issue +### Lock issue You can [lock the threads](../../discussions/index.md#lock-discussions) in the issue, to prevent further comments from being added. -#### 12. Participants +### Participants All the users involved in that issue. Either they participated in the [thread](../../discussions/index.md), or were mentioned in the description or threads. -#### 13. Notifications +### Notifications Click on the icon to enable/disable [notifications](../../profile/notifications.md#issue--epics--merge-request-events) for the issue. This will automatically enable if you participate in the issue in any way. @@ -139,27 +167,27 @@ for the issue. This will automatically enable if you participate in the issue in - **Disable**: If you are receiving notifications for updates to that issue but no longer want to receive them, unsubscribe from it. -#### 14. Reference +### Reference - A quick "copy" button for that issue's reference, which looks like `foo/bar#xxx`, where `foo` is the `username` or `groupname`, `bar` is the `project-name`, and `xxx` is the issue number. -#### 15. Edit +### Edit Clicking this icon opens the issue for editing, and you will have access to all the same fields as when the issue was created. This icon will not display if the user does not have permission to edit the issue. -#### 16. Description +### Description The plain text title and description of the issue fill the top center of the issue page. The description fully supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm), allowing many formatting options. -> [Since GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/issues/10103), changes to an issue's description are listed in the [issue history](#23-issue-history).**(STARTER)** +> [Since GitLab 12.6](https://gitlab.com/gitlab-org/gitlab/issues/10103), changes to an issue's description are listed in the [issue history](#issue-history).**(STARTER)** -#### 17. Mentions +### Mentions You can mention a user or a group present in your GitLab instance with `@username` or `@groupname` and they will be notified via todos and email, unless they have disabled @@ -174,19 +202,19 @@ TIP: **Tip:** Avoid mentioning `@all` in issues and merge requests, as it sends an email notification to all the members of that project's group, which can be interpreted as spam. -#### 18. Related Issues **(STARTER)** +### Related Issues **(STARTER)** Issues that were mentioned as [related issues](related_issues.md) are listed here. You can also click the `+` to add more related issues. -#### 19. Related Merge Requests +### Related Merge Requests Merge requests that were mentioned in that issue's description or in the issue thread are listed as [related merge requests](crosslinking_issues.md#from-merge-requests) here. Also, if the current issue was mentioned as related in another merge request, that merge request will be listed here. -#### 20. Award emoji +### Award emoji You can award an emoji to that issue. There are shortcuts to "thumbs_up" and "thumbs_down", or you can click on the light gray "face" to choose a different reaction from the @@ -197,7 +225,7 @@ Posting "+1" as a comment in a thread spams all subscribed participants of that clutters the threads, and is not recommended. Awarding an emoji is a way to let them know your reaction without spamming them. -#### 21. Show all activity +### Show all activity You can filter what is displayed in the issue history by clicking on **Show all activity** and selecting either: @@ -209,7 +237,7 @@ Also: - You can mention a user or a group present in your GitLab instance with `@username` or `@groupname` and they will be notified via To-Do items - and email, unless they have [disabled all notifications](#13-notifications) + and email, unless they have [disabled all notifications](#notifications) in their profile settings. - Mentions for yourself (the current logged in user), will be highlighted in a different color, allowing you to easily see which comments involve you, @@ -217,7 +245,7 @@ Also:  -#### 22. Create Merge Request +### Create Merge Request Create a new branch and [WIP merge request](../merge_requests/work_in_progress_merge_requests.md) in one action. The branch will be named `issuenumber-title` by default, but you can @@ -230,17 +258,30 @@ close the issue when it is merged. Optionally, you can choose to create a [new branch](../repository/web_editor.md#create-a-new-branch-from-an-issue) only, named after that issue. -#### 23. Issue history +### Issue history All comments and updates to the issue are tracked and listed here, but this can be filtered, as shown above. -#### 24. Comments +#### Activity sort order + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14588) in GitLab 12.10. + +You can reverse the default order and interact with the activity feed sorted by most recent items +at the top. Your preference is saved via local storage and automatically applied to every issue +you view. + +To change the activity sort order, click the **Oldest first** dropdown menu and select either oldest +or newest items to be shown first. + + + +### Comments Collaborate in the issue by posting comments in its thread. This text field also fully supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). -#### 25. Submit Comment, start a thread, or comment and close +### Submit comment, start a thread, or comment and close Once you write a comment, you can: @@ -253,7 +294,7 @@ Once you write a comment, you can: You can also close the issue from here, so you don't need to scroll to the top of the issue page. -#### 26. Zoom Meetings +### Zoom meetings > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/31103) in GitLab 12.3. diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index b51263b7528..4ee29f3357d 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -7,7 +7,7 @@ you can do with issues. ## Create a new Issue -When you create a new issue, you'll be prompted to fill in the [data and fields of the issue](issue_data_and_actions.md#parts-of-an-issue), as illustrated below. If you know +When you create a new issue, you'll be prompted to fill in the [data and fields of the issue](issue_data_and_actions.md), as illustrated below. If you know the values you want to assign to an issue, you can use the [Quick actions](../quick_actions.md) feature to input values, instead of selecting them from lists. diff --git a/doc/user/project/issues/multiple_assignees_for_issues.md b/doc/user/project/issues/multiple_assignees_for_issues.md index fefc690e073..dff6a6031d7 100644 --- a/doc/user/project/issues/multiple_assignees_for_issues.md +++ b/doc/user/project/issues/multiple_assignees_for_issues.md @@ -1,8 +1,6 @@ # Multiple Assignees for Issues **(STARTER)** -> **Note:** -[Introduced](https://gitlab.com/gitlab-org/gitlab/issues/1904) -in [GitLab Starter 9.2](https://about.gitlab.com/releases/2017/05/22/gitlab-9-2-released/#multiple-assignees-for-issues). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/1904) in [GitLab Starter 9.2](https://about.gitlab.com/releases/2017/05/22/gitlab-9-2-released/#multiple-assignees-for-issues). ## Overview diff --git a/doc/user/project/merge_requests/img/versions_compare_head_v12_10.png b/doc/user/project/merge_requests/img/versions_compare_head_v12_10.png Binary files differnew file mode 100644 index 00000000000..353cf458243 --- /dev/null +++ b/doc/user/project/merge_requests/img/versions_compare_head_v12_10.png diff --git a/doc/user/project/merge_requests/versions.md b/doc/user/project/merge_requests/versions.md index ffd0efb365a..87c10717671 100644 --- a/doc/user/project/merge_requests/versions.md +++ b/doc/user/project/merge_requests/versions.md @@ -46,6 +46,27 @@ This only applies to commits that are in the most recent version of a merge request - if a commit was in a merge request, then rebased out of that merge request, they will not be linked. +## `HEAD` comparison mode for Merge Requests + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27008) in GitLab 12.10. + +Merge Requests, particularly the **Changes** tab, is where source code +is reviewed and discussed. In circumstances where the target branch was +merged into the source branch of the merge request, the changes in the +source and target branch can be shown mixed together making it hard to +understand which changes are being added and which already exist in the +target branch. + +In GitLab 12.10, we added an **experimental** comparison mode, which +shows a diff calculated by simulating how it would look like once merged - a more accurate +representation of the changes rather than using the base of the two +branches. The new mode is available from the comparison target drop down +by selecting **master (HEAD)**. In the future it will +[replace](https://gitlab.com/gitlab-org/gitlab/issues/198458) the +current default comparison. + + + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 91138adb996..7937b7193d2 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -70,7 +70,7 @@ The following quick actions are applicable to descriptions, discussions and thre | `/wip` | | ✓ | | Toggle the Work In Progress status | | `/approve` | | ✓ | | Approve the merge request **(STARTER)** | | `/submit_review` | | ✓ | | Submit a pending review. ([Introduced in GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/issues/8041)) **(PREMIUM)** | -| `/merge` | | ✓ | | Merge (when pipeline succeeds) | +| `/merge` | | ✓ | | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), adding to a [Merge Train](../../ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md), etc). | | `/child_epic <epic>` | | | ✓ | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. ([Introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab/issues/7330)) **(ULTIMATE)** | | `/remove_child_epic <epic>` | | | ✓ | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. ([Introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab/issues/7330)) **(ULTIMATE)** | | `/parent_epic <epic>` | | | ✓ | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/issues/10556)) **(ULTIMATE)** | diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index bae514153ac..ca28a79abf4 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -64,9 +64,6 @@ A link is any URL which can point to whatever you like; documentation, built binaries, or other related materials. These can be both internal or external links from your GitLab instance. -NOTE: **NOTE** -You can manipulate links of each release entry with [Release Links API](../../../api/releases/links.md) - #### Permanent links to Release assets > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27300) in GitLab 12.9. @@ -201,6 +198,14 @@ milestones, or release date, use the [Releases API](../../../api/releases/index.md#update-a-release). Editing this information through the **Edit Release** page is planned for a future version of GitLab. +Please note that the ability to edit asset links is currently behind a feature +flag which is disabled by default. For self-managed instances, it can be enabled +through the Rails console by a GitLab administrator with the following command: + +```ruby +Feature.enable(:release_asset_link_editing) +``` + ## Notification for Releases > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/26001) in GitLab 12.4. diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index c26f2bd6b1d..126144de703 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -34,6 +34,12 @@ CAUTION: **Caution:** In GitLab 12.6 and later, when project owners [reduce a project's visibility](../../../public_access/public_access.md#reducing-visibility), it **removes the relationship** between a project and all its forks. +CAUTION: **Caution:** +When a public project with the repository feature set to "Members +only" is forked, the repository will be public in the fork. The owner +of the fork will need to manually change the visibility. This is being +fixed in [#36662](https://gitlab.com/gitlab-org/gitlab/-/issues/36662). + ## Repository mirroring You can use [repository mirroring](repository_mirroring.md) to keep your fork synced with the original repository. You can also use `git remote add upstream` to achieve the same result. diff --git a/doc/user/project/repository/img/file_ext_icons_repo_v12_10.png b/doc/user/project/repository/img/file_ext_icons_repo_v12_10.png Binary files differnew file mode 100644 index 00000000000..346f0e58325 --- /dev/null +++ b/doc/user/project/repository/img/file_ext_icons_repo_v12_10.png diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index a9647a9ed0f..f9c953db3e3 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -27,6 +27,12 @@ that you [connect with GitLab via SSH](../../../ssh/README.md). ## Files +Use a repository to store your files in GitLab. From [GitLab 12.10 onwards](https://gitlab.com/gitlab-org/gitlab/issues/33806), +you'll see on the repository's file tree an icon next to the file name +according to its extension: + + + ### Create and edit files Host your codebase in GitLab repositories by pushing your files to GitLab. diff --git a/doc/user/project/requirements/img/requirement_archive_view_v12_10.png b/doc/user/project/requirements/img/requirement_archive_view_v12_10.png Binary files differnew file mode 100644 index 00000000000..b3a52caba6c --- /dev/null +++ b/doc/user/project/requirements/img/requirement_archive_view_v12_10.png diff --git a/doc/user/project/requirements/img/requirement_create_view_v12_10.png b/doc/user/project/requirements/img/requirement_create_view_v12_10.png Binary files differnew file mode 100644 index 00000000000..ecb08fe8a8b --- /dev/null +++ b/doc/user/project/requirements/img/requirement_create_view_v12_10.png diff --git a/doc/user/project/requirements/img/requirement_edit_save_v12_10.png b/doc/user/project/requirements/img/requirement_edit_save_v12_10.png Binary files differnew file mode 100644 index 00000000000..6cf7db361b8 --- /dev/null +++ b/doc/user/project/requirements/img/requirement_edit_save_v12_10.png diff --git a/doc/user/project/requirements/img/requirement_edit_view_v12_10.png b/doc/user/project/requirements/img/requirement_edit_view_v12_10.png Binary files differnew file mode 100644 index 00000000000..5251e7eae1e --- /dev/null +++ b/doc/user/project/requirements/img/requirement_edit_view_v12_10.png diff --git a/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png b/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png Binary files differnew file mode 100644 index 00000000000..a5487b46894 --- /dev/null +++ b/doc/user/project/requirements/img/requirements_archived_list_view_v12_10.png diff --git a/doc/user/project/requirements/img/requirements_list_view_v12_10.png b/doc/user/project/requirements/img/requirements_list_view_v12_10.png Binary files differnew file mode 100644 index 00000000000..cee1f3781f6 --- /dev/null +++ b/doc/user/project/requirements/img/requirements_list_view_v12_10.png diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md new file mode 100644 index 00000000000..8f4ec7bbbed --- /dev/null +++ b/doc/user/project/requirements/index.md @@ -0,0 +1,67 @@ +--- +type: reference, howto +--- + +# Requirements **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2703) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. + +Requirements allow you to create criteria to check your products against. They +can be based on users, stakeholders, system, software, or anything else you +find important to capture. + + + +## Create a requirement + +A paginated list of requirements is available in each project, and there you +can create a new requirement. + +To create a requirement: + +1. From your project page, go to **{requirements}** **Requirements**. +1. Click **New requirement**. +1. Enter a descriptive title and click **Create requirement**. + +You will see the newly created requirement on the top of the list, as the requirements +list is sorted by creation date in descending order. + + + +## Edit a requirement + +You can edit a requirement (if you have the necessary privileges) from the requirements +list page. + +To edit a requirement: + +1. From the requirements list, click the **Edit** (**{pencil}**) button. +1. Update the title in text input field. +1. Click **Save changes**. + + + +The requirements list shows the new title immediately. + + + +## Archive a requirement + +You can archive an open requirement (if you have the necessary privileges) while +you're in the **Open** tab. + +From the requirements list page, click the **Archive** (**{archive}**) button. + + + +As soon as a requirement is archived, it no longer appears in the **Open** tab. + +## Reopen a requirement + +You can view the list of archived requirements in the **Archived** tab. + + + +To reopen an archived requirement, click the **Reopen** button. + +As soon as a requirement is reopened, it no longer appears in the **Archived** tab. diff --git a/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png b/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png Binary files differnew file mode 100644 index 00000000000..380d96f1db9 --- /dev/null +++ b/doc/user/project/static_site_editor/img/edit_this_page_button_v12_10.png diff --git a/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png b/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png Binary files differnew file mode 100644 index 00000000000..130dff9f30d --- /dev/null +++ b/doc/user/project/static_site_editor/img/static_site_editor_v12_10.png diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md new file mode 100644 index 00000000000..f186ff9919e --- /dev/null +++ b/doc/user/project/static_site_editor/index.md @@ -0,0 +1,89 @@ +--- +type: reference, how-to +description: "The static site editor enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture or Git commands." +--- + +# Static Site Editor + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28758) in GitLab 12.10. + +Static Site Editor enables users to edit content on static websites without +prior knowledge of the underlying templating language, site architecture, or +Git commands. A contributor to your project can quickly edit a Markdown page +and submit the changes for review. + +## Use cases + +The Static Site Editors allows collaborators to submit changes to static site +files seamlessly. For example: + +- Non-technical collaborators can easily edit a page directly from the browser; they don't need to know Git and the details of your project to be able to contribute. +- Recently hired team members can quickly edit content. +- Temporary collaborators can jump from project to project and quickly edit pages instead of having to clone or fork every single project they need to submit changes to. + +## Requirements + +- In order use the Static Site Editor feature, your project needs to be +pre-configured with the [Static Site Editor Middleman template](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman). +- The editor needs to be logged into GitLab and needs to be a member of the +project (with Developer or higher permission levels). + +## How it works + +The Static Site Editor is in an early stage of development and only works for +Middleman sites for now. You have to use a specific site template to start +using it. The project template is configured to deploy a [Middleman](https://middlemanapp.com/) +static website with [GitLab Pages](../pages/index.md). + +Once your website is up and running, you'll see a button **Edit this page** on +the bottom-left corner of its pages: + + + +When clicking it, GitLab will open up an editor window from which the content +can be directly edited. When you're ready, you can submit your changes in a +click of a button: + + + +When an editor submits their changes, in the background, GitLab automatically +creates a new branch, commits their changes, and opens a merge request. The +editor will land directly on the merge request, and then they can assign it to +a colleague for review. + +## Getting started + +First, set up the project. Once done, you can use the Static Site Editor to +easily edit your content. + +### Set up your project + +1. To get started, create a new project from the +[Static Site Editor - Middleman](https://gitlab.com/gitlab-org/project-templates/static-site-editor-middleman) +template. You can either [fork it](../repository/forking_workflow.md#creating-a-fork) +or [create a new project from a template](../../../gitlab-basics/create-project.md#built-in-templates). +1. Edit the `data/config.yml` file adding your project's path. +1. Editing the file will trigger a CI/CD pipeline to deploy your project with GitLab Pages. +1. When the pipeline finishes, from your project's left-side menu, go to **Settings > Pages** to find the URL of your new website. +1. Visit your website and look at the bottom-left corner of the screen to see the new **Edit this page** button. + +Anyone satisfying the [requirements](#requirements) will be able to edit the +content of the pages without prior knowledge of Git nor of your site's +codebase. + +### Use the Static Site Editor to edit your content + +For instance, suppose you are a recently hired technical writer at a large +company and a new feature has been added to the company product. + +1. You are assigned the task of updating the documentation. +1. You visit a page and see content that needs to be edited. +1. Click the **Edit this page** button on the production site. +1. The file is opened in the Static Site Editor. +1. You edit the file right there and click **Submit changes**. +1. A new merge request is automatically created and you assign it to your colleague for review. + +## Limitations + +- Currently, the Static Site Editor only works for files ending in `.md`. For example, it will not work for a file `index.html.md.erb` while it works for `index.html.md`. +- The Static Site Editor still cannot be quickly added to existing Middleman sites. Follow this [epic](https://gitlab.com/groups/gitlab-org/-/epics/2784) for updates. diff --git a/doc/user/project/status_page/index.md b/doc/user/project/status_page/index.md index 2a022fe472d..d292ca94ba9 100644 --- a/doc/user/project/status_page/index.md +++ b/doc/user/project/status_page/index.md @@ -78,10 +78,16 @@ To publish an Incident, you first need to create an issue in the Project you ena Once this issue is created, a background worker will publish the issue onto the status page using the credentials you provided during setup. +NOTE: **Note:** +Confidential issues are not published. If a published issue is made confidential it will be unpublished. + ### Publishing updates To publish an update to the Incident, update the incident issue's description. +CAUTION: **Caution:** +When referenced issues are changed (e.g. title, confidentiality) the incident they were referenced in are not updated automatically. + ### Adding comments To add comments to the Status Page Incident, create a comment on the incident issue. diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 3255e3a5477..88b729272ec 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -140,6 +140,43 @@ number.  +## Wiki activity records + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14902) in GitLab 12.10. +> - It's deployed behind a feature flag, disabled by default. +> - It's enabled on GitLab.com. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-wiki-events-core-only). **(CORE ONLY)** + +Wiki events (creation, deletion, and updates) are tracked by GitLab and +displayed on the [user profile](../../profile/index.md#user-profile), +[group](../../group/index.md#view-group-activity), +and [project](../index.md#project-activity) activity pages. + +### Limitations + +Only edits made in the browser or through the API have their activity recorded. +Edits made and pushed through Git are not currently listed in the activity list. + +### Enable or disable Wiki Events **(CORE ONLY)** + +Wiki event activity is under development and not ready for production use. It is +deployed behind a feature flag that is **disabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/troubleshooting/navigating_gitlab_via_rails_console.md#starting-a-rails-console-session) +can enable it for your instance. You're welcome to test it, but use it at your +own risk. + +To enable it: + +```ruby +Feature.enable(:wiki_events) +``` + +To disable it: + +```ruby +Feature.disable(:wiki_events) +``` + ## Adding and editing wiki pages locally Since wikis are based on Git repositories, you can clone them locally and edit diff --git a/doc/user/search/img/issue_search_by_id.png b/doc/user/search/img/issue_search_by_id.png Binary files differnew file mode 100644 index 00000000000..96c0b5c31e1 --- /dev/null +++ b/doc/user/search/img/issue_search_by_id.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 2166e8ddbd5..ed2a4b36372 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -81,6 +81,14 @@ You can filter issues and merge requests by specific terms included in titles or  +### Filtering by ID + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/39908) in GitLab 12.1. + +You can filter the **Issues** list to individual instances by their ID. For example, enter filter `#10` to return only issue 10. The same applies to the **Merge Requests** list. Enter filter `#30` to return only merge request 30. + + + ### Filtering merge requests by approvers **(STARTER)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/9468) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.9. @@ -98,7 +106,9 @@ You can view recent searches by clicking on the little arrow-clock icon, which i ## Removing search filters -Individual filters can be removed by clicking on the filter's (x) button or backspacing. The entire search filter can be cleared by clicking on the search box's (x) button. +Individual filters can be removed by clicking on the filter's (x) button or backspacing. The entire search filter can be cleared by clicking on the search box's (x) button or via <kbd>⌘</kbd> (Mac) + <kbd>⌫</kbd>. + +To delete filter tokens one at a time, the <kbd>⌥</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>⌫</kbd> keyboard combination can be used. ## Filtering with multiple filters of the same type diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index dcc4753a794..fa466fdb3b9 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -127,6 +127,15 @@ This shortcut is available when viewing a [wiki page](project/wiki/index.md): | ----------------- | ----------- | | <kbd>e</kbd> | Edit wiki page. | +### Filtered Search + +These shortcuts are available when using a [filtered search input](search/index.md): + +| Keyboard Shortcut | Description | +| ----------------------------------------------------- | ----------- | +| <kbd>⌘</kbd> (Mac) + <kbd>⌫</kbd> | Clear entire search filter. | +| <kbd>⌥</kbd> (Mac) / <kbd>Ctrl</kbd> + <kbd>⌫</kbd> | Clear one token at a time. | + ## Epics **(ULTIMATE)** These shortcuts are available when viewing [Epics](group/epics/index.md): diff --git a/doc/user/todos.md b/doc/user/todos.md index 69c26660c87..d19f1b8f14b 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -36,7 +36,9 @@ A To Do appears on your To-Do List when: - Issue - Merge Request - Epic **(ULTIMATE)** -- You are `@mentioned` in a comment on a commit +- You are `@mentioned` in a comment on a: + - Commit + - Design **(PREMIUM)** - A job in the CI pipeline running for your merge request failed, but this job is not allowed to fail - An open merge request becomes unmergeable due to conflict, and you are either: |
