diff options
Diffstat (limited to 'doc')
33 files changed, 571 insertions, 413 deletions
diff --git a/doc/README.md b/doc/README.md index 1c0911cdbc0..44d8b3b1932 100644 --- a/doc/README.md +++ b/doc/README.md @@ -359,14 +359,14 @@ The following documentation relates to the DevOps **Secure** stage: | Secure Topics | Description | |:------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------| -| [Compliance Dashboard](user/application_security/compliance_dashboard/index.md) **(ULTIMATE)** | View the most recent Merge Request activity in a group. | +| [Compliance Dashboard](user/compliance/compliance_dashboard/index.md) **(ULTIMATE)** | View the most recent Merge Request activity in a group. | | [Container Scanning](user/application_security/container_scanning/index.md) **(ULTIMATE)** | Use Clair to scan docker images for known vulnerabilities. | | [Dependency List](user/application_security/dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](user/application_security/dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | | [Group Security Dashboard](user/application_security/security_dashboard/index.md#group-security-dashboard) **(ULTIMATE)** | View vulnerabilities in all the projects in a group and its subgroups. | | [Instance Security Dashboard](user/application_security/security_dashboard/index.md#instance-security-dashboard) **(ULTIMATE)** | View vulnerabilities in all the projects you're interested in. | -| [License Compliance](user/application_security/license_compliance/index.md) **(ULTIMATE)** | Search your project's dependencies for their licenses. | +| [License Compliance](user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project's dependencies for their licenses. | | [Pipeline Security Dashboard](user/application_security/security_dashboard/index.md#pipeline-security-dashboard) **(ULTIMATE)** | View the security reports for your project's pipelines. | | [Project Security Dashboard](user/application_security/security_dashboard/index.md#project-security-dashboard) **(ULTIMATE)** | View the latest security reports for your project. | | [Static Application Security Testing (SAST)](user/application_security/sast/index.md) **(ULTIMATE)** | Analyze source code for known vulnerabilities. | diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index c8783b8ff5f..130ed9f3b25 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -20,12 +20,12 @@ watch [this 1 hour Q&A](https://www.youtube.com/watch?v=uCU8jdYzpac) with [John Northrup](https://gitlab.com/northrup), and live questions coming in from some of our customers. -## Recommended Setups based on number of users +## Recommended setups based on number of users - 1 - 1000 Users: A single-node [Omnibus](https://docs.gitlab.com/omnibus/) setup with frequent backups. Refer to the [requirements page](../../install/requirements.md) for further details of the specs you will require. - 2000 - 50000+ Users: A scaled HA environment based on one of our [Reference Architectures](#reference-architectures) below. -## GitLab Components and Configuration Instructions +## GitLab components and configuration instructions The GitLab application depends on the following [components](../../development/architecture.md#component-diagram) and services. They are included in the reference architectures along with our @@ -48,7 +48,7 @@ in which you would typically configure them. In some cases, components can be combined on the same nodes to reduce complexity as well. -## Reference Architectures +## Reference architectures In this section we'll detail the Reference Architectures that can support large numbers of users. These were built, tested and verified by our Quality and Support teams. @@ -69,12 +69,11 @@ how much automation you use, mirroring, and repo/change size. Additionally the shown memory values are given directly by [GCP machine types](https://cloud.google.com/compute/docs/machine-types). On different cloud vendors a best effort like for like can be used. -### 2,000 User Configuration +### 2,000 user configuration -- **Supported Users (approximate):** 2,000 -- **Test RPS Rates:** API: 40 RPS, Web: 4 RPS, Git: 4 RPS -- **Known Issues:** For the latest list of known performance issues head -[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). +- **Supported users (approximate):** 2,000 +- **Test RPS rates:** API: 40 RPS, Web: 4 RPS, Git: 4 RPS +- **Known issues:** [List of known performance issues](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues) | Service | Nodes | Configuration[^8] | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -91,12 +90,11 @@ On different cloud vendors a best effort like for like can be used. | External load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | | Internal load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | -### 5,000 User Configuration +### 5,000 user configuration -- **Supported Users (approximate):** 5,000 -- **Test RPS Rates:** API: 100 RPS, Web: 10 RPS, Git: 10 RPS -- **Known Issues:** For the latest list of known performance issues head -[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). +- **Supported users (approximate):** 5,000 +- **Test RPS rates:** API: 100 RPS, Web: 10 RPS, Git: 10 RPS +- **Known issues:** [List of known performance issues](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues) | Service | Nodes | Configuration[^8] | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -113,12 +111,11 @@ On different cloud vendors a best effort like for like can be used. | External load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | | Internal load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | -### 10,000 User Configuration +### 10,000 user configuration -- **Supported Users (approximate):** 10,000 -- **Test RPS Rates:** API: 200 RPS, Web: 20 RPS, Git: 20 RPS -- **Known Issues:** For the latest list of known performance issues head -[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). +- **Supported users (approximate):** 10,000 +- **Test RPS rates:** API: 200 RPS, Web: 20 RPS, Git: 20 RPS +- **Known issues:** [List of known performance issues](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues) | Service | Nodes | Configuration[^8] | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -138,12 +135,11 @@ On different cloud vendors a best effort like for like can be used. | External load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | | Internal load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | -### 25,000 User Configuration +### 25,000 user configuration -- **Supported Users (approximate):** 25,000 -- **Test RPS Rates:** API: 500 RPS, Web: 50 RPS, Git: 50 RPS -- **Known Issues:** For the latest list of known performance issues head -[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). +- **Supported users (approximate):** 25,000 +- **Test RPS rates:** API: 500 RPS, Web: 50 RPS, Git: 50 RPS +- **Known issues:** [List of known performance issues](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues) | Service | Nodes | Configuration[^8] | GCP type | | ----------------------------|-------|-----------------------|---------------| @@ -163,12 +159,11 @@ On different cloud vendors a best effort like for like can be used. | External load balancing node[^6] | 1 | 2 vCPU, 1.8GB Memory | n1-highcpu-2 | | Internal load balancing node[^6] | 1 | 4 vCPU, 3.6GB Memory | n1-highcpu-4 | -### 50,000 User Configuration +### 50,000 user configuration -- **Supported Users (approximate):** 50,000 -- **Test RPS Rates:** API: 1000 RPS, Web: 100 RPS, Git: 100 RPS -- **Known Issues:** For the latest list of known performance issues head -[here](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues). +- **Supported users (approximate):** 50,000 +- **Test RPS rates:** API: 1000 RPS, Web: 100 RPS, Git: 100 RPS +- **Known issues:** [List of known performance issues](https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=Quality%3Aperformance-issues) | Service | Nodes | Configuration[^8] | GCP type | | ----------------------------|-------|-----------------------|---------------| diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index 6a8394752c5..7685a564076 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -61,6 +61,7 @@ The following API resources are available in the project context: | [Protected tags](protected_tags.md) | `/projects/:id/protected_tags` | | [Releases](releases/index.md) | `/projects/:id/releases` | | [Release links](releases/links.md) | `/projects/:id/releases/.../assets/links` | +| [Remote mirrors](remote_mirrors.md) | `/projects/:id/remote_mirrors` | | [Repositories](repositories.md) | `/projects/:id/repository` | | [Repository files](repository_files.md) | `/projects/:id/repository/files` | | [Repository submodules](repository_submodules.md) | `/projects/:id/repository/submodules` | diff --git a/doc/api/remote_mirrors.md b/doc/api/remote_mirrors.md new file mode 100644 index 00000000000..0ffff194976 --- /dev/null +++ b/doc/api/remote_mirrors.md @@ -0,0 +1,121 @@ +# Project remote mirrors API + +[Push mirrors](../user/project/repository/repository_mirroring.md#pushing-to-a-remote-repository-core) +defined on a project's repository settings are called "remote mirrors", and the +state of these mirrors can be queried and modified via the remote mirror API +outlined below. + +## List a project's remote mirrors + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/38121) in GitLab 12.9. + +Returns an Array of remote mirrors and their statuses: + +```text +GET /projects/:id/remote_mirrors +``` + +Example request: + +```sh +curl --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/projects/42/remote_mirrors' +``` + +Example response: + +```json +[ + { + "enabled": true, + "id": 101486, + "last_error": null, + "last_successful_update_at": "2020-01-06T17:32:02.823Z", + "last_update_at": "2020-01-06T17:32:02.823Z", + "last_update_started_at": "2020-01-06T17:31:55.864Z", + "only_protected_branches": true, + "update_status": "finished", + "url": "https://*****:*****@gitlab.com/gitlab-org/security/gitlab.git" + } +] +``` + +NOTE: **Note:** +For security reasons, the `url` attribute will always be scrubbed of username +and password information. + +## Create a remote mirror + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24189) in GitLab 12.9. + +Create a remote mirror for a project. The mirror will be disabled by default. You can enable it by including the optional parameter `enabled` when creating it: + +```text +POST /projects/:id/remote_mirrors +``` + +| Attribute | Type | Required | Description | +| :---------- | :----- | :--------- | :------------ | +| `url` | String | yes | The URL of the remote repository to be mirrored. | +| `enabled` | Boolean | no | Determines if the mirror is enabled. | +| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. | + +Example request: + +```sh +curl --request POST --data "url=https://username:token@example.com/gitlab/example.git" --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/projects/42/remote_mirrors' +``` + +Example response: + +```json +{ + "enabled": false, + "id": 101486, + "last_error": null, + "last_successful_update_at": null, + "last_update_at": null, + "last_update_started_at": null, + "only_protected_branches": false, + "update_status": "none", + "url": "https://*****:*****@example.com/gitlab/example.git" +} +``` + +## Update a remote mirror's attributes + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/38121) in GitLab 12.9. + +Toggle a remote mirror on or off, or change which types of branches are +mirrored: + +```text +PUT /projects/:id/remote_mirrors/:mirror_id +``` + +| Attribute | Type | Required | Description | +| :---------- | :----- | :--------- | :------------ | +| `mirror_id` | Integer | yes | The remote mirror ID. | +| `enabled` | Boolean | no | Determines if the mirror is enabled. | +| `only_protected_branches` | Boolean | no | Determines if only protected branches are mirrored. | + +Example request: + +```sh +curl --request PUT --data "enabled=false" --header "PRIVATE-TOKEN: <your_access_token>" 'https://gitlab.example.com/api/v4/projects/42/remote_mirrors/101486' +``` + +Example response: + +```json +{ + "enabled": false, + "id": 101486, + "last_error": null, + "last_successful_update_at": "2020-01-06T17:32:02.823Z", + "last_update_at": "2020-01-06T17:32:02.823Z", + "last_update_started_at": "2020-01-06T17:31:55.864Z", + "only_protected_branches": true, + "update_status": "finished", + "url": "https://*****:*****@gitlab.com/gitlab-org/security/gitlab.git" +} +``` diff --git a/doc/ci/README.md b/doc/ci/README.md index 3cf2efeae45..32747651d4f 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -133,7 +133,7 @@ Its feature set is listed on the table below according to DevOps stages. | **Secure** || | [Container Scanning](../user/application_security/container_scanning/index.md) **(ULTIMATE)** | Check your Docker containers for known vulnerabilities.| | [Dependency Scanning](../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | -| [License Compliance](../user/application_security/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | +| [License Compliance](../user/compliance/license_compliance/index.md) **(ULTIMATE)** | Search your project dependencies for their licenses. | | [Security Test reports](../user/application_security/index.md) **(ULTIMATE)** | Check for app vulnerabilities. | ## Examples diff --git a/doc/ci/examples/license_management.md b/doc/ci/examples/license_management.md index 0d12c9a20f2..df9af4db929 100644 --- a/doc/ci/examples/license_management.md +++ b/doc/ci/examples/license_management.md @@ -1,5 +1,5 @@ --- -redirect_to: '../../user/application_security/license_compliance/index.md' +redirect_to: '../../user/compliance/license_compliance/index.md' --- -This document was moved to [another location](../../user/application_security/license_compliance/index.md). +This document was moved to [another location](../../user/compliance/license_compliance/index.md). diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index e5949473d01..9348ba2f21c 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -140,15 +140,15 @@ workaround you'd be able to take immediately. If it's not available or acceptabl please read through this section. Merge train is enabled by default when you enable [Pipelines for merged results](../index.md), -however, you can forcibly disable this feature by disabling the feature flag `:merge_trains_enabled`. -After you disabled this feature, all the existing merge trains will be aborted and -you will no longer see the **Start/Add Merge Train** button in merge requests. +however, you can disable this feature by setting the `:disable_merge_trains` feature flag to `enable`. +When you disable this feature, all existing merge trains are aborted and +the **Start/Add Merge Train** button no longer appears in merge requests. To check if the feature flag is enabled on your GitLab instance, -please ask administrator to execute the following commands: +please ask an administrator to execute the following commands **(CORE ONLY)**: ```shell > sudo gitlab-rails console # Login to Rails console of GitLab instance. -> Feature.enabled?(:merge_trains_enabled) # Check if it's enabled or not. -> Feature.disable(:merge_trains_enabled) # Disable the feature flag. +> Feature.enabled?(:disable_merge_trains) # Check if it's disabled or not. +> Feature.enable(:disable_merge_trains) # Disable Merge Trains. ``` diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 9976a423996..e86668cbe11 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -2268,7 +2268,7 @@ introduced in GitLab 12.8. > Introduced in GitLab 11.5. Requires GitLab Runner 11.5 and above. -The `license_management` report collects [Licenses](../../user/application_security/license_compliance/index.md) +The `license_management` report collects [Licenses](../../user/compliance/license_compliance/index.md) as artifacts. The collected License Compliance report will be uploaded to GitLab as an artifact and will @@ -2279,7 +2279,7 @@ dashboards. It is not available for download through the web interface. > Introduced in GitLab 12.8. Requires GitLab Runner 11.5 and above. -The `license_scanning` report collects [Licenses](../../user/application_security/license_compliance/index.md) +The `license_scanning` report collects [Licenses](../../user/compliance/license_compliance/index.md) as artifacts. The License Compliance report will be uploaded to GitLab as an artifact and will diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index ae215026f56..95167c4adf1 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -109,7 +109,7 @@ become available, you will be able to share job templates like this Dependencies should be kept to the minimum. The introduction of a new dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). Both [License -Management](../../user/application_security/license_compliance/index.md) +Management](../../user/compliance/license_compliance/index.md) **(ULTIMATE)** and [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) **(ULTIMATE)** should be activated on all projects to ensure new dependencies diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 7eb5bb21be8..97ebe9bd5a7 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -52,6 +52,13 @@ bundle exec guard When using spring and guard together, use `SPRING=1 bundle exec guard` instead to make use of spring. +Use [Factory Doctor](https://test-prof.evilmartians.io/#/factory_doctor.md) to find cases on un-necessary database manipulation, which can cause slow tests. + +```shell +# run test for path +FDOC=1 bin/rspec spec/[path]/[to]/[spec].rb +``` + ### General guidelines - Use a single, top-level `describe ClassName` block. diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 4958768f772..b078eb25b6a 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -474,7 +474,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any licenses are also shown in the merge request widget. Read more how -[License Compliance works](../../user/application_security/license_compliance/index.md). +[License Compliance works](../../user/compliance/license_compliance/index.md). ### Auto Container Scanning **(ULTIMATE)** diff --git a/doc/user/application_security/compliance_dashboard/index.md b/doc/user/application_security/compliance_dashboard/index.md index afe3ce185e6..d9af9d66c36 100644 --- a/doc/user/application_security/compliance_dashboard/index.md +++ b/doc/user/application_security/compliance_dashboard/index.md @@ -1,31 +1,5 @@ --- -type: reference, howto +redirect_to: '../../compliance/compliance_dashboard/index.md' --- -# Compliance Dashboard **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/36524) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.8. - -The Compliance Dashboard gives you the ability to see a group's Merge Request activity -by providing a high-level view for all projects in the group. For example, code approved -for merging into production. - -## Overview - -To access the Compliance Dashboard for a group, navigate to **{shield}** **Security & Compliance > Compliance** on the group's menu. - - - -## Use cases - -This feature is for people who care about the compliance status of projects within their group. - -You can use the dashboard to: - -- Get an overview of the latest Merge Request for each project. -- See if Merge Requests were approved and by whom. - -## Permissions - -- On [GitLab Ultimate](https://about.gitlab.com/pricing/) tier. -- By **Administrators** and **Group Owners**. +This document was moved to [another location](../../compliance/compliance_dashboard/index.md). diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index 992f4137bb8..b9c3b6521d6 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -48,8 +48,8 @@ vulnerability will then be displayed below it. > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/10536) in GitLab Ultimate 12.3. -If the [License Compliance](../license_compliance/index.md) CI job is configured, -the [discovered licenses](../license_compliance/index.md#supported-languages-and-package-managers) will be displayed on this page. +If the [License Compliance](../../compliance/license_compliance/index.md) CI job is configured, +the [discovered licenses](../../compliance/license_compliance/index.md#supported-languages-and-package-managers) will be displayed on this page. ## Downloading the Dependency List diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 805c2d0c140..8d7891fb973 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -19,12 +19,10 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | Secure scanning tool | Description | |:-----------------------------------------------------------------------------|:-----------------------------------------------------------------------| -| [Compliance Dashboard](compliance_dashboard/index.md) **(ULTIMATE)** | View the most recent Merge Request activity in a group. | | [Container Scanning](container_scanning/index.md) **(ULTIMATE)** | Scan Docker containers for known vulnerabilities. | | [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | -| [License Compliance](license_compliance/index.md) **(ULTIMATE)** | Search your project's dependencies for their licenses. | | [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) **(ULTIMATE)** | Analyze source code for known vulnerabilities. | @@ -185,7 +183,7 @@ with the number of approvals required greater than zero. Once this group is added to your project, the approval rule is enabled for all Merge Requests. To configure how this rule behaves, you can choose which licenses to `approve` or `blacklist` in the -[project policies for License Compliance](license_compliance/index.md#project-policies-for-license-compliance) +[project policies for License Compliance](../compliance/license_compliance/index.md#project-policies-for-license-compliance) section. Any code changes cause the approvals required to reset. diff --git a/doc/user/application_security/license_compliance/index.md b/doc/user/application_security/license_compliance/index.md index 03a7ffd195a..ed81eb8ca10 100644 --- a/doc/user/application_security/license_compliance/index.md +++ b/doc/user/application_security/license_compliance/index.md @@ -1,326 +1,5 @@ --- -type: reference, howto +redirect_to: '../../compliance/license_compliance/index.md' --- -# License Compliance **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5483) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. - -## Overview - -If you are using [GitLab CI/CD](../../../ci/README.md), you can search your project dependencies for their licenses -using License Compliance. - -You can take advantage of License Compliance by either [including the job](#configuration) -in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto License Compliance](../../../topics/autodevops/index.md#auto-license-compliance-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). - -GitLab checks the License Compliance report, compares the licenses between the -source and target branches, and shows the information right on the merge request. -Blacklisted licenses will be clearly visible with an `x` red icon next to them -as well as new licenses which need a decision from you. In addition, you can -[manually approve or blacklist](#project-policies-for-license-compliance) -licenses in your project's settings. - -NOTE: **Note:** -If the license compliance report doesn't have anything to compare to, no information -will be displayed in the merge request area. That is the case when you add the -`license_scanning` job in your `.gitlab-ci.yml` for the first time. -Consecutive merge requests will have something to compare to and the license -compliance report will be shown properly. - - - -If you are a project or group Maintainer, you can click on a license to be given -the choice to approve it or blacklist it. - - - -## Use cases - -It helps you find what licenses your project uses in its dependencies, and decide for each of then -whether to allow it or forbid it. For example, your application is using an external (open source) -library whose license is incompatible with yours. - -## Supported languages and package managers - -The following languages and package managers are supported. - -| Language | Package managers | Scan Tool | -|------------|-------------------------------------------------------------------|----------------------------------------------------------| -| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Go | [Godep](https://github.com/tools/godep), go get ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), gvt ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), glide ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), dep ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), trash ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) and govendor ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), [go mod](https://github.com/golang/go/wiki/Modules) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Java | [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| .NET | [Nuget](https://www.nuget.org/) (.NET Framework is supported via the [mono project](https://www.mono-project.com/). Windows specific dependencies are not supported at this time.) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Python | [pip](https://pip.pypa.io/en/stable/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Ruby | [gem](https://rubygems.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Erlang | [rebar](https://www.rebar3.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage) , [CocoaPods v0.39 and below](https://cocoapods.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| Elixir | [mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| -| C++/C | [conan](https://conan.io/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Scala | [sbt](https://www.scala-sbt.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| Rust | [cargo](https://crates.io) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| -| PHP | [composer](https://getcomposer.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| - -## Requirements - -To run a License Compliance scanning job, you need GitLab Runner with the -[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). - -## Configuration - -For GitLab 12.8 and later, to enable License Compliance, you must -[include](../../../ci/yaml/README.md#includetemplate) the -[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) -that's provided as a part of your GitLab installation. -For older versions of GitLab from 11.9 to 12.7, you must -[include](../../../ci/yaml/README.md#includetemplate) the -[`License-Management.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/License-Management.gitlab-ci.yml). -For GitLab versions earlier than 11.9, you can copy and use the job as defined -that template. - -NOTE: **Note:** -In GitLab 13.0, the `License-Management.gitlab-ci.yml` template is scheduled to be removed. -Use `License-Scanning.gitlab-ci.yml` instead. - -Add the following to your `.gitlab-ci.yml` file: - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml -``` - -The included template will create a `license_scanning` job in your CI/CD pipeline -and scan your dependencies to find their licenses. - -NOTE: **Note:** -Before GitLab 12.8, the `license_scanning` job was named `license_management`. -In GitLab 13.0, the `license_management` job is scheduled to be removed completely, -so you're advised to migrate to the `license_scanning` job and used the new -`License-Scanning.gitlab-ci.yml` template. - -The results will be saved as a -[License Compliance report artifact](../../../ci/yaml/README.md#artifactsreportslicense_scanning-ultimate) -that you can later download and analyze. Due to implementation limitations, we -always take the latest License Compliance artifact available. Behind the scenes, the -[GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) -is used to detect the languages/frameworks and in turn analyzes the licenses. - -The License Compliance settings can be changed through [environment variables](#available-variables) by using the -[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. - -### Available variables - -License Compliance can be configured using environment variables. - -| Environment variable | Required | Description | -|-----------------------|----------|-------------| -| `MAVEN_CLI_OPTS` | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. | -| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if your project has both Golang and Ruby code stored in different directories and you want to only scan the Ruby code, you can update your `.gitlab-ci-yml` template to specify which project directories to scan, like `LICENSE_FINDER_CLI_OPTS: '--debug --aggregate-paths=. ruby'`. | -| `LM_JAVA_VERSION` | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. | -| `LM_PYTHON_VERSION` | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. | -| `SETUP_CMD` | no | Custom setup for the dependency installation. (experimental) | - -### Installing custom dependencies - -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. - -The `license_management` image already embeds many auto-detection scripts, languages, -and packages. Nevertheless, it's almost impossible to cover all cases for all projects. -That's why sometimes it's necessary to install extra packages, or to have extra steps -in the project automated setup, like the download and installation of a certificate. -For that, a `LICENSE_MANAGEMENT_SETUP_CMD` environment variable can be passed to the container, -with the required commands to run before the license detection. - -If present, this variable will override the setup step necessary to install all the packages -of your application (e.g.: for a project with a `Gemfile`, the setup step could be -`bundle install`). - -For example: - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml - -variables: - LICENSE_MANAGEMENT_SETUP_CMD: sh my-custom-install-script.sh -``` - -In this example, `my-custom-install-script.sh` is a shell script at the root -directory of your project. - -### Overriding the template - -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `license_scanning` job -after the template inclusion and specify any additional keys under it. For example: - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml - -license_scanning: - variables: - CI_DEBUG_TRACE: "true" -``` - -### Configuring Maven projects - -The License Compliance tool provides a `MAVEN_CLI_OPTS` environment variable which can hold -the command line arguments to pass to the `mvn install` command which is executed under the hood. -Feel free to use it for the customization of Maven execution. For example: - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml - -license_scanning: - variables: - MAVEN_CLI_OPTS: --debug -``` - -`mvn install` runs through all of the [build life cycle](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) -stages prior to `install`, including `test`. Running unit tests is not directly -necessary for the license scanning purposes and consumes time, so it's skipped -by having the default value of `MAVEN_CLI_OPTS` as `-DskipTests`. If you want -to supply custom `MAVEN_CLI_OPTS` and skip tests at the same time, don't forget -to explicitly add `-DskipTests` to your options. -If you still need to run tests during `mvn install`, add `-DskipTests=false` to -`MAVEN_CLI_OPTS`. - -### Selecting the version of Python - -> - [Introduced](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/36) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. -> - In [GitLab 12.2](https://gitlab.com/gitlab-org/gitlab/issues/12032), Python 3.5 became the default. -> - In [GitLab 12.7](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/101), Python 3.8 became the default. - -License Compliance uses Python 3.8 and pip 19.1 by default. -If your project requires Python 2, you can switch to Python 2.7 and pip 10.0 -by setting the `LM_PYTHON_VERSION` environment variable to `2`. - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml - -license_scanning: - variables: - LM_PYTHON_VERSION: 2 -``` - -### Migration from `license_management` to `license_scanning` - -In GitLab 12.8 a new name for `license_management` job was introduced. This change was made to improve clarity around the purpose of the scan, which is to scan and collect the types of licenses present in a projects dependencies. -The support of `license_management` is scheduled to be dropped in GitLab 13.0. -If you're using a custom setup for License Compliance, you're required -to update your CI config accordingly: - -1. Change the CI template to `License-Scanning.gitlab-ci.yml`. -1. Change the job name to `license_scanning` (if you mention it in `.gitlab-ci.yml`). -1. Change the artifact name to `gl-license-scanning-report.json` (if you mention it in `.gitlab-ci.yml`). - -For example, the following `.gitlab-ci.yml`: - -```yaml -include: - - template: License-Management.gitlab-ci.yml - -license_management: - artifacts: - reports: - license_management: gl-license-management-report.json -``` - -Should be changed to: - -```yaml -include: - - template: License-Scanning.gitlab-ci.yml - -license_scanning: - artifacts: - reports: - license_scanning: gl-license-scanning-report.json -``` - -## Project policies for License Compliance - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5940) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. - -From the project's settings: - -- The list of licenses and their status can be managed. -- Licenses can be manually approved or blacklisted. - -To approve or blacklist a license: - -1. Either use the **Manage licenses** button in the merge request widget, or - navigate to the project's **Settings > CI/CD** and expand the - **License Compliance** section. -1. Click the **Add a license** button. - -  - -1. In the **License name** dropdown, either: - - Select one of the available licenses. You can search for licenses in the field - at the top of the list. - - Enter arbitrary text in the field at the top of the list. This will cause the text to be - added as a license name to the list. -1. Select the **Approve** or **Blacklist** radio button to approve or blacklist respectively - the selected license. - -To modify an existing license: - -1. In the **License Compliance** list, click the **Approved/Declined** dropdown to change it to the desired status. - -  - -Searching for Licenses: - -1. Use the **Search** box to search for a specific license. - -  - -## License Compliance report under pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5491) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2. - -From your project's left sidebar, navigate to **CI/CD > Pipelines** and click on the -pipeline ID that has a `license_management` job to see the Licenses tab with the listed -licenses (if any). - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> - -## License list - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13582) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.7. - -The License list allows you to see your project's licenses and key -details about them. - -In order for the licenses to appear under the license list, the following -requirements must be met: - -1. The License Compliance CI job must be [configured](#configuration) for your project. -1. Your project must use at least one of the - [supported languages and package managers](#supported-languages-and-package-managers). - -Once everything is set, navigate to **Security & Compliance > License Compliance** -in your project's sidebar, and you'll see the licenses displayed, where: - -- **Name:** The name of the license. -- **Component:** The components which have this license. - - +This document was moved to [another location](../../compliance/license_compliance/index.md). diff --git a/doc/user/application_security/license_management/index.md b/doc/user/application_security/license_management/index.md index 319da2c3a6e..df44041600b 100644 --- a/doc/user/application_security/license_management/index.md +++ b/doc/user/application_security/license_management/index.md @@ -1,5 +1,5 @@ --- -redirect_to: ../license_compliance/index.md +redirect_to: ../../compliance/license_compliance/index.md --- -This document was moved to [another location](../license_compliance/index.md). +This document was moved to [another location](../../compliance/license_compliance/index.md). diff --git a/doc/user/application_security/compliance_dashboard/img/compliance_dashboard_v12_8.png b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_8.png Binary files differindex 5fc54927de7..5fc54927de7 100644 --- a/doc/user/application_security/compliance_dashboard/img/compliance_dashboard_v12_8.png +++ b/doc/user/compliance/compliance_dashboard/img/compliance_dashboard_v12_8.png diff --git a/doc/user/compliance/compliance_dashboard/index.md b/doc/user/compliance/compliance_dashboard/index.md new file mode 100644 index 00000000000..afe3ce185e6 --- /dev/null +++ b/doc/user/compliance/compliance_dashboard/index.md @@ -0,0 +1,31 @@ +--- +type: reference, howto +--- + +# Compliance Dashboard **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/36524) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.8. + +The Compliance Dashboard gives you the ability to see a group's Merge Request activity +by providing a high-level view for all projects in the group. For example, code approved +for merging into production. + +## Overview + +To access the Compliance Dashboard for a group, navigate to **{shield}** **Security & Compliance > Compliance** on the group's menu. + + + +## Use cases + +This feature is for people who care about the compliance status of projects within their group. + +You can use the dashboard to: + +- Get an overview of the latest Merge Request for each project. +- See if Merge Requests were approved and by whom. + +## Permissions + +- On [GitLab Ultimate](https://about.gitlab.com/pricing/) tier. +- By **Administrators** and **Group Owners**. diff --git a/doc/user/compliance/index.md b/doc/user/compliance/index.md new file mode 100644 index 00000000000..fd4af74e086 --- /dev/null +++ b/doc/user/compliance/index.md @@ -0,0 +1,10 @@ +# Compliance **(ULTIMATE)** + +The compliance tools provided by GitLab let you keep an eye on various aspects of your project. The +following compliance tools are available: + +- [Compliance Dashboard](compliance_dashboard/index.md): View recent merge request activity across + all projects in a group. This lets you see if merge requests were approved, and by whom. +- [License Compliance](license_compliance/index.md): Search your project's dependencies for their + licenses. This lets you determine if the licenses of your project's dependencies are compatible + with your project's license. diff --git a/doc/user/application_security/license_compliance/img/license_compliance.png b/doc/user/compliance/license_compliance/img/license_compliance.png Binary files differindex cdce6b5fe38..cdce6b5fe38 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance.png +++ b/doc/user/compliance/license_compliance/img/license_compliance.png diff --git a/doc/user/application_security/license_compliance/img/license_compliance_add_license_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_add_license_v12_3.png Binary files differindex 79f6160e63f..79f6160e63f 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance_add_license_v12_3.png +++ b/doc/user/compliance/license_compliance/img/license_compliance_add_license_v12_3.png diff --git a/doc/user/application_security/license_compliance/img/license_compliance_decision.png b/doc/user/compliance/license_compliance/img/license_compliance_decision.png Binary files differindex fbf90bec7fd..fbf90bec7fd 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance_decision.png +++ b/doc/user/compliance/license_compliance/img/license_compliance_decision.png diff --git a/doc/user/application_security/license_compliance/img/license_compliance_pipeline_tab_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v12_3.png Binary files differindex fd519d63b3e..fd519d63b3e 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance_pipeline_tab_v12_3.png +++ b/doc/user/compliance/license_compliance/img/license_compliance_pipeline_tab_v12_3.png diff --git a/doc/user/application_security/license_compliance/img/license_compliance_search_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_search_v12_3.png Binary files differindex 4a7cff2e85c..4a7cff2e85c 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance_search_v12_3.png +++ b/doc/user/compliance/license_compliance/img/license_compliance_search_v12_3.png diff --git a/doc/user/application_security/license_compliance/img/license_compliance_settings_v12_3.png b/doc/user/compliance/license_compliance/img/license_compliance_settings_v12_3.png Binary files differindex 72d0888a9dc..72d0888a9dc 100644 --- a/doc/user/application_security/license_compliance/img/license_compliance_settings_v12_3.png +++ b/doc/user/compliance/license_compliance/img/license_compliance_settings_v12_3.png diff --git a/doc/user/application_security/license_compliance/img/license_list_v12_6.png b/doc/user/compliance/license_compliance/img/license_list_v12_6.png Binary files differindex 8f2b510be0d..8f2b510be0d 100644 --- a/doc/user/application_security/license_compliance/img/license_list_v12_6.png +++ b/doc/user/compliance/license_compliance/img/license_list_v12_6.png diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md new file mode 100644 index 00000000000..03a7ffd195a --- /dev/null +++ b/doc/user/compliance/license_compliance/index.md @@ -0,0 +1,326 @@ +--- +type: reference, howto +--- + +# License Compliance **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5483) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. + +## Overview + +If you are using [GitLab CI/CD](../../../ci/README.md), you can search your project dependencies for their licenses +using License Compliance. + +You can take advantage of License Compliance by either [including the job](#configuration) +in your existing `.gitlab-ci.yml` file or by implicitly using +[Auto License Compliance](../../../topics/autodevops/index.md#auto-license-compliance-ultimate) +that is provided by [Auto DevOps](../../../topics/autodevops/index.md). + +GitLab checks the License Compliance report, compares the licenses between the +source and target branches, and shows the information right on the merge request. +Blacklisted licenses will be clearly visible with an `x` red icon next to them +as well as new licenses which need a decision from you. In addition, you can +[manually approve or blacklist](#project-policies-for-license-compliance) +licenses in your project's settings. + +NOTE: **Note:** +If the license compliance report doesn't have anything to compare to, no information +will be displayed in the merge request area. That is the case when you add the +`license_scanning` job in your `.gitlab-ci.yml` for the first time. +Consecutive merge requests will have something to compare to and the license +compliance report will be shown properly. + + + +If you are a project or group Maintainer, you can click on a license to be given +the choice to approve it or blacklist it. + + + +## Use cases + +It helps you find what licenses your project uses in its dependencies, and decide for each of then +whether to allow it or forbid it. For example, your application is using an external (open source) +library whose license is incompatible with yours. + +## Supported languages and package managers + +The following languages and package managers are supported. + +| Language | Package managers | Scan Tool | +|------------|-------------------------------------------------------------------|----------------------------------------------------------| +| JavaScript | [Bower](https://bower.io/), [npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Go | [Godep](https://github.com/tools/godep), go get ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), gvt ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), glide ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), dep ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), trash ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) and govendor ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)), [go mod](https://github.com/golang/go/wiki/Modules) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Java | [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| .NET | [Nuget](https://www.nuget.org/) (.NET Framework is supported via the [mono project](https://www.mono-project.com/). Windows specific dependencies are not supported at this time.) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Python | [pip](https://pip.pypa.io/en/stable/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Ruby | [gem](https://rubygems.org/) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Erlang | [rebar](https://www.rebar3.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| +| Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage) , [CocoaPods v0.39 and below](https://cocoapods.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| Elixir | [mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types)) |[License Finder](https://github.com/pivotal/LicenseFinder)| +| C++/C | [conan](https://conan.io/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| +| Scala | [sbt](https://www.scala-sbt.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| +| Rust | [cargo](https://crates.io) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| +| PHP | [composer](https://getcomposer.org/) ([experimental support](https://github.com/pivotal/LicenseFinder#experimental-project-types))|[License Finder](https://github.com/pivotal/LicenseFinder)| + +## Requirements + +To run a License Compliance scanning job, you need GitLab Runner with the +[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). + +## Configuration + +For GitLab 12.8 and later, to enable License Compliance, you must +[include](../../../ci/yaml/README.md#includetemplate) the +[`License-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/License-Scanning.gitlab-ci.yml) +that's provided as a part of your GitLab installation. +For older versions of GitLab from 11.9 to 12.7, you must +[include](../../../ci/yaml/README.md#includetemplate) the +[`License-Management.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/License-Management.gitlab-ci.yml). +For GitLab versions earlier than 11.9, you can copy and use the job as defined +that template. + +NOTE: **Note:** +In GitLab 13.0, the `License-Management.gitlab-ci.yml` template is scheduled to be removed. +Use `License-Scanning.gitlab-ci.yml` instead. + +Add the following to your `.gitlab-ci.yml` file: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml +``` + +The included template will create a `license_scanning` job in your CI/CD pipeline +and scan your dependencies to find their licenses. + +NOTE: **Note:** +Before GitLab 12.8, the `license_scanning` job was named `license_management`. +In GitLab 13.0, the `license_management` job is scheduled to be removed completely, +so you're advised to migrate to the `license_scanning` job and used the new +`License-Scanning.gitlab-ci.yml` template. + +The results will be saved as a +[License Compliance report artifact](../../../ci/yaml/README.md#artifactsreportslicense_scanning-ultimate) +that you can later download and analyze. Due to implementation limitations, we +always take the latest License Compliance artifact available. Behind the scenes, the +[GitLab License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/license-management) +is used to detect the languages/frameworks and in turn analyzes the licenses. + +The License Compliance settings can be changed through [environment variables](#available-variables) by using the +[`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. + +### Available variables + +License Compliance can be configured using environment variables. + +| Environment variable | Required | Description | +|-----------------------|----------|-------------| +| `MAVEN_CLI_OPTS` | no | Additional arguments for the mvn executable. If not supplied, defaults to `-DskipTests`. | +| `LICENSE_FINDER_CLI_OPTS` | no | Additional arguments for the `license_finder` executable. For example, if your project has both Golang and Ruby code stored in different directories and you want to only scan the Ruby code, you can update your `.gitlab-ci-yml` template to specify which project directories to scan, like `LICENSE_FINDER_CLI_OPTS: '--debug --aggregate-paths=. ruby'`. | +| `LM_JAVA_VERSION` | no | Version of Java. If set to `11`, Maven and Gradle use Java 11 instead of Java 8. | +| `LM_PYTHON_VERSION` | no | Version of Python. If set to `3`, dependencies are installed using Python 3 instead of Python 2.7. | +| `SETUP_CMD` | no | Custom setup for the dependency installation. (experimental) | + +### Installing custom dependencies + +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. + +The `license_management` image already embeds many auto-detection scripts, languages, +and packages. Nevertheless, it's almost impossible to cover all cases for all projects. +That's why sometimes it's necessary to install extra packages, or to have extra steps +in the project automated setup, like the download and installation of a certificate. +For that, a `LICENSE_MANAGEMENT_SETUP_CMD` environment variable can be passed to the container, +with the required commands to run before the license detection. + +If present, this variable will override the setup step necessary to install all the packages +of your application (e.g.: for a project with a `Gemfile`, the setup step could be +`bundle install`). + +For example: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +variables: + LICENSE_MANAGEMENT_SETUP_CMD: sh my-custom-install-script.sh +``` + +In this example, `my-custom-install-script.sh` is a shell script at the root +directory of your project. + +### Overriding the template + +If you want to override the job definition (for example, change properties like +`variables` or `dependencies`), you need to declare a `license_scanning` job +after the template inclusion and specify any additional keys under it. For example: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +license_scanning: + variables: + CI_DEBUG_TRACE: "true" +``` + +### Configuring Maven projects + +The License Compliance tool provides a `MAVEN_CLI_OPTS` environment variable which can hold +the command line arguments to pass to the `mvn install` command which is executed under the hood. +Feel free to use it for the customization of Maven execution. For example: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +license_scanning: + variables: + MAVEN_CLI_OPTS: --debug +``` + +`mvn install` runs through all of the [build life cycle](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) +stages prior to `install`, including `test`. Running unit tests is not directly +necessary for the license scanning purposes and consumes time, so it's skipped +by having the default value of `MAVEN_CLI_OPTS` as `-DskipTests`. If you want +to supply custom `MAVEN_CLI_OPTS` and skip tests at the same time, don't forget +to explicitly add `-DskipTests` to your options. +If you still need to run tests during `mvn install`, add `-DskipTests=false` to +`MAVEN_CLI_OPTS`. + +### Selecting the version of Python + +> - [Introduced](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/36) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. +> - In [GitLab 12.2](https://gitlab.com/gitlab-org/gitlab/issues/12032), Python 3.5 became the default. +> - In [GitLab 12.7](https://gitlab.com/gitlab-org/security-products/license-management/-/merge_requests/101), Python 3.8 became the default. + +License Compliance uses Python 3.8 and pip 19.1 by default. +If your project requires Python 2, you can switch to Python 2.7 and pip 10.0 +by setting the `LM_PYTHON_VERSION` environment variable to `2`. + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +license_scanning: + variables: + LM_PYTHON_VERSION: 2 +``` + +### Migration from `license_management` to `license_scanning` + +In GitLab 12.8 a new name for `license_management` job was introduced. This change was made to improve clarity around the purpose of the scan, which is to scan and collect the types of licenses present in a projects dependencies. +The support of `license_management` is scheduled to be dropped in GitLab 13.0. +If you're using a custom setup for License Compliance, you're required +to update your CI config accordingly: + +1. Change the CI template to `License-Scanning.gitlab-ci.yml`. +1. Change the job name to `license_scanning` (if you mention it in `.gitlab-ci.yml`). +1. Change the artifact name to `gl-license-scanning-report.json` (if you mention it in `.gitlab-ci.yml`). + +For example, the following `.gitlab-ci.yml`: + +```yaml +include: + - template: License-Management.gitlab-ci.yml + +license_management: + artifacts: + reports: + license_management: gl-license-management-report.json +``` + +Should be changed to: + +```yaml +include: + - template: License-Scanning.gitlab-ci.yml + +license_scanning: + artifacts: + reports: + license_scanning: gl-license-scanning-report.json +``` + +## Project policies for License Compliance + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5940) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.4. + +From the project's settings: + +- The list of licenses and their status can be managed. +- Licenses can be manually approved or blacklisted. + +To approve or blacklist a license: + +1. Either use the **Manage licenses** button in the merge request widget, or + navigate to the project's **Settings > CI/CD** and expand the + **License Compliance** section. +1. Click the **Add a license** button. + +  + +1. In the **License name** dropdown, either: + - Select one of the available licenses. You can search for licenses in the field + at the top of the list. + - Enter arbitrary text in the field at the top of the list. This will cause the text to be + added as a license name to the list. +1. Select the **Approve** or **Blacklist** radio button to approve or blacklist respectively + the selected license. + +To modify an existing license: + +1. In the **License Compliance** list, click the **Approved/Declined** dropdown to change it to the desired status. + +  + +Searching for Licenses: + +1. Use the **Search** box to search for a specific license. + +  + +## License Compliance report under pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5491) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2. + +From your project's left sidebar, navigate to **CI/CD > Pipelines** and click on the +pipeline ID that has a `license_management` job to see the Licenses tab with the listed +licenses (if any). + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + +## License list + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13582) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.7. + +The License list allows you to see your project's licenses and key +details about them. + +In order for the licenses to appear under the license list, the following +requirements must be met: + +1. The License Compliance CI job must be [configured](#configuration) for your project. +1. Your project must use at least one of the + [supported languages and package managers](#supported-languages-and-package-managers). + +Once everything is set, navigate to **Security & Compliance > License Compliance** +in your project's sidebar, and you'll see the licenses displayed, where: + +- **Name:** The name of the license. +- **Component:** The components which have this license. + + diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index a81c0beb6d0..4ab615a1008 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -25,8 +25,18 @@ specify themselves as a code owner, all before the new changes get merged to the default branch. When a file matches multiple entries in the `CODEOWNERS` file, -the users from all entries are displayed on the blob page of -the given file. +the users from last pattern matching the file are displayed on the +blob page of the given file. For example, you have the following +`CODEOWNERS` file: + +``` +README.md @user1 + +# This line would also match the file README.md +*.md @user2 +``` + +The user that would show for `README.md` would be `@user2`. ## Approvals by Code Owners diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index a4c844b415d..2ee4bef97ce 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -29,15 +29,14 @@ If you only need to migrate Git repos, you can [import each project by URL](repo If you want to retain all metadata like issues and merge requests, you can use the [import/export feature](../settings/import_export.md) to export projects from self-managed GitLab and import those projects into GitLab.com. -NOTE: **Note:** -This approach assumes all users from the self-managed instance have already been migrated. -If the users haven't been migrated yet, the user conducting the import -will take the place of all references to the missing user(s). +All GitLab user associations (such as comment author) will be changed to the user importing the project. For more information, please see [the import notes](../settings/import_export.md#important-notes). If you need to migrate all data over, you can leverage our [API](../../../api/README.md) to migrate from self-managed to GitLab.com. The order of assets to migrate from a self-managed instance to GitLab.com is the following: -1. [Users](../../../api/users.md) +NOTE: **Note:** +When migrating to GitLab.com, users would need to be manually created unless [SCIM](../../../user/group/saml_sso/scim_setup.md) is going to be used. Creating users with the API is limited to self-hosted instances as it requires administrator access. + 1. [Groups](../../../api/groups.md) 1. [Projects](../../../api/projects.md) 1. [Project variables](../../../api/project_level_variables.md) @@ -56,3 +55,5 @@ then restore it on the new server. In the event of merging two GitLab instances together (for example, both instances have existing data on them and one can't be wiped), refer to the instructions in [Migrating from self-managed GitLab to GitLab.com](#migrating-from-self-managed-gitlab-to-gitlabcom). + +Additionally, you can migrate users using the [Users API](../../../api/users.md) with an admin user. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 661c1eebf6d..ea36092168d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -100,7 +100,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Maven packages](../packages/maven_repository/index.md): your private Maven repository in GitLab. **(PREMIUM)** - [NPM packages](../packages/npm_registry/index.md): your private NPM package registry in GitLab. **(PREMIUM)** - [Code owners](code_owners.md): specify code owners for certain files **(STARTER)** -- [License Compliance](../application_security/license_compliance/index.md): approve and blacklist licenses for projects. **(ULTIMATE)** +- [License Compliance](../compliance/license_compliance/index.md): approve and blacklist licenses for projects. **(ULTIMATE)** - [Dependency List](../application_security/dependency_list/index.md): view project dependencies. **(ULTIMATE)** ### Project integrations diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index edf34c666f8..e0bf32baea6 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -21,7 +21,7 @@ A. Consider you are a software developer working in a team: 1. You gather feedback from your team 1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) **(STARTER)** 1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD -1. You avoid using dependencies whose license is not compatible with your project with [License Compliance reports](../../application_security/license_compliance/index.md) **(ULTIMATE)** +1. You avoid using dependencies whose license is not compatible with your project with [License Compliance reports](../../compliance/license_compliance/index.md) **(ULTIMATE)** 1. You request the [approval](merge_request_approvals.md) from your manager **(STARTER)** 1. Your manager: 1. Pushes a commit with their final review @@ -97,6 +97,7 @@ or link to useful information directly in the merge request page: | [Display arbitrary job artifacts](../../../ci/yaml/README.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../pipelines/job_artifacts.md) in merge requests. | | [GitLab CI/CD](../../../ci/README.md) | Build, test, and deploy your code in a per-branch basis with built-in CI/CD. | | [JUnit test reports](../../../ci/junit_test_reports.md) | Configure your CI jobs to use JUnit test reports, and let GitLab display a report on the merge request so that it’s easier and faster to identify the failure without having to check the entire job log. | +| [License Compliance](../../compliance/license_compliance/index.md) **(ULTIMATE)** | Manage the licenses of your dependencies. | | [Metrics Reports](../../../ci/metrics_reports.md) **(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. | | [Multi-Project pipelines](../../../ci/multi_project_pipelines.md) **(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. | | [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. | @@ -112,7 +113,6 @@ generated by scanning and reporting any vulnerabilities found in your project: | [Container Scanning](../../application_security/container_scanning/index.md) | Analyze your Docker images for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](../../application_security/dast/index.md) | Analyze your running web applications for known vulnerabilities. | | [Dependency Scanning](../../application_security/dependency_scanning/index.md) | Analyze your dependencies for known vulnerabilities. | -| [License Compliance](../../application_security/license_compliance/index.md) | Manage the licenses of your dependencies. | | [Static Application Security Testing (SAST)](../../application_security/sast/index.md) | Analyze your source code for known vulnerabilities. | ## Authorization for merge requests diff --git a/doc/user/project/merge_requests/license_management.md b/doc/user/project/merge_requests/license_management.md index df5bd073ade..ed81eb8ca10 100644 --- a/doc/user/project/merge_requests/license_management.md +++ b/doc/user/project/merge_requests/license_management.md @@ -1,5 +1,5 @@ --- -redirect_to: '../../application_security/license_compliance/index.md' +redirect_to: '../../compliance/license_compliance/index.md' --- -This document was moved to [another location](../../application_security/license_compliance/index.md). +This document was moved to [another location](../../compliance/license_compliance/index.md). diff --git a/doc/user/project/repository/repository_mirroring.md b/doc/user/project/repository/repository_mirroring.md index fe57df38d37..33fa09255e1 100644 --- a/doc/user/project/repository/repository_mirroring.md +++ b/doc/user/project/repository/repository_mirroring.md @@ -69,6 +69,11 @@ Changes pushed to files in the repository are automatically pushed to the remote In the case of a diverged branch, you will see an error indicated at the **Mirroring repositories** section. +### Configuring push mirrors through the API + +You can also create and modify project push mirrors through the +[remote mirrors API](../../../api/remote_mirrors.md). + ### Push only protected branches **(CORE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3350) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. |
