diff options
Diffstat (limited to 'doc/user/application_security')
29 files changed, 715 insertions, 682 deletions
diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index ed94686b7a3..f97e09f33bb 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -139,7 +139,7 @@ OpenAPI 2.x lets you specify the accepted media types globally or per operation, - In [GitLab 14.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/333304), the default behavior is to select one of the supported media types to use. The first supported media type is chosen from the list. This behavior is configurable. - In GitLab 14.9 and earlier, the default behavior is to perform testing using all supported media types. This means if two media types are listed (for example, `application/json` and `application/xml`), tests are performed using JSON, and then the same tests using XML. -Testing the same operation (for example, `POST /user`) using different media types (for example, `application/json` and `application/xml`) is not always desirable. +Testing the same operation (for example, `POST /user`) using different media types (for example, `application/json` and `application/xml`) is not always desirable. For example, if the target application executes the same code regardless of the request content type, it will take longer to finish the test session, and it may report duplicate vulnerabilities related to the request body depending on the target app. The environment variable `FUZZAPI_OPENAPI_ALL_MEDIA_TYPES` lets you specify whether or not to use all supported media types instead of one when generating requests for a given operation. When the environmental variable `FUZZAPI_OPENAPI_ALL_MEDIA_TYPES` is set to any value, API Fuzzing will try to generate requests for all supported media types instead of one in a given operation. This will cause testing to take longer as testing is repeated for each provided media type. @@ -1087,7 +1087,7 @@ You can provide the following properties to exclude specific parameters during t - `body-json`: Use this property to exclude specific JSON nodes from a request that uses the media type `application/json`. The property's value is an array, each entry of the array is a [JSON Path](https://goessner.net/articles/JsonPath/) expression. - `body-xml`: Use this property to exclude specific XML nodes from a request that uses media type `application/xml`. The property's value is an array, each entry of the array is a [XPath v2](https://www.w3.org/TR/xpath20/) expression. -The following JSON document is an example of the expected structure to exclude parameters. +The following JSON document is an example of the expected structure to exclude parameters. ```json { @@ -1155,11 +1155,11 @@ To exclude the `password` field in a request that uses `application/x-www-form-u The exclude parameters uses `body-form` when the request uses a content type `application/x-www-form-urlencoded`. -##### Excluding a specific JSON nodes using JSON Path +##### Excluding a specific JSON nodes using JSON Path To exclude the `schema` property in the root object, set the `body-json` property's value to an array with the JSON Path expression `[ "$.schema" ]`. -The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. +The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. For instance, the JSON document looks like this: ```json @@ -1168,13 +1168,13 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path, characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path, characters like `$`, `*`, `.` among others have special meaning. -##### Excluding multiple JSON nodes using JSON Path +##### Excluding multiple JSON nodes using JSON Path To exclude the property `password` on each entry of an array of `users` at the root level, set the `body-json` property's value to an array with the JSON Path expression `[ "$.users[*].paswword" ]`. -The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. +The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. For instance, the JSON document looks like this: @@ -1184,7 +1184,7 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. ##### Excluding an XML attribute @@ -1196,17 +1196,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/@isEnabled" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions, characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions, characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element's text -To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. +To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. In the XPath expression `/credentials/username/text()`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. Last part has a `/` that refers to the current element, and uses a XPath function called `text()` which identifies the text of the current element. @@ -1214,17 +1214,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username/text()" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element -To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. +To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. In the XPath expression `/credentials/username`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. @@ -1232,7 +1232,7 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username" ] } @@ -1242,21 +1242,21 @@ The exclude parameters uses `body-xml` when the request uses a content type `app ##### Excluding an XML node with namespaces -To exclude a XML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. +To exclude a XML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. -In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. +In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. The namespace name should have been defined in the XML document which is part of the body request. You may check the namespace in the specification document HAR, OpenAPI, or Postman Collection file. ```json { - "body-xml": [ + "body-xml": [ "/credentials/s:login" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. #### Using a JSON string @@ -1294,7 +1294,7 @@ variables: FUZZAPI_EXCLUDE_PARAMETER_FILE: api-fuzzing-exclude-parameters.json ``` -The `api-fuzzing-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). +The `api-fuzzing-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). ### Exclude URLS @@ -1348,7 +1348,7 @@ variables: ##### Excluding URL using regular expressions -In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. +In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml variables: @@ -1467,7 +1467,7 @@ reported. Faults detected by API Fuzzing occur in the live web application, and require manual investigation to determine if they are vulnerabilities. Fuzzing faults are included as vulnerabilities with a severity of Unknown. To facilitate investigation of the fuzzing faults, detailed information is -provided about the HTTP messages sent and received along with a description of the modification(s) +provided about the HTTP messages sent and received along with a description of the modifications made. Follow these steps to view details of a fuzzing fault: diff --git a/doc/user/application_security/cluster_image_scanning/index.md b/doc/user/application_security/cluster_image_scanning/index.md index 293645b8de6..aba28a5ca89 100644 --- a/doc/user/application_security/cluster_image_scanning/index.md +++ b/doc/user/application_security/cluster_image_scanning/index.md @@ -28,10 +28,17 @@ To integrate GitLab with security scanners other than those listed here, see You can use cluster image scanning through the following methods: -- [The cluster image scanning analyzer](#use-the-cluster-image-scanning-analyzer) +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> +- [The cluster image scanning analyzer](#use-the-cluster-image-scanning-analyzer-removed) ([Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in GitLab 15.0. Use [the GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) instead.) +<!--- end_remove --> - [The GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) -## Use the cluster image scanning analyzer +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> + +## Use the cluster image scanning analyzer (removed) + +This feature was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in GitLab 15.0. +Use [the GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) instead. You can use the cluster image scanning analyzer to run cluster image scanning with [GitLab CI/CD](../../../ci/index.md). To enable the cluster image scanning analyzer, [include the CI job](#configuration) @@ -277,6 +284,7 @@ Here's an example cluster image scanning report: } ``` +<!--- end_remove --> ## Cluster image scanning with the GitLab agent You can use the [GitLab agent](../../clusters/agent/index.md) to @@ -284,14 +292,12 @@ scan images from within your Kubernetes cluster and record the vulnerabilities i ### Prerequisites -- [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/installation/kubectl/) - installed and configured in your cluster. - [GitLab agent](../../clusters/agent/install/index.md) set up in GitLab, installed in your cluster, and configured using a configuration repository. ### Configuration -The agent runs the cluster image scanning once the `cluster_image_scanning` +The agent runs the cluster image scanning once the `starboard` directive is added to your [agent's configuration repository](../../clusters/agent/vulnerabilities.md). ## Security Dashboard @@ -304,9 +310,12 @@ the security vulnerabilities in your groups, projects, and pipelines. After you find a vulnerability, you can address it in the [vulnerability report](../vulnerabilities/index.md) or the [GitLab agent's](../../clusters/agent/vulnerabilities.md) details section. +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> ## Troubleshooting ### Getting warning message `gl-cluster-image-scanning-report.json: no matching files` For information on this error, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + +<!--- end_remove --> diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 64566e458ee..2828b56a5d1 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -5,7 +5,7 @@ group: Container Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Container Scanning **(ULTIMATE)** +# Container Scanning **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3672) in GitLab 10.4. @@ -44,6 +44,26 @@ information directly in the merge request.  +### Capabilities + +| Capability | In Free | In Ultimate | +| --- | ------ | ------ | +| [Configure Scanners](#configuration) | Yes | Yes | +| Customize Settings ([Variables](#available-cicd-variables), [Overriding](#overriding-the-container-scanning-template), [offline environment support](#running-container-scanning-in-an-offline-environment), etc) | Yes | Yes | +| [View JSON Report](#reports-json-format) as a CI job artifact | Yes | Yes | +| Generation of a JSON report of [dependencies](#dependency-list) as a CI job artifact | Yes | Yes | +| Ability to enable container scanning via an MR in the GitLab UI | Yes | Yes | +| [UBI Image Support](#fips-enabled-images) | Yes | Yes | +| Support for Trivy | Yes | Yes | +| Support for Grype | Yes | Yes | +| Inclusion of GitLab Advisory Database | Limited to the time-delayed content from GitLab [advisories-communities](https://gitlab.com/gitlab-org/advisories-community/) project | Yes - all the latest content from [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) | +| Presentation of Report data in Merge Request and Security tab of the CI pipeline job | No | Yes | +| [Interaction with Vulnerabilities](#interacting-with-the-vulnerabilities) such as merge request approvals | No | Yes | +| [Solutions for vulnerabilities (auto-remediation)](#solutions-for-vulnerabilities-auto-remediation) | No | Yes | +| Support for the [vulnerability allow list](#vulnerability-allowlisting) | No | Yes | +| [Access to Security Dashboard page](#security-dashboard) | No | Yes | +| [Access to Dependency List page](../dependency_list/) | No | Yes | + ## Requirements To enable container scanning in your pipeline, you need the following: @@ -164,8 +184,8 @@ include: The `CS_DISABLE_DEPENDENCY_LIST` CI/CD variable controls whether the scan creates a [Dependency List](../dependency_list/) -report. The variable's default setting of `false` causes the scan to create the report. To disable -the report, set the variable to `true`: +report. This variable is currently only supported when the `trivy` analyzer is used. The variable's default setting of `"false"` causes the scan to create the report. To disable +the report, set the variable to `"true"`: For example: @@ -205,8 +225,9 @@ container_scanning: When you enable this feature, you may see [duplicate findings](../terminology/#duplicate-finding) in the [Vulnerability Report](../vulnerability_report/) if [Dependency Scanning](../dependency_scanning/) -is enabled for your project. This happens because GitLab can't automatically deduplicate the -findings reported by the two different analyzers. +is enabled for your project. This happens because GitLab can't automatically deduplicate findings +across different types of scanning tools. Please reference [this comparison](../dependency_scanning/#dependency-scanning-compared-to-container-scanning) +between GitLab Dependency Scanning and Container Scanning for more details on which types of dependencies are likely to be duplicated. #### Available CI/CD variables @@ -217,7 +238,7 @@ You can [configure](#customizing-the-container-scanning-settings) analyzers by u | `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | All | | `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | All | | `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | All | -| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:4` | Docker image of the analyzer. | All | +| `CS_ANALYZER_IMAGE` | `registry.gitlab.com/security-products/container-scanning:5` | Docker image of the analyzer. | All | | `CS_DEFAULT_BRANCH_IMAGE` | `""` | The name of the `DOCKER_IMAGE` on the default branch. See [Setting the default branch image](#setting-the-default-branch-image) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338877) in GitLab 14.5. | All | | `CS_DISABLE_DEPENDENCY_LIST` | `"false"` | Disable Dependency Scanning for packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` | `"true"` | Disable scanning for language-specific packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | @@ -234,10 +255,24 @@ You can [configure](#customizing-the-container-scanning-settings) analyzers by u ### Supported distributions -Support depends on the scanner: - -- [Grype](https://github.com/anchore/grype#grype) -- [Trivy](https://aquasecurity.github.io/trivy/latest/docs/vulnerability/detection/os/) (Default). +Support depends on which scanner is used: + +| Distribution | Grype | Trivy | +| -------------- | ----- | ----- | +| Alma Linux | | ✅ | +| Alpine Linux | ✅ | | +| Amazon Linux | ✅ | ✅ | +| BusyBox | ✅ | | +| CentOS | ✅ | ✅ | +| CBL-Mariner | | ✅ | +| Debian | ✅ | ✅ | +| Distroless | ✅ | ✅ | +| Oracle Linux | ✅ | ✅ | +| Photon OS | | ✅ | +| Red Hat (RHEL) | ✅ | ✅ | +| Rocky Linux | | ✅ | +| SUSE | | ✅ | +| Ubuntu | ✅ | ✅ | #### FIPS-enabled images @@ -250,9 +285,9 @@ standard tag plus the `-fips` extension. | Scanner name | `CS_ANALYZER_IMAGE` | | --------------- | ------------------- | -| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:4-fips` | -| Grype | `registry.gitlab.com/security-products/container-scanning/grype:4-fips` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4-fips` | +| Default (Trivy) | `registry.gitlab.com/security-products/container-scanning:5-fips` | +| Grype | `registry.gitlab.com/security-products/container-scanning/grype:5-fips` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5-fips` | NOTE: Prior to GitLab 15.0, the `-ubi` image extension is also available. GitLab 15.0 and later only @@ -305,9 +340,9 @@ The following options are available: | Scanner name | `CS_ANALYZER_IMAGE` | | ------------ | ------------------- | -| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:4` | -| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:4` | -| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4` | +| Default ([Trivy](https://github.com/aquasecurity/trivy)) | `registry.gitlab.com/security-products/container-scanning:5` | +| [Grype](https://github.com/anchore/grype) | `registry.gitlab.com/security-products/container-scanning/grype:5` | +| Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:5` | If you're migrating from a GitLab 13.x release to a GitLab 14.x release and have customized the `container_scanning` job or its CI variables, you might need to perform these migration steps in @@ -320,7 +355,7 @@ your CI file: - `SECURE_ANALYZERS_PREFIX` 1. Review the `CS_ANALYZER_IMAGE` variable. It no longer depends on the variables above and its new - default value is `registry.gitlab.com/security-products/container-scanning:4`. If you have an + default value is `registry.gitlab.com/security-products/container-scanning:5`. If you have an offline environment, see [Running container scanning in an offline environment](#running-container-scanning-in-an-offline-environment). @@ -405,7 +440,7 @@ container_scanning: The `ADDITIONAL_CA_CERT_BUNDLE` value can also be configured as a [custom variable in the UI](../../../ci/variables/index.md#custom-cicd-variables), either as a `file`, which requires the path to the certificate, or as a variable, which requires the text representation of the certificate. -### Vulnerability allowlisting +### Vulnerability allowlisting **(ULTIMATE)** To allowlist specific vulnerabilities, follow these steps: @@ -532,9 +567,9 @@ For container scanning, import the following images from `registry.gitlab.com` i [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/container-scanning:4 -registry.gitlab.com/security-products/container-scanning/grype:4 -registry.gitlab.com/security-products/container-scanning/trivy:4 +registry.gitlab.com/security-products/container-scanning:5 +registry.gitlab.com/security-products/container-scanning/grype:5 +registry.gitlab.com/security-products/container-scanning/trivy:5 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -574,7 +609,7 @@ following `.gitlab-ci.yml` example as a template. ```yaml variables: - SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:4 + SOURCE_IMAGE: registry.gitlab.com/security-products/container-scanning:5 TARGET_IMAGE: $CI_REGISTRY/namespace/gitlab-container-scanning image: docker:stable @@ -753,15 +788,38 @@ Here's an example container scanning report: The [Security Dashboard](../security_dashboard/index.md) shows you an overview of all the security vulnerabilities in your groups, projects and pipelines. -## Vulnerabilities database update +## Vulnerabilities database All analyzer images are [updated daily](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/blob/master/README.md#image-updates). -The images include the latest advisory database available for their respective scanner. Each -scanner includes data from multiple sources: - -- [Grype](https://github.com/anchore/grype#grypes-database). -- [Trivy](https://aquasecurity.github.io/trivy/latest/docs/vulnerability/detection/data-source/). +The images use data from upstream advisory databases depending on which scanner is used: + +| Data Source | Trivy | Grype | +| ------------------------------ | ----- | ----- | +| AlmaLinux Security Advisory | ✅ | ✅ | +| Amazon Linux Security Center | ✅ | ✅ | +| Arch Linux Security Tracker | ✅ | | +| SUSE CVRF | ✅ | ✅ | +| CWE Advisories | ✅ | | +| Debian Security Bug Tracker | ✅ | ✅ | +| GitHub Security Advisory | ✅ | ✅ | +| Go Vulnerability Database | ✅ | | +| CBL-Mariner Vulnerability Data | ✅ | | +| NVD | ✅ | ✅ | +| OSV | ✅ | | +| Red Hat OVAL v2 | ✅ | ✅ | +| Red Hat Security Data API | ✅ | ✅ | +| Photon Security Advisories | ✅ | | +| Rocky Linux UpdateInfo | ✅ | | +| Ubuntu CVE Tracker (only data sources from mid 2021 and later) | ✅ | ✅ | + +In addition to the sources provided by these scanners, GitLab maintains the following vulnerability databases: + +- The proprietary +[GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). +- The open source [GitLab Advisory Database (Open Source Edition)](https://gitlab.com/gitlab-org/advisories-community). + +In the GitLab Ultimate tier, the data from the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is merged in to augment the data from the external sources. In the GitLab Premium and Free tiers, the data from the [GitLab Advisory Database (Open Source Edition)](https://gitlab.com/gitlab-org/advisories-community) is merged in to augment the data from the external sources. This augmentation currently only applies to the analyzer images for the Trivy scanner. Database update information for other analyzers is available in the [maintenance table](../index.md#vulnerability-scanner-maintenance). @@ -770,7 +828,7 @@ Database update information for other analyzers is available in the After a vulnerability is found, you can [address it](../vulnerabilities/index.md). -## Solutions for vulnerabilities (auto-remediation) +## Solutions for vulnerabilities (auto-remediation) **(ULTIMATE)** Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. @@ -827,6 +885,7 @@ For information on this, see the [general Application Security troubleshooting s as the default for container scanning, and also [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326279) an integration with [Grype](https://github.com/anchore/grype) as an alternative scanner. +- GitLab 15.0 changed the major analyzer version from `4` to `5`. Other changes to the container scanning analyzer can be found in the project's [changelog](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/blob/master/CHANGELOG.md). diff --git a/doc/user/application_security/dast/checks/16.3.md b/doc/user/application_security/dast/checks/16.3.md index e4fc2468dae..6f80a2a32c6 100644 --- a/doc/user/application_security/dast/checks/16.3.md +++ b/doc/user/application_security/dast/checks/16.3.md @@ -32,4 +32,4 @@ information from the `X-Powered-By` header. ## Links - [CWE](https://cwe.mitre.org/data/definitions/16.html) -- [PHP expose_php](https://www.php.net/manual/en/ini.core.php#ini.expose-php) +- [PHP `expose_php`](https://www.php.net/manual/en/ini.core.php#ini.expose-php) diff --git a/doc/user/application_security/dast/checks/16.5.md b/doc/user/application_security/dast/checks/16.5.md index 28bb9f7ee4b..e03da3043ef 100644 --- a/doc/user/application_security/dast/checks/16.5.md +++ b/doc/user/application_security/dast/checks/16.5.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -The target website returns AspNet header(s) and version information of this website. By +The target website returns AspNet headers and version information of this website. By exposing these values attackers may attempt to identify if the target software is vulnerable to known vulnerabilities, or catalog known sites running particular versions to exploit in the future when a vulnerability is identified in the particular version. diff --git a/doc/user/application_security/dast/checks/16.6.md b/doc/user/application_security/dast/checks/16.6.md index ddd3a10c5f8..9cbcde669a0 100644 --- a/doc/user/application_security/dast/checks/16.6.md +++ b/doc/user/application_security/dast/checks/16.6.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -The target website returns AspNet header(s) along with version information of this website. By +The target website returns AspNet headers along with version information of this website. By exposing these values attackers may attempt to identify if the target software is vulnerable to known vulnerabilities. Or catalog known sites running particular versions to exploit in the future when a vulnerability is identified in the particular version. diff --git a/doc/user/application_security/dast/checks/359.1.md b/doc/user/application_security/dast/checks/359.1.md new file mode 100644 index 00000000000..af1fdf8a596 --- /dev/null +++ b/doc/user/application_security/dast/checks/359.1.md @@ -0,0 +1,34 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of Private Personal Information (PII) to an unauthorized actor (credit card) + +## Description + +The target application was found to return credit card information in the response. Organizations +found returning such information may be in violation of industry regulations and could face fines. + +## Remediation + +PII such as credit cards should never be directly returned to the user. The majority of the information should masked except +the last few digits or characters of the identifier. For example, credit card numbers should +only return the last four digits: `****-****-****-1234`. Ensure this masking is done on the server +and only then send the masked data back to the client. Do not rely on client side JavaScript or other methods +to mask these values as the data could still be intercepted or unmasked. + +Additionally, credit card information should never be stored un-encrypted in files or databases. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 359.1 | true | 359 | Passive | Medium | + +## Links + +- [OWASP Top 10 A3 2017 - Sensitive Data Exposure](https://owasp.org/www-project-top-ten/2017/A3_2017-Sensitive_Data_Exposure) +- [CWE](https://cwe.mitre.org/data/definitions/359.html) +- [PCI-DSS](https://www.pcisecuritystandards.org/pdfs/pci_fs_data_storage.pdf) diff --git a/doc/user/application_security/dast/checks/359.2.md b/doc/user/application_security/dast/checks/359.2.md new file mode 100644 index 00000000000..beb99e26097 --- /dev/null +++ b/doc/user/application_security/dast/checks/359.2.md @@ -0,0 +1,34 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of Private Personal Information (PII) to an unauthorized actor (United States social security number) + +## Description + +The target application was found to return social security number (SSN) information in the response. Organizations +found returning such information may be in violation of (United States) state or federal laws and may face stiff penalties. + +## Remediation + +PII such as social security numbers should never be directly returned to the user. The majority of the information +should masked except the last few digits or characters of the identifier. For example, social security numbers +only be displayed with the last four digits: `***-**-1234`. Ensure this masking is done on the server +and only then send the masked data back to the client. Do not rely on client side JavaScript or other methods +to mask these values as the data could still be intercepted or unmasked. + +Additionally, social security numbers should never be stored un-encrypted in files or databases. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 359.2 | true | 359 | Passive | Medium | + +## Links + +- [OWASP Top 10 A3 2017 - Sensitive Data Exposure](https://owasp.org/www-project-top-ten/2017/A3_2017-Sensitive_Data_Exposure) +- [CWE](https://cwe.mitre.org/data/definitions/359.html) +- [Privacy Act (CMPPA)](https://www.ssa.gov/dataexchange/privacyinfo.html) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index 764e3c4a839..629ff1c3a8d 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -18,6 +18,8 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne | [16.5](16.5.md) | AspNet header exposes version information | Low | Passive | | [16.6](16.6.md) | AspNetMvc header exposes version information | Low | Passive | | [200.1](200.1.md) | Exposure of sensitive information to an unauthorized actor (private IP address) | Low | Passive | +| [359.1](359.1.md) | Exposure of Private Personal Information (PII) to an unauthorized actor (credit card) | Medium | Passive | +| [359.2](359.2.md) | Exposure of Private Personal Information (PII) to an unauthorized actor (United States social security number) | Medium | Passive | | [548.1](548.1.md) | Exposure of information through directory listing | Low | Passive | | [598.1](598.1.md) | Use of GET request method with sensitive query strings (session ID) | Medium | Passive | | [598.2](598.2.md) | Use of GET request method with sensitive query strings (password) | Medium | Passive | diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index ee57803dfc7..1389db65713 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -627,7 +627,7 @@ These CI/CD variables are specific to DAST. They can be used to customize the be | `DAST_AGGREGATE_VULNERABILITIES` | boolean | Vulnerability aggregation is set to `true` by default. To disable this feature and see each vulnerability individually set to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/254043) in GitLab 14.0. | | `DAST_API_HOST_OVERRIDE` <sup>1</sup> | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080`. | | `DAST_API_OPENAPI` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | -| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290241) in GitLab 13.12 and replaced by `DAST_API_OPENAPI`. To be removed in GitLab 15.0. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_API_SPECIFICATION` <sup>1</sup> | URL or string | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290241)** in GitLab 15.0. Replaced by `DAST_API_OPENAPI`. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. The variable `DAST_WEBSITE` must be specified if this is omitted. | | `DAST_AUTH_REPORT` <sup>2</sup> | boolean | Used in combination with exporting the `gl-dast-debug-auth-report.html` artifact to aid in debugging authentication issues. | | `DAST_AUTH_EXCLUDE_URLS` <sup>2</sup> | URLs | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/289959)** in GitLab 14.0. Replaced by `DAST_EXCLUDE_URLS`. The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | | `DAST_AUTH_URL` <sup>1,2</sup> | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. Example: `https://login.example.com`. | @@ -1156,6 +1156,7 @@ A site profile contains the following: - **Password**: The password used to authenticate to the website. - **Username form field**: The name of username field at the sign-in HTML form. - **Password form field**: The name of password field at the sign-in HTML form. + - **Submit form field**: The `id` or `name` of the element that when clicked submits the sign-in HTML form. When an API site type is selected, a [host override](#host-override) is used to ensure the API being scanned is on the same host as the target. This is done to reduce the risk of running an active scan against the wrong API. @@ -1199,7 +1200,14 @@ The site profile is created. #### Edit a site profile -To edit an existing site profile: +If a site profile is linked to a security policy, a user cannot edit the profile from this page. See +[Scan execution policies](../policies/scan-execution-policies.md) +for more information. + +When a validated site profile's file, header, or meta tag is edited, the site's +[validation status](#site-profile-validation) is revoked. + +To edit a site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. @@ -1207,42 +1215,37 @@ To edit an existing site profile: 1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the fields then select **Save profile**. -If a site profile is linked to a security policy, a user cannot edit the profile from this page. See -[Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Delete a site profile -To delete an existing site profile: +If a site profile is linked to a security policy, a user cannot delete the profile from this page. +See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + +To delete a site profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. 1. Select the **Site Profiles** tab. -1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. +1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete** to confirm the deletion. -If a site profile is linked to a security policy, a user cannot delete the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Validate a site profile -Prerequisites: - -- A site profile. +Validating a site is required to run an active scan. To validate a site profile: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Configuration**. -1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage scans**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. -1. In the profile's row select **Validate** or **Retry validation**. +1. In the profile's row, select **Validate**. 1. Select the validation method. 1. For **Text file validation**: 1. Download the validation file listed in **Step 2**. - 1. Upload the validation file to the host. Upload the file to the location in - **Step 3** or any location you prefer. + 1. Upload the validation file to the host, to the location in **Step 3** or any location you + prefer. + 1. If required, edit the file location in **Step 3**. 1. Select **Validate**. 1. For **Header validation**: 1. Select the clipboard icon in **Step 2**. @@ -1255,9 +1258,8 @@ To validate a site profile: 1. Select the input field in **Step 3** and enter the location of the meta tag. 1. Select **Validate**. -The site is validated and an active scan can run against it. - -If a validated site profile's target URL is edited, the site's validation status is revoked. +The site is validated and an active scan can run against it. A site profile's validation status is +revoked only when it's revoked manually, or its file, header, or meta tag is edited. #### Retry a failed validation @@ -1265,22 +1267,28 @@ If a validated site profile's target URL is edited, the site's validation status > - [Deployed behind the `dast_failed_site_validations` flag](../../../administration/feature_flags.md), enabled by default. > - [Feature flag `dast_failed_site_validations` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323961) in GitLab 14.4. -If a site profile's validation fails, you can retry it by selecting the **Retry validation** button -in the profiles list. +Failed site validation attempts are listed on the **Site profiles** tab of the **Manage profiles** +page. + +To retry a site profile's failed validation: -When loading the DAST profiles library, past failed validations are listed above the profiles -list. You can also retry the validation from there by selecting the **Retry validation** link in -the alert. You can also dismiss the alert to revoke failed validations. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select the **Site Profiles** tab. +1. In the profile's row, select **Retry validation**. #### Revoke a site profile's validation status -Note that all site profiles with the same URL have their validation status revoked. +WARNING: +When a site profile's validation status is revoked, all site profiles that share the same URL also +have their validation status revoked. To revoke a site profile's validation status: 1. From your project's home page, go to **Security & Compliance > Configuration**. 1. In the **DAST Profiles** row select **Manage**. -1. Select **Revoke validation** beside the validated profile. +1. Beside the validated profile, select **Revoke validation**. The site profile's validation status is revoked. @@ -1348,40 +1356,40 @@ A scanner profile defines the scanner settings used to run an on-demand scan: To create a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row select **Manage**. +1. In the **DAST Profiles** row, select **Manage**. 1. Select **New > Scanner Profile**. 1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile). -1. Click **Save profile**. +1. Select **Save profile**. #### Edit a scanner profile +If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. +See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + To edit a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. +1. In the **DAST Profiles** row, select **Manage**. 1. Select the **Scanner Profiles** tab. -1. In the scanner's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. +1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the form. 1. Select **Save profile**. -If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - #### Delete a scanner profile +If a scanner profile is linked to a security policy, a user cannot delete the profile from this +page. See [Scan execution policies](../policies/scan-execution-policies.md) +for more information. + To delete a scanner profile: 1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Click **Manage** in the **DAST Profiles** row. +1. In the **DAST Profiles** row, select **Manage**. 1. Select the **Scanner Profiles** tab. -1. In the scanner's row select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. +1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete**. -If a scanner profile is linked to a security policy, a user cannot delete the profile from this -page. See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. - ## Auditing > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index a4908204b60..a1b19c52b20 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -1041,7 +1041,7 @@ You can provide the following properties to exclude specific parameters during t - `body-json`: Use this property to exclude specific JSON nodes from a request that uses the media type `application/json`. The property's value is an array, each entry of the array is a [JSON Path](https://goessner.net/articles/JsonPath/) expression. - `body-xml`: Use this property to exclude specific XML nodes from a request that uses media type `application/xml`. The property's value is an array, each entry of the array is a [XPath v2](https://www.w3.org/TR/xpath20/) expression. -Thus, the following JSON document is an example of the expected structure to exclude parameters. +Thus, the following JSON document is an example of the expected structure to exclude parameters. ```json { @@ -1109,11 +1109,11 @@ To exclude the `password` field in a request that uses `application/x-www-form-u The exclude parameters uses `body-form` when the request uses a content type `application/x-www-form-urlencoded`. -##### Excluding a specific JSON nodes using JSON Path +##### Excluding a specific JSON nodes using JSON Path To exclude the `schema` property in the root object, set the `body-json` property's value to an array with the JSON Path expression `[ "$.schema" ]`. -The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. +The JSON Path expression uses special syntax to identify JSON nodes: `$` refers to the root of the JSON document, `.` refers to the current object (in our case the root object), and the text `schema` refers to a property name. Thus, the JSON path expression `$.schema` refers to a property `schema` in the root object. For instance, the JSON document looks like this: ```json @@ -1122,13 +1122,13 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. -##### Excluding multiple JSON nodes using JSON Path +##### Excluding multiple JSON nodes using JSON Path To exclude the property `password` on each entry of an array of `users` at the root level, set the `body-json` property's value to an array with the JSON Path expression `[ "$.users[*].paswword" ]`. -The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. +The JSON Path expression starts with `$` to refer to the root node and uses `.` to refer to the current node. Then, it uses `users` to refer to a property and the characters `[` and `]` to enclose the index in the array you want to use, instead of providing a number as an index you use `*` to specify any index. After the index reference, we find `.` which now refers to any given selected index in the array, preceded by a property name `password`. For instance, the JSON document looks like this: @@ -1138,7 +1138,7 @@ For instance, the JSON document looks like this: } ``` -The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. +The exclude parameters uses `body-json` when the request uses a content type `application/json`. Each entry in `body-json` is expected to be a [JSON Path expression](https://goessner.net/articles/JsonPath/). In JSON Path characters like `$`, `*`, `.` among others have special meaning. ##### Excluding a XML attribute @@ -1150,17 +1150,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/@isEnabled" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding a XML text's element -To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. +To exclude the text of the `username` element contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username/text()" ]`. In the XPath expression `/credentials/username/text()`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. Last part has a `/` that refers to the current element, and uses a XPath function called `text()` which identifies the text of the current element. @@ -1168,17 +1168,17 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username/text()" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML element -To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. +To exclude the element `username` contained in root node `credentials`, set the `body-xml` property's value to an array with the XPath expression `[/credentials/username" ]`. In the XPath expression `/credentials/username`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `username`. @@ -1186,31 +1186,31 @@ For instance, the JSON document looks like this: ```json { - "body-xml": [ + "body-xml": [ "/credentials/username" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be a [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. ##### Excluding an XML node with namespaces -To exclude anXML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. +To exclude anXML element `login` which is defined in namespace `s`, and contained in `credentials` root node, set the `body-xml` property's value to an array with the XPath expression `[ "/credentials/s:login" ]`. -In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. +In the XPath expression `/credentials/s:login`, the first character `/` refers to the root XML node, and then after it indicates an XML element's name `credentials`. Similarly, the character `/` refers to the current element, followed by a new XML element's name `s:login`. Notice that name contains the character `:`, this character separates the namespace from the node name. The namespace name should have been defined in the XML document which is part of the body request. You may check the namespace in the specification document HAR, OpenAPI, or Postman Collection file. ```json { - "body-xml": [ + "body-xml": [ "/credentials/s:login" ] } ``` -The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath, expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. +The exclude parameters uses `body-xml` when the request uses a content type `application/xml`. Each entry in `body-xml` is expected to be an [XPath v2 expression](https://www.w3.org/TR/xpath20/). In XPath, expressions characters like `@`, `/`, `:`, `[`, `]` among others have special meanings. #### Using a JSON string @@ -1248,7 +1248,7 @@ variables: DAST_API_EXCLUDE_PARAMETER_FILE: dast-api-exclude-parameters.json ``` -The `dast-api-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). +The `dast-api-exclude-parameters.json` is a JSON document that follows the structure of [exclude parameters document](#exclude-parameters-using-a-json-document). ### Exclude URLS @@ -1302,7 +1302,7 @@ variables: ##### Excluding URL using regular expressions -In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. +In order to exclude exactly `https://target/api/v1/user/create` and `https://target/api/v2/user/create` or any other version (`v3`,`v4`, and more). We could use `https://target/api/v.*/user/create$`, in the previous regular expression `.` indicates any character and `*` indicates zero or more times, additionally `$` indicates that the URL should end there. ```yaml variables: diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 665d29c4017..acbc94cba47 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -20,11 +20,9 @@ This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/ Dependency Scanning supports the following official analyzers: -- [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) - [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) - [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) - [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python) -- [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) The analyzers are published as Docker images, which Dependency Scanning uses to launch dedicated containers for each analysis. @@ -34,11 +32,13 @@ The Dependency Scanning analyzers' current major version number is 2. Dependency Scanning is pre-configured with a set of **default images** that are maintained by GitLab, but users can also integrate their own **custom images**. -WARNING: -The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> -WARNING: -The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). +The [`bundler-audit`](https://gitlab.com/gitlab-org/gitlab/-/issues/289832) and [`retire.js`](https://gitlab.com/gitlab-org/gitlab/-/issues/350510) analyzers were deprecated +in GitLab 14.8 and [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86704) in 15.0. +Use Gemnasium instead. + +<!--- end_remove --> ## Official default analyzers @@ -67,7 +67,7 @@ the official analyzers. ### Disable specific analyzers You can select the official analyzers you don't want to run. Here's how to disable -`bundler-audit` and `gemnasium` analyzers. +the `gemnasium` analyzer. In `.gitlab-ci.yml` define: ```yaml @@ -75,7 +75,7 @@ include: template: Security/Dependency-Scanning.gitlab-ci.yml variables: - DS_EXCLUDED_ANALYZERS: "bundler-audit, gemnasium" + DS_EXCLUDED_ANALYZERS: "gemnasium" ``` ### Disabling default analyzers @@ -88,7 +88,7 @@ include: template: Security/Dependency-Scanning.gitlab-ci.yml variables: - DS_EXCLUDED_ANALYZERS: "gemnasium, gemnasium-maven, gemnasium-python, bundler-audit, retire.js" + DS_EXCLUDED_ANALYZERS: "gemnasium, gemnasium-maven, gemnasium-python" ``` This is used when one totally relies on [custom analyzers](#custom-analyzers). @@ -117,25 +117,25 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Analyzers data -The following table lists the data available for each official analyzer. - -| Property \ Tool | Gemnasium | bundler-audit | Retire.js | -|---------------------------------------|:------------------:|:------------------:|:------------------:| -| Severity | 𐄂 | ✓ | ✓ | -| Title | ✓ | ✓ | ✓ | -| File | ✓ | ⚠ | ✓ | -| Start line | 𐄂 | 𐄂 | 𐄂 | -| End line | 𐄂 | 𐄂 | 𐄂 | -| External ID (for example, CVE) | ✓ | ✓ | ⚠ | -| URLs | ✓ | ✓ | ✓ | -| Internal doc/explanation | ✓ | 𐄂 | 𐄂 | -| Solution | ✓ | ✓ | 𐄂 | -| Confidence | 𐄂 | 𐄂 | 𐄂 | -| Affected item (for example, class or package) | ✓ | ✓ | ✓ | -| Source code extract | 𐄂 | 𐄂 | 𐄂 | -| Internal ID | ✓ | 𐄂 | 𐄂 | -| Date | ✓ | 𐄂 | 𐄂 | -| Credits | ✓ | 𐄂 | 𐄂 | +The following table lists the data available for the Gemnasium analyzer. + +| Property \ Tool | Gemnasium | +|---------------------------------------|:------------------:| +| Severity | 𐄂 | +| Title | ✓ | +| File | ✓ | +| Start line | 𐄂 | +| End line | 𐄂 | +| External ID (for example, CVE) | ✓ | +| URLs | ✓ | +| Internal doc/explanation | ✓ | +| Solution | ✓ | +| Confidence | 𐄂 | +| Affected item (for example, class or package) | ✓ | +| Source code extract | 𐄂 | +| Internal ID | ✓ | +| Date | ✓ | +| Credits | ✓ | - ✓ => we have that data - ⚠ => we have that data, but it's partially reliable, or we need to extract that data from unstructured content diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 924e3838d91..87d49ffa324 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -17,9 +17,11 @@ aspects of inspecting the items your code uses. These items typically include ap dependencies that are almost always imported from external sources, rather than sourced from items you wrote yourself. +## Dependency Scanning compared to Container Scanning + GitLab offers both Dependency Scanning and Container Scanning to ensure coverage for all of these dependency types. To cover as much of your risk area as -possible, we encourage you to use all of our security scanners: +possible, we encourage you to use all of our security scanning tools: - Dependency Scanning analyzes your project and tells you which software dependencies, including upstream dependencies, have been included in your project, and what known @@ -41,6 +43,21 @@ possible, we encourage you to use all of our security scanners: efforts to de-duplicate these findings can be tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348655). +The following table summarizes which types of dependencies each scanning tool can detect: + +| Feature | Dependency Scanning | Container Scanning | +| ----------------------------------------------------------- | ------------------- | ------------------ | +| Identify the manifest, lock file, or static file that introduced the dependency | **{check-circle}** | **{dotted-circle}** | +| Development dependencies | **{check-circle}** | **{dotted-circle}** | +| Dependencies in a lock file committed to your repository | **{check-circle}** | **{check-circle}** <sup>1</sup> | +| Binaries built by Go | **{dotted-circle}** | **{check-circle}** <sup>2</sup> | +| Dynamically-linked language-specific dependencies installed by the Operating System | **{dotted-circle}** | **{check-circle}** | +| Operating system dependencies | **{dotted-circle}** | **{check-circle}** | +| Language-specific dependencies installed on the operating system (not built by your project) | **{dotted-circle}** | **{check-circle}** | + +1. Lock file must be present in the image to be detected. +1. Binary file must be present in the image to be detected. + ## Overview If you're using [GitLab CI/CD](../../../ci/index.md), you can use dependency scanning to analyze @@ -136,9 +153,9 @@ table.supported-languages ul { </thead> <tbody> <tr> - <td rowspan="2">Ruby</td> - <td rowspan="2">N/A</td> - <td rowspan="2"><a href="https://bundler.io/">Bundler</a></td> + <td>Ruby</td> + <td>N/A</td> + <td><a href="https://bundler.io/">Bundler</a></td> <td> <ul> <li><code>Gemfile.lock</code></li> @@ -149,11 +166,6 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td><code>Gemfile.lock</code></td> - <td><a href="https://github.com/rubysec/bundler-audit">bundler-audit</a></td> - <td>N</td> - </tr> - <tr> <td>PHP</td> <td>N/A</td> <td><a href="https://getcomposer.org/">Composer</a></td> @@ -200,9 +212,9 @@ table.supported-languages ul { <td>N</td> </tr> <tr> - <td rowspan="3">JavaScript</td> - <td rowspan="2">N/A</td> - <td rowspan="2"><a href="https://www.npmjs.com/">npm</a></td> + <td rowspan="2">JavaScript</td> + <td>N/A</td> + <td><a href="https://www.npmjs.com/">npm</a></td> <td> <ul> <li><code>package-lock.json</code></li> @@ -213,11 +225,6 @@ table.supported-languages ul { <td>Y</td> </tr> <tr> - <td><code>package.json</code></td> - <td><a href="https://retirejs.github.io/retire.js/">Retire.js</a></td> - <td>N</td> - </tr> - <tr> <td>N/A</td> <td><a href="https://classic.yarnpkg.com/en/">yarn</a></td> <td><code>yarn.lock</code></td> @@ -236,8 +243,8 @@ table.supported-languages ul { <td>C#</td> </tr> <tr> - <td rowspan="3">Python</td> - <td rowspan="3">3.6</td> + <td rowspan="4">Python</td> + <td rowspan="4">3.9</td> <td><a href="https://setuptools.readthedocs.io/en/latest/">setuptools</a></td> <td><code>setup.py</code></td> <td><a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a></td> @@ -267,6 +274,12 @@ table.supported-languages ul { <td>N</td> </tr> <tr> + <td><a href="https://python-poetry.org/">Poetry</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-4">4</a></b></sup></td> + <td><code>poetry.lock</code></td> + <td><a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a></td> + <td>N</td> + </tr> + <tr> <td>Scala</td> <td>N/A</td> <td><a href="https://www.scala-sbt.org/">sbt</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-3">3</a></b></sup></td> @@ -305,6 +318,14 @@ table.supported-languages ul { Support for <a href="https://www.scala-sbt.org/">sbt</a> 1.3 and above was added in GitLab 13.9. </p> </li> + <li> + <a id="notes-regarding-supported-languages-and-package-managers-4"></a> + <p> + Support for <a href="https://python-poetry.org/">Poetry</a> projects with a <code>poetry.lock</code> file was <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/7006">added in GitLab 15.0</a>. + Support for projects without a <code>poetry.lock</code> file is tracked in issue: + <a href="https://gitlab.com/gitlab-org/gitlab/-/issues/32774">Poetry's pyproject.toml support for dependency scanning.</a> + </p> + </li> </ol> <!-- markdownlint-enable MD044 --> @@ -321,13 +342,14 @@ The following package managers use lockfiles that GitLab analyzers are capable o | Package Manager | Supported File Format Versions | Tested Versions | | ------ | ------ | ------ | -| Bundler | N/A | [1.17.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/ruby-bundler/main/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | -| Composer | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/php-composer/main/composer.lock) | -| Conan | 0.4 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/c-conan/main/conan.lock) | -| Go | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/main/go.sum) | -| NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/main/src/web.api/packages.lock.json#L2) | -| npm | v1, v2 | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/main/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4) | -| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/main/yarn.lock#L2) | +| Bundler | N/A | [1.17.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/ruby-bundler/default/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | +| Composer | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/php-composer/default/composer.lock) | +| Conan | 0.4 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/c-conan/default/conan.lock) | +| Go | N/A | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/default/go.sum) | +| NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | +| npm | v1, v2 | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/default/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4) | +| yarn | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/default/yarn.lock#L2) | +| Poetry | v1 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v3/qa/fixtures/python-poetry/default/poetry.lock) | #### Obtaining dependency information by running a package manager to generate a parsable file @@ -338,25 +360,18 @@ To support the following package managers, the GitLab analyzers proceed in two s | Package Manager | Pre-installed Versions | Tested Versions | | ------ | ------ | ------ | -| Bundler | [2.1.4](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/blob/v2.11.3/Dockerfile#L15)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [1.17.3](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/master/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | | sbt | [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L443-447), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L449-453), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L455-459), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L461-465), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L467-471), [1.5.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L473-477), [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L479-483) | | Maven | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L95-97) | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L95-97) | -| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.26.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup> | [5.6.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L319-323), [6.7](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L286-288)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup>, [6.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L331-335), [7.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L300-302)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup> | +| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup>, [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.26.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [5.6.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L319-323), [6.7](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L286-288)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [6.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L331-335), [7.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.28.1/spec/image_spec.rb#L300-302)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup> | | setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L224-247) | | pip | [20.2.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L26) | [20.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L77-91) | -| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L168-191)<sup><b><a href="#exported-dependency-information-notes-4">4</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L143-166) | +| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L168-191)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.22.0/spec/image_spec.rb#L143-166) | <!-- markdownlint-disable MD044 --> <ol> <li> <a id="exported-dependency-information-notes-1"></a> <p> - The pre-installed and tested version of <code>Bundler</code> is only used for the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit">bundler-audit</a> analyzer, and is not used for <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">gemnasium</a>. - </p> - </li> - <li> - <a id="exported-dependency-information-notes-2"></a> - <p> Different versions of Java require different versions of Gradle. The versions of Gradle listed in the above table are pre-installed in the analyzer image. The version of Gradle used by the analyzer depends on whether your project uses a <code>gradlew</code> (Gradle wrapper) file or not: @@ -383,13 +398,13 @@ To support the following package managers, the GitLab analyzers proceed in two s </ul> </li> <li> - <a id="exported-dependency-information-notes-3"></a> + <a id="exported-dependency-information-notes-2"></a> <p> These tests confirm that if a <code>gradlew</code> file does not exist, the version of <code>Gradle</code> pre-installed in the analyzer image is used. </p> </li> <li> - <a id="exported-dependency-information-notes-4"></a> + <a id="exported-dependency-information-notes-3"></a> <p> This test confirms that if a <code>Pipfile.lock</code> file is found, it will be used by <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a> to scan the exact package versions listed in this file. </p> @@ -411,35 +426,28 @@ NOTE: If you've run into problems while scanning multiple files, please contribute a comment to [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337056). -#### Ruby - -The following analyzers are executed, each of which have different behavior when processing multiple files: - -- [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) - - Supports multiple lockfiles. - -- [bundler-audit](https://github.com/rubysec/bundler-audit) - - Does not support multiple lockfiles. When multiple lockfiles exist, `bundler-audit` - analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. +#### Python -WARNING: -The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). +We only execute one installation in the directory where either a requirements file or a lock file has been detected. Dependencies are only analyzed by `gemnasium-python` for the first file that is detected. Files are searched for in the following order: -#### Python +1. `requirements.txt`, `requirements.pip`, or `requires.txt` for projects using Pip. +1. `Pipfile` or `Pipfile.lock` for projects using Pipenv. +1. `poetry.lock` for projects using Poetry. +1. `setup.py` for project using Setuptools. -We only execute one installation in the directory where a requirements file has been detected, such as `requirements.txt` or any -variation of this file (for example, `requirements.pip` or `requires.txt`). +The search begins with the root directory and then continues with subdirectories if no builds are found in the root directory. Consequently a Poetry lock file in the root directory would be detected before a Pipenv file in a subdirectory. #### Java and Scala -We only execute one build in the directory where a build file has been detected, such as `build.sbt` or `build.gradle`. -Please note, we support the following types of Java project structures: +We only execute one build in the directory where a build file has been detected. For large projects that include +multiple Gradle, Maven, or sbt builds, or any combination of these, `gemnasium-maven` only analyzes dependencies for the first build file +that is detected. Build files are searched for in the following order: + +1. `build.gradle` or `build.gradle.kts` for single or [multi-project](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) Gradle builds. +1. `pom.xml` for single or [multi-module](https://maven.apache.org/pom.html#Aggregation) Maven projects. +1. `build.sbt` for single or [multi-project](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) sbt builds. -- [multi-project sbt builds](https://www.scala-sbt.org/1.x/docs/Multi-Project.html) -- [multi-project Gradle builds](https://docs.gradle.org/current/userguide/intro_multi_project_builds.html) -- [multi-module maven projects](https://maven.apache.org/pom.html#Aggregation) +The search begins with the root directory and then continues with subdirectories if no builds are found in the root directory. Consequently an sbt build file in the root directory would be detected before a Gradle build file in a subdirectory. #### JavaScript @@ -454,26 +462,20 @@ The following analyzers are executed, each of which have different behavior when Does not support multiple lockfiles. When multiple lockfiles exist, `Retire.js` analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. -From GitLab 14.8 the `Gemnasium` analyzer scans supported JavaScript projects for vendored libraries +From GitLab 14.8 the `gemnasium` analyzer scans supported JavaScript projects for vendored libraries (that is, those checked into the project but not managed by the package manager). -WARNING: -The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). -We execute both analyzers because they use different sources of vulnerability data. The result is more comprehensive analysis than if only one was executed. - -#### PHP, Go, C, C++, .NET, C# +#### PHP, Go, C, C++, .NET, C#, Ruby, JavaScript The analyzer for these languages supports multiple lockfiles. -### Support for additional languages +#### Support for additional languages Support for additional languages, dependency managers, and dependency files are tracked in the following issues: | Package Managers | Languages | Supported files | Scan tools | Issue | | ------------------- | --------- | --------------- | ---------- | ----- | -| [Poetry](https://python-poetry.org/) | Python | `poetry.lock` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | [GitLab#7006](https://gitlab.com/gitlab-org/gitlab/-/issues/7006) | - -For workarounds, see the [Troubleshooting section](#troubleshooting). +| [Poetry](https://python-poetry.org/) | Python | `pyproject.toml` | [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | [GitLab#32774](https://gitlab.com/gitlab-org/gitlab/-/issues/32774) | ## Contribute your scanner @@ -506,7 +508,7 @@ always take the latest dependency scanning artifact available. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4908) in GitLab 14.1 [with a flag](../../../administration/feature_flags.md) named `sec_dependency_scanning_ui_enable`. Enabled by default. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/282533) in GitLab 14.1. -> - [Feature flag sec_dependency_scanning_ui_enable removed](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. +> - [Feature flag `sec_dependency_scanning_ui_enable` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/326005) in GitLab 14.2. To enable Dependency Scanning in a project, you can create a merge request: @@ -577,9 +579,9 @@ The following variables allow configuration of global dependency scanning settin | ----------------------------|------------ | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | | `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). | -| `DS_DEFAULT_ANALYZERS` | ([**DEPRECATED - use `DS_EXCLUDED_ANALYZERS` instead**](https://gitlab.com/gitlab-org/gitlab/-/issues/287691)) Override the names of the official default images. For more information, see [Dependency Scanning Analyzers](analyzers.md). | +| `DS_DEFAULT_ANALYZERS` | This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in GitLab 14.8 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/351963) in 15.0. Use `DS_EXCLUDED_ANALYZERS` instead. | | `DS_EXCLUDED_PATHS` | Exclude files and directories from the scan based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. | -| `DS_IMAGE_SUFFIX` | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10. | +| `DS_IMAGE_SUFFIX` | Suffix added to the image name. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10.) Automatically set to `"-fips"` when FIPS mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info`. | @@ -589,16 +591,13 @@ The following variables are used for configuring specific analyzers (used for a | CI/CD variable | Analyzer | Default | Description | |--------------------------------------| ------------------ | ---------------------------- |------------ | -| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Use if you're running dependency scanning in an offline, air-gapped environment.| -| `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | -| `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | | `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local Gemnasium database. | | `GEMNASIUM_DB_UPDATE_DISABLED` | `gemnasium` | `"false"` | Disable automatic updates for the `gemnasium-db` advisory database (For usage see: [examples](#hosting-a-copy-of-the-gemnasium_db-advisory-database))| | `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | | `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | | `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. | -| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. | +| `DS_JAVA_VERSION` | `gemnasium-maven` | `17` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | @@ -607,9 +606,6 @@ The following variables are used for configuring specific analyzers (used for a | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | -| `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | -| `RETIREJS_NODE_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/npmrepository.json` | Path or URL to `retire.js` node vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | -| `RETIREJS_ADVISORY_DB_INSECURE` | `retire.js` | `false` | Enable fetching remote JS and Node vulnerability data files (defined by the `RETIREJS_JS_ADVISORY_DB` and `RETIREJS_NODE_ADVISORY_DB` variables) from hosts using an insecure or self-signed SSL (TLS) certificate. | #### Other variables @@ -667,32 +663,10 @@ Read more on [how to use private Maven repositories](../index.md#using-private-m GitLab also offers [FIPS-enabled Red Hat UBI](https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image) versions of the Gemnasium images. You can therefore replace standard images with FIPS-enabled images. -To use FIPS-enabled images, set the `DS_IMAGE_SUFFIX` to `-fips`, -and set `DS_EXCLUDED_ANALYZERS` to `bundler-audit, retire.js` -to exclude the analyzers that don't support FIPS. +Gemnasium scanning jobs automatically use FIPS-enabled image when FIPS mode is enabled in the GitLab instance. +([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) -```yaml -variables: - DS_IMAGE_SUFFIX: "-fips" - DS_EXCLUDED_ANALYZERS: "bundler-audit, retire.js" -``` - -If you want to execute `bundler-audit` or `retire.js` in your project pipeline, you can override the -Gemnasium scanning jobs, and set `DS_IMAGE_SUFFIX` to `-fips` only for those jobs. - -```yaml -gemnasium-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" - -gemnasium-maven-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" - -gemnasium-python-dependency_scanning: - variables: - DS_IMAGE_SUFFIX: "-fips" -``` +To manually switch to FIPS-enabled images, set the variable `DS_IMAGE_SUFFIX` to `"-fips"`. ## Interacting with the vulnerabilities @@ -944,9 +918,6 @@ Here are the requirements for using dependency scanning in an offline environmen This advisory database is constantly being updated, so you must periodically sync your local copy with GitLab. -- _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). -- _Only if scanning npm/yarn projects_: Host an offline copy of the [`retire.js`](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [`js`](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. - Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) @@ -961,11 +932,9 @@ import the following default dependency scanning analyzer images from `registry. your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/gemnasium:2 -registry.gitlab.com/security-products/gemnasium-maven:2 -registry.gitlab.com/security-products/gemnasium-python:2 -registry.gitlab.com/security-products/retire.js:2 -registry.gitlab.com/security-products/bundler-audit:2 +registry.gitlab.com/security-products/gemnasium:3 +registry.gitlab.com/security-products/gemnasium-maven:3 +registry.gitlab.com/security-products/gemnasium-python:3 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -987,8 +956,6 @@ Support for custom certificate authorities was introduced in the following versi | `gemnasium` | [v2.8.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/releases/v2.8.0) | | `gemnasium-maven` | [v2.9.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/releases/v2.9.0) | | `gemnasium-python` | [v2.7.0](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/releases/v2.7.0) | -| `retire.js` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js/-/releases/v2.4.0) | -| `bundler-audit` | [v2.4.0](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/releases/v2.4.0) | ### Set dependency scanning CI/CD job variables to use local dependency scanning analyzers @@ -1121,22 +1088,6 @@ intended to obtain a private package from a private index. This only affects use requires that the package does not already exist in the public index (and thus the attacker can put the package there with an arbitrary version number). -## Limitations - -### Referencing local dependencies using a path in JavaScript projects - -The [Retire.js](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) analyzer -doesn't support dependency references made with [local paths](https://docs.npmjs.com/cli/v6/configuring-npm/package-json/#local-paths) -in the `package.json` of JavaScript projects. The dependency scan outputs the following error for -such references: - -```plaintext -ERROR: Could not find dependencies: <dependency-name>. You may need to run npm install -``` - -As a workaround, add the [`retire.js`](analyzers.md) analyzer to -[`DS_EXCLUDED_ANALYZERS`](#configuring-dependency-scanning). - ## Troubleshooting ### Working around missing support for certain languages or package managers @@ -1156,9 +1107,8 @@ Generally, the approach is the following: 1. Add [`dependencies: [<your-converter-job>]`](../../../ci/yaml/index.md#dependencies) to your `dependency_scanning` job to make use of the converted definitions files. -For example, the unsupported `poetry.lock` file can be -[converted](https://python-poetry.org/docs/cli/#export) -to the supported `requirements.txt` as follows. +For example, Poetry projects that _only_ have a `pyproject.toml` +file can generate the `poetry.lock` file as follows. ```yaml include: @@ -1167,15 +1117,11 @@ include: stages: - test -variables: - PIP_REQUIREMENTS_FILE: "requirements-converted.txt" - gemnasium-python-dependency_scanning: - # Work around https://gitlab.com/gitlab-org/gitlab/-/issues/7006 + # Work around https://gitlab.com/gitlab-org/gitlab/-/issues/32774 before_script: - - pip install poetry # Or via another method: https://python-poetry.org/docs/#installation - - poetry export --output="$PIP_REQUIREMENTS_FILE" - - rm poetry.lock pyproject.toml + - pip install "poetry>=1,<2" # Or via another method: https://python-poetry.org/docs/#installation + - poetry update --lock # Generates the lock file to be analyzed. ``` ### `Error response from daemon: error processing tar file: docker-tar: relocation error` @@ -1197,11 +1143,6 @@ syntax. This directive is limited to 10000 checks and always returns `true` afte number. Because of this, and depending on the number of files in your repository, a dependency scanning job might be triggered even if the scanner doesn't support your project. -### Issues building projects with npm or yarn packages relying on Python 2 - -[Python 2 was removed](https://www.python.org/doc/sunset-python-2/) from the `retire.js` analyzer in GitLab 13.7 (analyzer version 2.10.1). Projects using packages -with a dependency on this version of Python should use `retire.js` version 2.10.0 or lower (for example, `registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2.10.0`). - ### Error: `dependency_scanning is used for configuration only, and its script should not be executed` For information on this, see the [GitLab Secure troubleshooting section](../index.md#error-job-is-used-for-configuration-only-and-its-script-should-not-be-executed). @@ -1260,7 +1201,7 @@ We recommend committing the lock files, which prevents this warning. If you have manually set `DS_MAJOR_VERSION` or `DS_ANALYZER_IMAGE` for specific reasons, and now must update your configuration to again get the latest patched versions of our -analyzers, edit your `gitlab-ci.yml` file and either: +analyzers, edit your `.gitlab-ci.yml` file and either: - Set your `DS_MAJOR_VERSION` to match the latest version as seen in [our current Dependency Scanning template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml#L18). @@ -1290,5 +1231,22 @@ To work around this error, downgrade the analyzer's version of `setuptools` (e.g gemnasium-python-dependency_scanning: before_script: - pip install setuptools==57.5.0 - image: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 +``` + +### Dependency Scanning of projects using psycopg2 fails with `pg_config executable not found` error + +Scanning a Python project that depends on `psycopg2` can fail with this message: + +```plaintext +Error: pg_config executable not found. +``` + +[psycopg2](https://pypi.org/project/psycopg2/) depends on the `libpq-dev` Debian package, +which is not installed in the `gemnasium-python` Docker image. To work around this error, +install the `libpq-dev` package in a `before_script`: + +```yaml +gemnasium-python-dependency_scanning: + before_script: + - apt-get update && apt-get install -y libpq-dev ``` diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 3a6aa8e3485..b5d39f3b32a 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -10,7 +10,7 @@ GitLab can check your application for security vulnerabilities including: - Unauthorized access. - Data leaks. -- Denial of service attacks. +- Denial of Service (DoS) attacks. Statistics and details on vulnerabilities are included in the merge request. Providing actionable information _before_ changes are merged enables you to be proactive. @@ -19,9 +19,6 @@ GitLab also provides high-level statistics of vulnerabilities across projects an - The [Security Dashboard](security_dashboard/index.md) provides a high-level view of vulnerabilities detected in your projects, pipeline, and groups. -- The [Threat Monitoring](threat_monitoring/index.md) page provides runtime security metrics - for application environments. With the information provided, - you can immediately begin risk analysis and remediation. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview of GitLab application security, see [Shifting Security Left](https://www.youtube.com/watch?v=XnYstHObqlA&t). @@ -51,8 +48,8 @@ The following vulnerability scanners and their databases are regularly updated: | Secure scanning tool | Vulnerabilities database updates | |:----------------------------------------------------------------|:---------------------------------| -| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database-update). | -| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | +| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database). | +| [Dependency Scanning](dependency_scanning/index.md) | Relies on the [GitLab Advisory Database](https://gitlab.com/gitlab-org/security-products/gemnasium-db). It is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | | [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | | [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | @@ -145,7 +142,7 @@ Jobs pass if they are able to complete a scan. A _pass_ result does NOT indicate Jobs fail if they are unable to complete a scan. You can view the pipeline logs for more information. -All jobs are permitted to fail by default. This means that if they fail it do not fail the pipeline. +All jobs are permitted to fail by default. This means that if they fail, it does not fail the pipeline. If you want to prevent vulnerabilities from being merged, you should do this by adding [Security Approvals in Merge Requests](#security-approvals-in-merge-requests) which prevents unknown, high or critical findings from being merged without an approval from a specific group of people that you choose. @@ -174,7 +171,7 @@ reports are available to download. To download a report, select A merge request contains a security widget which displays a summary of the NEW results. New results are determined by comparing the current findings against existing findings in the target (default) branch (if there are prior findings). -We recommended you run a scan of the `default` branch before enabling feature branch scans for your developers. Otherwise, there is no base for comparison and all feature branches display the full scan results in the merge request security widget. +We recommend you run a scan of the `default` branch before enabling feature branch scans for your developers. Otherwise, there is no base for comparison and all feature branches display the full scan results in the merge request security widget. The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both NEW and EXISTING findings. @@ -204,56 +201,24 @@ By default, the vulnerability report does not show vulnerabilities of `dismissed ## Security approvals in merge requests -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9928) in GitLab 12.2. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9928) in GitLab 12.2. +> - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357300) the Vulnerability-Check feature in GitLab 15.0. You can enforce an additional approval for merge requests that would introduce one of the following security issues: -- A security vulnerability. For more details, read - [Vulnerability-Check rule](#vulnerability-check-rule). +- A security vulnerability. For more details, read [Scan result policies](policies/scan-result-policies.md). - A software license compliance violation. For more details, read [Enabling license approvals within a project](../compliance/license_compliance/index.md#enabling-license-approvals-within-a-project). -### Vulnerability-Check rule +### Migration of existing Vulnerability-Check rules -WARNING: -This feature is in its end-of-life process. It is [deprecated](../../update/deprecations.md#vulnerability-check) -in GitLab 14.8, and is planned for removal in GitLab 15.0. Users should migrate to the new -[Security Approval Policies](policies/scan-result-policies.md). +If your projects have rules that have a security orchestration project, a new MR with +the existing rule's content is created automatically against the default branch belonging +to the security orchestration project. To maintain the same security approval rules you +had before GitLab 15.0, we recommend merging this new MR. -To prevent a merge request introducing a security vulnerability in a project, enable the -Vulnerability-Check rule. While this rule is enabled, additional merge request approval by -[eligible approvers](../project/merge_requests/approvals/rules.md#eligible-approvers) -is required when the latest security report in a merge request: - -- Contains vulnerabilities with states (for example, `previously detected`, `dismissed`) matching the rule's vulnerability states. Only `newly detected` are considered if the target branch differs from the project default branch. -- Contains vulnerabilities with severity levels (for example, `high`, `critical`, or `unknown`) - matching the rule's severity levels. -- Contains a vulnerability count higher than the rule allows. -- Is not yet generated (until pipeline completion). - -An approval is optional when the security report: - -- Contains only vulnerabilities with states (for example, `newly detected`, `resolved`) **NOT** matching the rule's vulnerability states. -- Contains only vulnerabilities with severity levels (for example, `low`, `medium`) **NOT** matching - the rule's severity levels. -- Contains a vulnerability count equal to or less than what the rule allows. - -Project members with at least the Maintainer role can enable or edit -the Vulnerability-Check rule. - -#### Enable the Vulnerability-Check rule - -To enable or edit the Vulnerability-Check rule: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > General**. -1. Expand **Merge request approvals**. -1. Select **Activate** or **Edit** of the Vulnerability-Check. -1. Complete the fields. **Approvals required** must be at least 1. -1. Select **Add approval rule**. - -The approval rule is enabled for all merge requests. Any code changes reset the approvals required. +If your projects have rules without a security orchestration project, a new security orchestration project is created automatically with the content of the existing rule. No additional action is required. ## Using private Maven repositories @@ -443,7 +408,7 @@ You can interact with the results of the security scanning tools in several loca For more details about which findings or vulnerabilities you can view in each of those locations, select the respective link. Each page details the ways in which you can interact with the findings -and vulnerabilities. As an example, in most cases findings start out as _detected_ status. +and vulnerabilities. As an example, in most cases findings start out as a _detected_ status. You have the option to: @@ -507,7 +472,7 @@ Additional details about the differences between the two solutions are outlined | ------ | ------ | ------ | | **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, Secret Detection, and Container Scanning scans are supported. | | **Usability** | Requires knowledge of CI YAML. | Follows a `rules` and `actions`-based YAML structure. | -| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `gitlab-ci.yml` file. To include the project's `gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. | +| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `.gitlab-ci.yml` file. To include the project's `.gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. | | **Schedulable** | Can be scheduled through a scheduled pipeline on the group. | Can be scheduled natively through the policy configuration itself. | | **Separation of Duties** | Only group owners can create compliance framework labels. Only project owners can apply compliance framework labels to projects. The ability to make or approve changes to the compliance pipeline definition is limited to individuals who are explicitly given access to the project that contains the compliance pipeline. | Only project owners can define a linked security policy project. The ability to make or approve changes to security policies is limited to individuals who are explicitly given access to the security policy project. | | **Ability to apply one standard to multiple projects** | The same compliance framework label can be applied to multiple projects inside a group. | The same security policy project can be used for multiple projects across GitLab with no requirement of being located in the same group. | @@ -657,7 +622,7 @@ involve pinning to the previous template versions, for example: ``` Additionally, we provide a dedicated project containing the versioned legacy templates. -This can be used for offline setups or anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). +This can be used for offline setups or for anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). @@ -694,3 +659,6 @@ These security pages can be populated by running the jobs from the manual step o There is [an issue open to handle this scenario](https://gitlab.com/gitlab-org/gitlab/-/issues/346843). Please upvote the issue to help with prioritization, and [contributions are welcomed](https://about.gitlab.com/community/contribute/). + doc/user/project/merge_requests/approvals/settings.md ++ +0 diff --git a/doc/user/application_security/policies/img/policies_list_v14_3.png b/doc/user/application_security/policies/img/policies_list_v14_3.png Binary files differdeleted file mode 100644 index 7a24860d4a7..00000000000 --- a/doc/user/application_security/policies/img/policies_list_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/policies_list_v15_0.png b/doc/user/application_security/policies/img/policies_list_v15_0.png Binary files differnew file mode 100644 index 00000000000..4089c311fe4 --- /dev/null +++ b/doc/user/application_security/policies/img/policies_list_v15_0.png diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 81d24104340..f790164d9a0 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -19,7 +19,6 @@ GitLab supports the following security policies: - [Scan Execution Policy](scan-execution-policies.md) - [Scan Result Policy](scan-result-policies.md) -- [Container Network Policy](#container-network-policy) (DEPRECATED) ## Security policy project @@ -42,8 +41,9 @@ stored there. Examples and schema information are available for the following po - [Scan execution policy](scan-execution-policies.md#example-security-policies-project) - [Scan result policy](scan-result-policies.md#example-security-scan-result-policies-project) -Policies created in this project are applied through a background job that runs once every 10 -minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect. ### Managing the linked security policy project @@ -81,22 +81,7 @@ status), and create and edit deployed policies: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Security & Compliance > Policies**. - - -Network policies are fetched directly from the selected environment's -deployment platform while other policies are fetched from the project's -security policy project. Changes performed outside of this tab are -reflected upon refresh. - -By default, the policy list contains predefined network policies in a -disabled state. Once enabled, a predefined policy deploys to the -selected environment's deployment platform and you can manage it like -the regular policies. - -Note that if you're using [Auto DevOps](../../../topics/autodevops/index.md) -and change a policy in this section, your `auto-deploy-values.yaml` file doesn't update. Auto DevOps -users must make changes by following the -[Container Network Policy documentation](../../../topics/autodevops/stages.md#network-policy). + ## Policy editor @@ -144,111 +129,6 @@ See [Scan execution policies](scan-execution-policies.md). See [Scan result policies](scan-result-policies.md). -## Container Network Policy - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32365) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Container Network Policy is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The **Container Network Policy** section provides packet flow metrics for -your application's Kubernetes namespace. This section has the following -prerequisites: - -- Your project contains at least one [environment](../../../ci/environments/index.md). -- You've [installed Cilium](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). -- You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration). - -If you're using custom Helm values for Cilium, you must enable Hubble -with flow metrics for each namespace by adding the following lines to -your [Cilium values](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium): - -```yaml -hubble: - enabled: true - metrics: - enabled: - - 'flow:sourceContext=namespace;destinationContext=namespace' -``` - -The **Container Network Policy** section displays the following information -about your packet flow: - -- The total amount of the inbound and outbound packets -- The proportion of packets dropped according to the configured - policies -- The per-second average rate of the forwarded and dropped packets - accumulated over time window for the requested time interval - -If a significant percentage of packets is dropped, you should -investigate it for potential threats by -examining the Cilium logs: - -```shell -kubectl -n gitlab-managed-apps logs -l k8s-app=cilium -c cilium-monitor -``` - -### Change the status - -To change a network policy's status: - -- Select the network policy you want to update. -- Select **Edit policy**. -- Select the **Policy status** toggle to update the selected policy. -- Select **Save changes** to deploy network policy changes. - -Disabled network policies have the `network-policy.gitlab.com/disabled_by: gitlab` selector inside -the `podSelector` block. This narrows the scope of such a policy and as a result it doesn't affect -any pods. The policy itself is still deployed to the corresponding deployment namespace. - -### Container Network Policy editor - -The policy editor only supports the [CiliumNetworkPolicy](https://docs.cilium.io/en/v1.8/policy/) -specification. Regular Kubernetes [NetworkPolicy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#networkpolicy-v1-networking-k8s-io) -resources aren't supported. - -Rule mode supports the following rule types: - -- [Labels](https://docs.cilium.io/en/v1.8/policy/language/#labels-based). -- [Entities](https://docs.cilium.io/en/v1.8/policy/language/#entities-based). -- [IP/CIDR](https://docs.cilium.io/en/v1.8/policy/language/#ip-cidr-based). Only - the `toCIDR` block without `except` is supported. -- [DNS](https://docs.cilium.io/en/v1.8/policy/language/#dns-based). -- [Level 4](https://docs.cilium.io/en/v1.8/policy/language/#layer-4-examples) - can be added to all other rules. - -Once your policy is complete, save it by selecting **Save policy** -at the bottom of the editor. Existing policies can also be -removed from the editor interface by selecting **Delete policy** -at the bottom of the editor. - -### Configure a Network Policy Alert - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3438) and [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/287676) in GitLab 13.9. -> - The feature flag was removed and the Threat Monitoring Alerts Project was [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/287676) in GitLab 14.0. - -You can use policy alerts to track your policy's impact. Alerts are only available if you've -[installed](../../clusters/agent/repository.md) -and [configured](../../clusters/agent/install/index.md#register-the-agent-with-gitlab) -an agent for this project. - -There are two ways to create policy alerts: - -- In the [policy editor UI](#container-network-policy-editor), - by clicking **Add alert**. -- In the policy editor's YAML mode, through the `metadata.annotations` property: - - ```yaml - metadata: - annotations: - app.gitlab.com/alert: 'true' - ``` - -Once added, the UI updates and displays a warning about the dangers of too many alerts. - ## Roadmap See the [Category Direction page](https://about.gitlab.com/direction/protect/security_orchestration/) diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 7e8e60768b9..aa23ad30a73 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -35,8 +35,9 @@ policy project is automatically created. Existing policies can also be removed from the editor interface by selecting **Delete policy** at the bottom of the editor. -All scan execution policy changes are applied through a background job that runs once every 10 -minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect.  @@ -84,9 +85,31 @@ This rule enforces the defined actions and schedules a scan on the provided date | `type` | `string` | `schedule` | The rule's type. | | `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | | `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | -| `clusters` | `object` | | The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | +| `agents` | `object` | | The name of the [GitLab agents](../../clusters/agent/index.md) where [cluster image scanning](../../clusters/agent/vulnerabilities.md) will run. The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | <!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> +| `clusters` (removed) | `object` | | This field was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. Use the `agents` field instead. The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | +<!--- end_remove --> -### `cluster` schema +GitLab supports the following types of CRON syntax for the `cadence` field: + +- A daily cadence of once per hour at a specified hour, for example: `0 18 * * *` +- A weekly cadence of once per week on a specified day and at a specified hour, for example: `0 13 * * 0` + +It is possible that other elements of the CRON syntax will work in the cadence field, however, GitLab does not officially test or support them. + +### `agent` schema + +Use this schema to define `agents` objects in the [`schedule` rule type](#schedule-rule-type). + +| Field | Type | Possible values | Description | +|--------------|---------------------|--------------------------|-------------| +| `namespaces` | `array` of `string` | | The namespace that is scanned. If empty, all namespaces will be scanned. | + +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> + +### `cluster` schema (removed) + +This schema was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/356465) in 15.0. +Use the [`agent` schema](#agent-schema) instead. Use this schema to define `clusters` objects in the [`schedule` rule type](#schedule-rule-type). @@ -97,6 +120,8 @@ Use this schema to define `clusters` objects in the [`schedule` rule type](#sche | `namespaces` | `array` of `string` | | The namespace that is scanned (only the first value is currently supported). | | `kinds` | `array` of `string` | `deployment`/`daemonset` | The resource kind that should be scanned (only the first value is currently supported). | +<!--- end_remove --> + ## `scan` action type This action executes the selected `scan` with additional parameters when conditions for at least one diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index d2cce207bfd..232a5c9f91c 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -9,7 +9,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can use scan result policies to take action based on scan results. For example, one type of scan result policy is a security approval policy that allows approval to be required based on the findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning -job is fully executed. +job is fully executed. The following video gives you an overview of GitLab scan result policies: + +<div class="video-fallback"> + See the video: <a href="https://youtu.be/w5I9gcUgr9U">Overview of GitLab Scan Result Policies</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/w5I9gcUgr9U" frameborder="0" allowfullscreen="true"> </iframe> +</figure> ## Scan result policy editor @@ -25,8 +32,9 @@ If a security policy project doesn't link to your project, GitLab creates such a Existing policies can also be removed from the editor interface by selecting **Delete policy** at the bottom of the editor. -All scan result policy changes are applied through a background job that runs once every 10 minutes. -Allow up to 10 minutes for any policy changes committed to this project to take effect. +Most policy changes take effect as soon as the merge request is merged. Any changes that +do not go through a merge request and are committed directly to the default branch may require up to 10 minutes +before the policy changes take effect. The [policy editor](index.md#policy-editor) supports YAML mode and rule mode. @@ -61,7 +69,7 @@ This rule enforces the defined actions based on the information provided. | Field | Type | Possible values | Description | |------------|------|-----------------|-------------| | `type` | `string` | `scan_finding` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `branches` | `array` of `string` | `[]` or the branch's name | Protected branches for this rule to consider. | | `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. | | `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | | `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical`| The severity levels for this rule to consider. | diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 7529bf90ccf..661f564828a 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -4,20 +4,25 @@ group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SAST Analyzers **(FREE)** +# SAST analyzers **(FREE)** > [Moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) from GitLab Ultimate to GitLab Free in 13.3. -SAST relies on underlying third party tools that are wrapped into what we call -"Analyzers". An analyzer is a -[dedicated project](https://gitlab.com/gitlab-org/security-products/analyzers) -that wraps a particular tool to: +Static Application Security Testing (SAST) uses analyzers +to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/#scanner), a third-party code analysis tool. -- Expose its detection logic. -- Handle its execution. -- Convert its output to the common format. +The analyzers are published as Docker images that SAST uses to launch dedicated containers for each +analysis. -This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/security-products/analyzers/common). +SAST default images are maintained by GitLab, but you can also integrate your own custom image. + +For each scanner, an analyzer: + +- Exposes its detection logic. +- Handles its execution. +- Converts its output to a [standard format](../terminology/#secure-report-format). + +## SAST analyzers SAST supports the following official analyzers: @@ -36,12 +41,6 @@ SAST supports the following official analyzers: - [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) (Sobelow (Elixir Phoenix)) - [`spotbugs`](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) (SpotBugs with the Find Sec Bugs plugin (Ant, Gradle and wrapper, Grails, Maven and wrapper, SBT)) -The analyzers are published as Docker images that SAST uses to launch -dedicated containers for each analysis. - -SAST is pre-configured with a set of **default images** that are maintained by -GitLab, but users can also integrate their own **custom images**. - ## SAST analyzer features For an analyzer to be considered Generally Available, it is expected to minimally @@ -55,34 +54,140 @@ support the following features: - [Emits JSON report format](index.md#reports-json-format) - [SELinux support](index.md#running-sast-in-selinux) -## Official default analyzers +## Post analyzers + +Post analyzers enrich the report output by an analyzer. A post analyzer doesn't modify report +content directly. Instead, it enhances the results with additional properties, including: + +- CWEs. +- Location tracking fields. +- A means of identifying false positives or insignificant findings. **(ULTIMATE)** + +## Data provided by analyzers + +Each analyzer provides data about the vulnerabilities it detects. The following table details the +data available from each analyzer. The values provided by these tools are heterogeneous so they are sometimes +normalized into common values, for example, `severity` and `confidence`. + +| Property / tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Semgrep | Sobelow | +|--------------------------------|------|--------|----------|-----------------|----------|------------|-------|-----------------|-------|------------|-----------------------|---------------------------|---------|---------| +| Affected item (for example, class or package) | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| Confidence | ✗ | ✓ | ✓ | ✗ | ✓ | x | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✓ | +| Description | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | +| End column | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| End line | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| External ID (for example, CVE) | ✗ | ✗ | ⚠ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | +| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Internal doc/explanation | ✓ | ⚠ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | +| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | +| Severity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ⚠ | ✗ | +| Solution | ✓ | ✗ | ✗ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | +| Source code extract | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | +| Start column | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | +| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | +| URLs | ✓ | ✗ | ✓ | ✗ | ⚠ | ✗ | ⚠ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | + +- ✓ => Data is available. +- ⚠ => Data is available, but it's partially reliable, or it has to be extracted from unstructured content. +- ✗ => Data is not available or it would require specific, inefficient or unreliable, logic to obtain it. + +## Transition to Semgrep-based scanning + +SAST includes a [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) that covers [multiple languages](index.md#supported-languages-and-frameworks). +GitLab maintains the analyzer and writes detection rules for it. + +If you use the [GitLab-managed CI/CD template](index.md#configuration), the Semgrep-based analyzer operates alongside other language-specific analyzers. +It runs with GitLab-managed detection rules that mimic the other analyzers' detection rules. +Work to remove language-specific analyzers and replace them with the Semgrep-based analyzer is tracked in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/5245). + +You can choose to disable the other analyzers early and use Semgrep-based scanning for supported languages before the default behavior changes. If you do so: + +- You'll enjoy significantly faster scanning, reduced CI minutes usage, and more customizable scanning rules. +- However, vulnerabilities previously reported by language-specific analyzers will be reported again under certain conditions, including if you've dismissed the vulnerabilities before. The system behavior depends on: + - whether you've excluded the Semgrep-based analyzer from running in the past. + - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/). + +### Vulnerability translation + +When you switch analyzers for a language, vulnerabilities may not match up. + +The Vulnerability Management system automatically moves vulnerabilities from the old analyzer to Semgrep for certain languages: + +- For C, a vulnerability is moved if it has only ever been detected by Flawfinder in pipelines where Semgrep also detected it. Semgrep coverage for C was introduced by default into the CI/CD template in GitLab 14.4 (October 2021). +- For Go, a vulnerability is moved if it has only ever been detected by Gosec in pipelines where Semgrep also detected it. Semgrep coverage for Go was introduced by default into the CI/CD template in GitLab 14.2 (August 2021). +- For JavaScript and TypeScript, a vulnerability is moved if it has only ever been detected by ESLint in pipelines where Semgrep also detected it. Semgrep coverage for these languages was introduced into the CI/CD template in GitLab 13.12 (May 2021). + +However, you'll see old vulnerabilities re-created based on Semgrep results if: + +- A vulnerability was created by Bandit or SpotBugs and you disable those analyzers. We only recommend disabling Bandit and SpotBugs now if the analyzers aren’t working. Work to automatically translate Bandit and SpotBugs vulnerabilities to Semgrep is tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/328062). +- A vulnerability was created by ESLint, Gosec, or Flawfinder in a default-branch pipeline where Semgrep scanning did not run successfully (before Semgrep coverage was introduced for the language, because you disabled Semgrep explicitly, or because the Semgrep scan failed in that pipeline). We do not currently plan to combine these vulnerabilities if they already exist. -Any custom change to the official analyzers can be achieved by using a -[CI/CD variable in your `.gitlab-ci.yml`](index.md#available-cicd-variables). +When a vulnerability is re-created, the original vulnerability is marked as “no longer detected” in the Vulnerability Report. +A new vulnerability is then created based on the Semgrep finding. -### Using a custom Docker mirror +### Activating Semgrep-based scanning early -You can switch to a custom Docker registry that provides the official analyzer -images under a different prefix. For instance, the following instructs -SAST to pull `my-docker-registry/gl-images/sast/bandit` -instead of `registry.gitlab.com/security-products/sast/bandit`. -In `.gitlab-ci.yml` define: +You can choose to use Semgrep-based scanning instead of language-specific analyzers before the default behavior changes. + +We recommend taking this approach if any of these cases applies: + +- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/). +- You're having trouble configuring one of the analyzers whose coverage overlaps with Semgrep-based coverage. For example, you might have trouble setting up the SpotBugs-based analyzer to compile your code. +- You've already seen and dismissed vulnerabilities created by ESLint, Gosec, or Flawfinder scanning, and you've kept the re-created vulnerabilities created by Semgrep. + +You can make a separate choice for each of the language-specific analyzers, or you can disable them all. + +#### Activate Semgrep-based scanning + +To switch to Semgrep-based scanning early, you can: + +1. Create a merge request (MR) to set the [`SAST_EXCLUDED_ANALYZERS` CI/CD variable](#disable-specific-default-analyzers) to `"bandit,gosec,eslint"`. + - If you also want to disable SpotBugs scanning, add `spotbugs` to the list. We only recommend this for Java projects. SpotBugs is the only current analyzer that can scan Groovy, Kotlin, and Scala. + - If you also want to disable Flawfinder scanning, add `flawfinder` to the list. We only recommend this for C projects. Flawfinder is the only current analyzer that can scan C++. +1. Verify that scanning jobs succeed in the MR. You'll notice findings from the removed analyzers in _Fixed_ and findings from Semgrep in _New_. (Some findings may show different names, descriptions, and severities, since GitLab manages and edits the Semgrep rulesets.) +1. Merge the MR and wait for the default-branch pipeline to run. +1. Use the Vulnerability Report to dismiss the findings that are no longer detected by the language-specific analyzers. + +## Customize analyzers + +Use [CI/CD variables](index.md#available-cicd-variables) +in your `.gitlab-ci.yml` file to customize the behavior of your analyzers. + +### Use a custom Docker mirror + +You can use a custom Docker registry, instead of the GitLab registry, to host the analyzers' images. + +Prerequisites: + +- The custom Docker registry must provide images for all the official analyzers. + +NOTE: +This variable affects all Secure analyzers, not just the analyzers for SAST. + +To have GitLab download the analyzers' images from a custom Docker registry, define the prefix with +the `SECURE_ANALYZERS_PREFIX` CI/CD variable. + +For example, the following instructs SAST to pull `my-docker-registry/gitlab-images/bandit` instead +of `registry.gitlab.com/security-products/bandit`: ```yaml include: - template: Security/SAST.gitlab-ci.yml variables: - SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gitlab-images ``` -This configuration requires that your custom registry provides images for all -the official analyzers. +### Disable all default analyzers -### Disabling all default analyzers +You can disable all default SAST analyzers, leaving only [custom analyzers](#custom-analyzers) +enabled. -Setting `SAST_DISABLED` to `true` disables all the official -default analyzers. In `.gitlab-ci.yml` define: +To disable all default analyzers, set the CI/CD variable `SAST_DISABLED` to `true` in your +`.gitlab-ci.yml` file. + +Example: ```yaml include: @@ -92,13 +197,15 @@ variables: SAST_DISABLED: true ``` -That's needed when one totally relies on [custom analyzers](#custom-analyzers). +### Disable specific default analyzers + +Analyzers are run automatically according to the +source code languages detected. However, you can disable select analyzers. -### Disabling specific default analyzers +To disable select analyzers, set the CI/CD variable `SAST_EXCLUDED_ANALYZERS` to a comma-delimited +string listing the analyzers that you want to prevent running. -Set `SAST_EXCLUDED_ANALYZERS` to a comma-delimited string that includes the official -default analyzers that you want to avoid running. In `.gitlab-ci.yml` define the -following to prevent the `eslint` analyzer from running: +For example, to disable the `eslint` analyzer: ```yaml include: @@ -108,27 +215,21 @@ variables: SAST_EXCLUDED_ANALYZERS: "eslint" ``` -## Post Analyzers **(ULTIMATE)** +### Custom analyzers -While analyzers are thin wrappers for executing scanners, post analyzers work to -enrich the data generated within our reports. +You can provide your own analyzers by defining jobs in your CI/CD configuration. For +consistency with the default analyzers, you should add the suffix `-sast` to your custom +SAST jobs. -GitLab SAST post analyzers never modify report contents directly but work by -augmenting results with additional properties (such as CWEs), location tracking fields, -and a means of identifying false positives or insignificant findings. +For more details on integrating a custom security scanner into GitLab, see [Security Scanner Integration](../../../development/integrations/secure.md). -The implementation of post analyzers is determined by feature availability tiers, where -simple data enrichment may occur within our free tier and most advanced processing is split -into separate binaries or pipeline jobs. +#### Example custom analyzer -## Custom Analyzers +This example shows how to add a scanning job that's based on the Docker image +`my-docker-registry/analyzers/csharp`. It runs the script `/analyzer run` and outputs a SAST report +`gl-sast-report.json`. -You can provide your own analyzers by -defining CI jobs in your CI configuration. For consistency, you should suffix your custom -SAST jobs with `-sast`. Here's how to add a scanning job that's based on the -Docker image `my-docker-registry/analyzers/csharp` and generates a SAST report -`gl-sast-report.json` when `/analyzer run` is executed. Define the following in -`.gitlab-ci.yml`: +Define the following in your `.gitlab-ci.yml` file: ```yaml csharp-sast: @@ -140,33 +241,3 @@ csharp-sast: reports: sast: gl-sast-report.json ``` - -The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate custom security scanners into GitLab. - -## Analyzers Data - -| Property / Tool | Apex | Bandit | Brakeman | ESLint security | SpotBugs | Flawfinder | Gosec | Kubesec Scanner | MobSF | NodeJsScan | PHP CS Security Audit | Security code Scan (.NET) | Semgrep | Sobelow | -|--------------------------------|------|--------|----------|-----------------|----------|------------|-------|-----------------|-------|------------|-----------------------|---------------------------|---------|---------| -| Affected item (for example, class or package) | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| Confidence | ✗ | ✓ | ✓ | ✗ | ✓ | x | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✓ | -| Description | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | -| End column | ✓ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| End line | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| External ID (for example, CVE) | ✗ | ✗ | ⚠ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | -| File | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Internal doc/explanation | ✓ | ⚠ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | -| Internal ID | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | -| Severity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ⚠ | ✗ | -| Solution | ✓ | ✗ | ✗ | ✗ | ⚠ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ⚠ | ✗ | -| Source code extract | ✗ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | -| Start column | ✓ | ✗ | ✗ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | -| Start line | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| Title | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | -| URLs | ✓ | ✗ | ✓ | ✗ | ⚠ | ✗ | ⚠ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | - -- ✓ => we have that data -- ⚠ => we have that data but it's partially reliable, or we need to extract it from unstructured content -- ✗ => we don't have that data or it would need to develop specific or inefficient/unreliable logic to obtain it. - -The values provided by these tools are heterogeneous so they are sometimes -normalized into common values (for example, `severity`, `confidence`, and so on). diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 8f006f258b6..38f26b7578d 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -13,12 +13,17 @@ The whitepaper ["A Seismic Shift in Application Security"](https://about.gitlab. explains how 4 of the top 6 attacks were application based. Download it to learn how to protect your organization. -If you're using [GitLab CI/CD](../../../ci/index.md), you can use Static Application Security -Testing (SAST) to check your source code for known vulnerabilities. -If the pipeline is associated with a merge request, the SAST analysis is compared with the results of -the target branch's analysis (if available). The results of that comparison are shown in the merge -request. If the pipeline is running from the default branch, the results of the SAST -analysis are available in the [security dashboards](../security_dashboard/index.md). +If you’re using [GitLab CI/CD](../../../ci/index.md), you can use Static Application Security +Testing (SAST) to check your source code for known vulnerabilities. You can run SAST analyzers in +any GitLab tier. The analyzers output JSON-formatted reports as job artifacts. + +With GitLab Ultimate, SAST results are also processed so you can: + +- See them in merge requests. +- Use them in approval workflows. +- Review them in the security dashboard. + +For more details, see the [Summary of features per tier](#summary-of-features-per-tier).  @@ -543,7 +548,7 @@ Several passthrouh types generate a configuration for the target analyzer: - Two `git` passthrough sections pull the head of branch `refs/remotes/origin/test` from the `myrules` Git repository, and revision - `97f7686` from the `sast-rules` Git repostory. From the `sast-rules` Git + `97f7686` from the `sast-rules` Git repository. From the `sast-rules` Git repository, only data from the `go` subdirectory is considered. - The `sast-rules` entry has a higher precedence because it appears later in the configuration. @@ -887,7 +892,7 @@ Some analyzers can be customized with CI/CD variables. | `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | | `JAVA_OPTS` | SpotBugs | Additional arguments for the `java` executable. | | `JAVA_PATH` | SpotBugs | Path to the `java` executable. | -| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | +| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. [Starting in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/352549), supported versions are `11` and `17` (default). Before GitLab 15.0, supported versions are `8` (default) and `11`. | | `MAVEN_CLI_OPTS` | SpotBugs | Additional arguments for the `mvn` or `mvnw` executable. | | `MAVEN_PATH` | SpotBugs | Path to the `mvn` executable. | | `MAVEN_REPO_PATH` | SpotBugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | @@ -977,19 +982,19 @@ import the following default SAST analyzer images from `registry.gitlab.com` int [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/security-products/sast/bandit:2 -registry.gitlab.com/security-products/sast/brakeman:2 -registry.gitlab.com/security-products/sast/eslint:2 -registry.gitlab.com/security-products/sast/flawfinder:2 -registry.gitlab.com/security-products/sast/gosec:3 -registry.gitlab.com/security-products/sast/kubesec:2 -registry.gitlab.com/security-products/sast/nodejs-scan:2 -registry.gitlab.com/security-products/sast/phpcs-security-audit:2 -registry.gitlab.com/security-products/sast/pmd-apex:2 -registry.gitlab.com/security-products/sast/security-code-scan:2 -registry.gitlab.com/security-products/sast/semgrep:2 -registry.gitlab.com/security-products/sast/sobelow:2 -registry.gitlab.com/security-products/sast/spotbugs:2 +registry.gitlab.com/security-products/bandit:2 +registry.gitlab.com/security-products/brakeman:2 +registry.gitlab.com/security-products/eslint:2 +registry.gitlab.com/security-products/flawfinder:2 +registry.gitlab.com/security-products/gosec:3 +registry.gitlab.com/security-products/kubesec:2 +registry.gitlab.com/security-products/nodejs-scan:2 +registry.gitlab.com/security-products/phpcs-security-audit:2 +registry.gitlab.com/security-products/pmd-apex:2 +registry.gitlab.com/security-products/security-code-scan:2 +registry.gitlab.com/security-products/semgrep:2 +registry.gitlab.com/security-products/sobelow:2 +registry.gitlab.com/security-products/spotbugs:2 ``` The process for importing Docker images into a local offline Docker registry depends on diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 0a18e7d5f45..3937cbd77b6 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -195,13 +195,18 @@ Secret Detection can be customized by defining available CI/CD variables: | CI/CD variable | Default value | Description | |-----------------------------------|---------------|-------------| -| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | -| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | -| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. | | `SECRET_DETECTION_EXCLUDED_PATHS` | "" | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225273) in GitLab 13.3. | | `SECRET_DETECTION_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | | `SECRET_DETECTION_IMAGE_SUFFIX` | Suffix added to the image name. If set to `-fips`, `FIPS-enabled` images are used for scan. See [FIPS-enabled images](#fips-enabled-images) for more details. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355519) in GitLab 14.10. | +In previous GitLab versions, the following variables were also available: + +| CI/CD variable | Default value | Description | +|-----------------------------------|---------------|-------------| +| `SECRET_DETECTION_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | +| `SECRET_DETECTION_COMMIT_TO` | - | The commit a Gitleaks scan ends at. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. Replaced with `SECRET_DETECTION_COMMITS`. | +| `SECRET_DETECTION_COMMITS` | - | The list of commits that Gitleaks should scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243564) in GitLab 13.5. [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/352565) in GitLab 15.0. | + ### Custom rulesets **(ULTIMATE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md index 643da47d876..9771658da4e 100644 --- a/doc/user/application_security/secret_detection/post_processing.md +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -46,7 +46,7 @@ sequenceDiagram Cloud Vendor-->>+RevocationAPI: ACCEPTED ``` -## Integrate your cloud provider service with GitLab Saas +## Integrate your cloud provider service with GitLab SaaS Third party cloud and SaaS providers can [express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 488ec336646..577606885ca 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -17,6 +17,7 @@ To use the Security Dashboards, you must: - Configure jobs to use the [`reports` syntax](../../../ci/yaml/index.md#artifactsreports). - Use [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or later. If you use the shared runners on GitLab.com, you are using the correct version. +- Have the [correct role](../../permissions.md) for the project or group. ## When Security Dashboards are updated diff --git a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png b/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png Binary files differdeleted file mode 100644 index a11a7fafc4a..00000000000 --- a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md deleted file mode 100644 index 9b8dd2825ea..00000000000 --- a/doc/user/application_security/threat_monitoring/index.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -type: reference, howto -stage: Protect -group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments ---- - -# Threat Monitoring **(ULTIMATE)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14707) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Threat Monitoring is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The **Threat Monitoring** page provides alerts and metrics -for the GitLab application runtime security features. You can access -these by navigating to your project's **Security & Compliance > Threat -Monitoring** page. - -GitLab supports statistics for the following security features: - -- [Container Network Policies](../../../topics/autodevops/stages.md#network-policy) - -## Container Network Policy Alert list - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3438) in GitLab 13.9. - -The policy alert list displays your policy's alert activity. You can sort the list by these columns: - -- Date and time -- Events -- Status - -You can filter the list with the **Policy Name** filter and the **Status** filter at the top. Use -the selector menu in the **Status** column to set the status for each alert: - -- Unreviewed -- In review -- Resolved -- Dismissed - -By default, the list doesn't display resolved or dismissed alerts. - - - -Clicking an alert's row opens the alert drawer, which shows more information about the alert. A user -can also create an incident from the alert and update the alert status in the alert drawer. - -Clicking an alert's name takes the user to the [alert details page](../../../operations/incident_management/alerts.md#alert-details-page). diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 18b99e06299..87344e4ff65 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -42,7 +42,7 @@ A vulnerability's status can be one of the following: | Dismissed | A user has seen this vulnerability and dismissed it because it is not accurate or otherwise not to be resolved. | | Resolved | The vulnerability has been fixed or is no longer present. | -Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an existing vulnerability is no longer detected in a project's `default` branch, you should change its status to Resolved. This ensures that if it is accidentally reintroduced in a future merge, it will be visible again as a new record. You can use the [Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are no longer detected, and [change their status](../vulnerability_report#change-status-of-multiple-vulnerabilities). +Dismissed vulnerabilities are ignored if detected in subsequent scans. Resolved vulnerabilities that are reintroduced and detected by subsequent scans have a _new_ vulnerability record created. When an existing vulnerability is no longer detected in a project's `default` branch, you should change its status to Resolved. This ensures that if it is accidentally reintroduced in a future merge, it will be visible again as a new record. You can use the [Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are no longer detected, and [change their status](../vulnerability_report#change-status-of-vulnerabilities). ## Change vulnerability status diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index 89464064ea3..967a6d9fa89 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -56,8 +56,6 @@ the following tables: | GitLab analyzer | Outputs severity levels? | Native severity level type | Native severity level example | |------------------------------------------------------------------------------------------|------------------------------|----------------------------|-------------------------------------| -| [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) | **{check-circle}** Yes | String | `low`, `medium`, `high`, `critical` | -| [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) | **{check-circle}** Yes | String | `low`, `medium`, `high`, `critical` | | [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) | **{check-circle}** Yes | CVSS v2.0 Rating and CVSS v3.1 Qualitative Severity Rating | `(AV:N/AC:L/Au:S/C:P/I:P/A:N)`, `CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H` | ## Container Scanning diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index a9cef15e3e8..e499ddbbd6b 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -11,7 +11,7 @@ The Vulnerability Report provides information about vulnerabilities from scans o The scan results from a pipeline are only ingested after all the jobs in the pipeline complete. Partial results for a pipeline with jobs in progress can be seen in the pipeline security tab. -The report is available for projects, groups, and the Security Center. +The report is available for users with the [correct role](../../permissions.md) on projects, groups, and the Security Center. At all levels, the Vulnerability Report contains: @@ -34,13 +34,18 @@ in that row: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6165) in GitLab 11.1. -The project-level Vulnerability Report also contains: +At the project level, the Vulnerability Report also contains: - A time stamp showing when it was updated, including a link to the latest pipeline. - The number of failures that occurred in the most recent pipeline. Select the failure notification to view the **Failed jobs** tab of the pipeline's page. -To access the report, navigate to **Security & Compliance > Vulnerability Report**. +### View the project-level vulnerability report + +To view the project-level vulnerability report: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Vulnerability report**. ## Vulnerability Report actions @@ -56,17 +61,22 @@ From the Vulnerability Report you can: ## Vulnerability Report filters -You can filter the vulnerabilities table by: +You can filter the Vulnerability Report to narrow focus on only vulnerabilities matching specific +criteria. + +The available filters are: <!-- vale gitlab.SubstitutionWarning = NO --> -| Filter | Available options | -|:---------|:------------------| -| Status | Detected, Confirmed, Dismissed, Resolved. | -| Severity | Critical, High, Medium, Low, Info, Unknown. | -| Tool | For more details, see [Tool filter](#tool-filter). | -| Project | For more details, see [Project filter](#project-filter). | -| Activity | For more details, see [Activity filter](#activity-filter). | +- **Status**: Detected, Confirmed, Dismissed, Resolved. +- **Severity**: Critical, High, Medium, Low, Info, Unknown. +- **Tool**: For more details, see [Tool filter](#tool-filter). +- **Project**: For more details, see [Project filter](#project-filter). +- **Activity**: For more details, see [Activity filter](#activity-filter). + +The filters' criteria are combined to show only vulnerabilities matching all criteria. +An exception to this behavior is the Activity filter. For more details about how it works, see +[Activity filter](#activity-filter). <!-- vale gitlab.SubstitutionWarning = YES --> @@ -75,7 +85,7 @@ You can filter the vulnerabilities table by: To filter the list of vulnerabilities: 1. Select a filter. -1. Select values from the dropdown. +1. Select values from the dropdown list. 1. Repeat the above steps for each desired filter. After each filter is selected: @@ -83,11 +93,9 @@ After each filter is selected: - The list of matching vulnerabilities is updated. - The vulnerability severity totals are updated. -The filters' criteria are combined to show only vulnerabilities matching all criteria. -An exception to this behavior is the Activity filter. For more details about how it works, see -[Activity filter](#activity-filter). +### Tool filter -## Tool filter +> The third-party tool filter was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229661) in GitLab 13.12. The tool filter allows you to focus on vulnerabilities detected by selected tools. @@ -95,7 +103,7 @@ When using the tool filter, you can choose: - **All tools** (default). - Individual GitLab-provided tools. -- Any integrated 3rd-party tool. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229661) in GitLab 13.12. +- Any integrated third-party tool. For details of each of the available tools, see [Security scanning tools](../index.md#security-scanning-tools). @@ -103,11 +111,9 @@ For details of each of the available tools, see [Security scanning tools](../ind The content of the Project filter depends on the current level: -| Level | Content of the Project filter | -|:---------------|:------------------------------| -| Security Center | Only projects you've [added to your personal Security Center](../security_dashboard/index.md#add-projects-to-the-security-center). | -| Group level | All projects in the group. | -| Project level | Not applicable. | +- **Security Center**: Only projects you've [added to your personal Security Center](../security_dashboard/index.md#add-projects-to-the-security-center). +- **Group level**: All projects in the group. +- **Project level**: Not applicable. ### Activity filter @@ -119,13 +125,11 @@ all options can be selected in combination. Selection behavior when using the Activity filter: -| Activity selection | Results displayed | -|:------------------------------------|:------------------| -| All | Vulnerabilities with any Activity status (same as ignoring this filter). Selecting this deselects any other Activity filter options. | -| No activity | Only vulnerabilities without either an associated Issue or that are no longer detected. Selecting this deselects any other Activity filter options. | -| With issues | Only vulnerabilities with one or more associated issues. Does not include vulnerabilities that also are no longer detected. | -| No longer detected | Only vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. Does not include vulnerabilities with one or more associated issues. | -| With issues and No longer detected | Only vulnerabilities that have one or more associated issues and also are no longer detected in the latest pipeline scan of the `default` branch. | +- **All**: Vulnerabilities with any Activity status (same as ignoring this filter). Selecting this deselects any other Activity filter options. +- **No activity**: Only vulnerabilities without either an associated issue or that are no longer detected. Selecting this deselects any other Activity filter options. +- **With issues**: Only vulnerabilities with one or more associated issues. Does not include vulnerabilities that also are no longer detected. +- **No longer detected**: Only vulnerabilities that are no longer detected in the latest pipeline scan of the `default` branch. Does not include vulnerabilities with one or more associated issues. +- **With issues** and **No longer detected**: Only vulnerabilities that have one or more associated issues and also are no longer detected in the latest pipeline scan of the `default` branch. ## View details of a vulnerability @@ -155,23 +159,32 @@ If Jira issue support is enabled, the issue link found in the Activity entry lin > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292636) in GitLab 13.10, all statuses became selectable. +From the Vulnerability Report you can change the status of one or more vulnerabilities. + To change the status of vulnerabilities in the table: -1. Select the checkbox for each vulnerability you want to update the status of. -1. In the dropdown that appears select the desired status, then select **Change status**. +1. Select the checkbox beside each vulnerability you want to update the status of. To select all, + select the checkbox in the table header. +1. In the **Set status** dropdown list, select the desired status. +1. Select **Change status**.  -### Change status of multiple vulnerabilities +## Dismissing a vulnerability -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35816) in GitLab 12.9. +When you evaluate a vulnerability and decide it requires no more action, you can mark it +as **Dismissed**. Dismissed vulnerabilities don't appear in the merge request security widget +when detected in future scans. -You can change the status of multiple vulnerabilities at once: +When a vulnerability is dismissed, a record is made of: -1. In the list of vulnerabilities, select the checkbox for each vulnerability you want to update. - To select all, select the checkbox in the table header. -1. Above the table, select a new status. -1. Click **Change status** to save. +- Who dismissed it. +- Date and time when it was dismissed. +- Optionally, a reason why it was dismissed. + +Vulnerability records cannot be deleted, so a permanent record always remains. + +If a vulnerability is dismissed in error, reverse the dismissal by changing its status. ## Export vulnerability details @@ -190,13 +203,19 @@ Fields included are: - Scanner name - Status - Vulnerability -- Details +- Basic details - Additional information - Severity - [CVE](https://cve.mitre.org/) (Common Vulnerabilities and Exposures) - [CWE](https://cwe.mitre.org/) (Common Weakness Enumeration) - Other identifiers +NOTE: +Full details are available through our +[Job Artifacts API](../../../api/job_artifacts.md#download-a-single-artifact-file-from-specific-tag-or-branch). +Use one of the `gl-*-report.json` report filenames in place of `*artifact_path` +to obtain, for example, the path of files in which vulnerabilities were detected. + ### Export details in CSV format To export details of all vulnerabilities listed in the Vulnerability Report, select **Export**. @@ -224,6 +243,7 @@ To undo this action, select a different status from the same menu. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301003) in GitLab 14.9. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/353796) in GitLab 14.10. +> - [Feature flag `new_vulnerability_form`](https://gitlab.com/gitlab-org/gitlab/-/issues/359049) removed in GitLab 15.0. To add a new vulnerability finding from your project level Vulnerability Report page: |
