diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
| commit | 9f46488805e86b1bc341ea1620b866016c2ce5ed (patch) | |
| tree | f9748c7e287041e37d6da49e0a29c9511dc34768 /doc/user/application_security | |
| parent | dfc92d081ea0332d69c8aca2f0e745cb48ae5e6d (diff) | |
| download | gitlab-ce-9f46488805e86b1bc341ea1620b866016c2ce5ed.tar.gz | |
Add latest changes from gitlab-org/gitlab@13-0-stable-ee
Diffstat (limited to 'doc/user/application_security')
38 files changed, 669 insertions, 451 deletions
diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 131247910ab..dfddbde379d 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -11,7 +11,7 @@ type: reference, howto The security configuration page displays the configuration state of each of the security features and can be accessed through a project's sidebar nav. - + The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines/index.md) to determine the configuration state of each feature. If a job with the expected security report artifact exists in the pipeline, diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png b/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png Binary files differdeleted file mode 100644 index 13cacc6a489..00000000000 --- a/doc/user/application_security/container_scanning/img/container_scanning_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png b/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png Binary files differnew file mode 100644 index 00000000000..7a079a65072 --- /dev/null +++ b/doc/user/application_security/container_scanning/img/container_scanning_v13_0.png diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 68ad2d427dd..6e52d7dbdcf 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -22,7 +22,7 @@ GitLab checks the Container Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. - + ## Contribute your scanner @@ -59,7 +59,7 @@ To enable Container Scanning in your pipeline, you need: [predefined environment variables](../../../ci/variables/predefined_variables.md) as defined below: - ```text + ```plaintext $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA ``` @@ -103,7 +103,7 @@ The included template will: and scan it for possible vulnerabilities. The results will be saved as a -[Container Scanning report artifact](../../../ci/yaml/README.md#artifactsreportscontainer_scanning-ultimate) +[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest Container Scanning artifact available. Behind the scenes, the @@ -169,6 +169,7 @@ using environment variables. | Environment Variable | Description | Default | | ------ | ------ | ------ | +| `SECURE_ANALYZERS_PREFIX` | Set the Docker registry base address from which to download the analyzer. | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | | `KLAR_TRACE` | Set to true to enable more verbose output from klar. | `"false"` | | `CLAIR_TRACE` | Set to true to enable more verbose output from the clair server process. | `"false"` | | `DOCKER_USER` | Username for accessing a Docker registry requiring authentication. | `$CI_REGISTRY_USER` | @@ -179,11 +180,11 @@ using environment variables. | `CLAIR_VULNERABILITIES_DB_URL` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | `clair-vulnerabilities-db` | | `CLAIR_DB_CONNECTION_STRING` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | | `CI_APPLICATION_REPOSITORY` | Docker repository URL for the image to be scanned. | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | -| `CI_APPLICATION_TAG` | Docker respository tag for the image to be scanned. | `$CI_COMMIT_SHA` | +| `CI_APPLICATION_TAG` | Docker repository tag for the image to be scanned. | `$CI_COMMIT_SHA` | | `CLAIR_DB_IMAGE` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | `arminc/clair-db:latest` | | `CLAIR_DB_IMAGE_TAG` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | `latest` | | `DOCKERFILE_PATH` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner will look for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | `Dockerfile` | -| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" | +| `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs that you want to trust. | "" | ### Overriding the Container Scanning template @@ -229,25 +230,29 @@ To use Container Scanning in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner may try to pull remote images even if a local copy is available. Set GitLab -Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) -in an offline environment if you prefer using only locally available Docker images. +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. #### Make GitLab Container Scanning analyzer images available inside your Docker registry -For Container Scanning, import and host the following images from `registry.gitlab.com` to your -offline [local Docker container registry](../../packages/container_registry/index.md): +For Container Scanning, import the following default images from `registry.gitlab.com` into your +[local Docker container registry](../../packages/container_registry/index.md): -- [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db) -- GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar` +```plaintext +registry.gitlab.com/gitlab-org/security-products/analyzers/klar +https://hub.docker.com/r/arminc/clair-db +``` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. - -Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +process by which you can import or temporarily access external resources. Note that these scanners +are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) with new definitions, so consider if you are able to make periodic updates yourself. -You can read more specific steps on how to do this [below](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). + +For more information, see [the specific steps on how to update an image with a pipeline](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -255,8 +260,6 @@ For details on saving and transporting Docker images as a file, see Docker's doc #### Set Container Scanning CI job variables to use local Container Scanner analyzers -Container Scanning can be executed on an offline GitLab Ultimate installation using the following process: - 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: ```yaml @@ -412,11 +415,11 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | | `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) only provides the following levels: `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. **Note:** Our current container scanning tool based on [klar](https://github.com/optiopay/klar) does not provide a confidence level, so this value is currently hardcoded to `Unknown`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | @@ -435,7 +438,7 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | | `remediations` | An array of objects containing information on cured vulnerabilities along with patch diffs to apply. Empty if no remediations provided by an underlying analyzer. | | `remediations[].fixes` | An array of strings that represent references to vulnerabilities fixed by this particular remediation. | -| `remediations[].fixes[].id` | The id of a fixed vulnerability. | +| `remediations[].fixes[].id` | The ID of a fixed vulnerability. | | `remediations[].fixes[].cve` | (**DEPRECATED - use `remediations[].fixes[].id` instead**) A string value that describes a fixed vulnerability in the same format as `vulnerabilities[].cve`. | | `remediations[].summary` | Overview of how the vulnerabilities have been fixed. | | `remediations[].diff` | base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | @@ -476,7 +479,7 @@ When the GitLab Runner uses the Docker executor and NFS is used (for example, `/var/lib/docker` is on an NFS mount), Container Scanning might fail with an error like the following: -```text +```plaintext docker: Error response from daemon: failed to copy xattrs: failed to set xattr "security.selinux" on /path/to/file: operation not supported. ``` diff --git a/doc/user/application_security/dast/img/dast_all_v12_9.png b/doc/user/application_security/dast/img/dast_all_v12_9.png Binary files differdeleted file mode 100644 index 548cea3f7f9..00000000000 --- a/doc/user/application_security/dast/img/dast_all_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_all_v13_0.png b/doc/user/application_security/dast/img/dast_all_v13_0.png Binary files differnew file mode 100644 index 00000000000..7b67fc44fae --- /dev/null +++ b/doc/user/application_security/dast/img/dast_all_v13_0.png diff --git a/doc/user/application_security/dast/img/dast_single_v12_9.png b/doc/user/application_security/dast/img/dast_single_v12_9.png Binary files differdeleted file mode 100644 index a8a4b1c1d4f..00000000000 --- a/doc/user/application_security/dast/img/dast_single_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/dast/img/dast_single_v13_0.png b/doc/user/application_security/dast/img/dast_single_v13_0.png Binary files differnew file mode 100644 index 00000000000..91ed4d916ab --- /dev/null +++ b/doc/user/application_security/dast/img/dast_single_v13_0.png diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index abf194aae48..1d5f96d96bb 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -19,42 +19,48 @@ Dynamic Application Security Testing (DAST) comes into place. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web application(s) +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your running web applications for known vulnerabilities using Dynamic Application Security Testing (DAST). - You can take advantage of DAST by either [including the CI job](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). +[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate), +provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the DAST report, compares the found vulnerabilities between the source and target -branches, and shows the information right on the merge request. +branches, and shows the information on the merge request. NOTE: **Note:** This comparison logic uses only the latest pipeline executed for the target branch's base commit. Running the pipeline on any other commit has no effect on the merge request. - + -By clicking on one of the detected linked vulnerabilities, you will be able to +By clicking on one of the detected linked vulnerabilities, you can see the details and the URL(s) affected. - + [Dynamic Application Security Testing (DAST)](https://en.wikipedia.org/wiki/Dynamic_Application_Security_Testing) -is using the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) +uses the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) to perform an analysis on your running web application. -By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) and will perform passive scanning only. It will not actively attack your application. - +By default, DAST executes [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan) +and performs passive scanning only. It won't actively attack your application. However, DAST can be [configured](#full-scan) -to also perform a so-called "active scan". That is, attack your application and produce a more extensive security report. +to also perform an *active scan*: attack your application and produce a more extensive security report. It can be very useful combined with [Review Apps](../../../ci/review_apps/index.md). +NOTE: **Note:** +A pipeline may consist of multiple jobs, including SAST and DAST scanning. If any +job fails to finish for any reason, the security dashboard won't show DAST scanner +output. For example, if the DAST job finishes but the SAST job fails, the security +dashboard won't show DAST results. The analyzer will output an +[exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Use cases It helps you automatically find security vulnerabilities in your running web -applications while you are developing and testing your applications. +applications while you're developing and testing your applications. ## Requirements @@ -66,9 +72,8 @@ To run a DAST job, you need GitLab Runner with the For GitLab 11.9 and later, to enable DAST, you must [include](../../../ci/yaml/README.md#includetemplate) the [`DAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) -that's provided as a part of your GitLab installation. -For GitLab versions earlier than 11.9, you can copy and use the job as defined -in that template. +that's provided as a part of your GitLab installation. For GitLab versions earlier +than 11.9, you can copy and use the job as defined in that template. Add the following to your `.gitlab-ci.yml` file: @@ -85,32 +90,39 @@ There are two ways to define the URL to be scanned by DAST: 1. Set the `DAST_WEBSITE` [variable](../../../ci/yaml/README.md#variables). 1. Add it in an `environment_url.txt` file at the root of your project. - This is great for testing in dynamic environments. In order to run DAST against - an app that is dynamically created during a GitLab CI/CD pipeline, have the app - persist its domain in an `environment_url.txt` file, and DAST will - automatically parse that file to find its scan target. - You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) - of this in our Auto DevOps CI YML. + This is great for testing in dynamic environments. In order to run DAST against + an app dynamically created during a GitLab CI/CD pipeline, have the app + persist its domain in an `environment_url.txt` file, and DAST + automatically parses that file to find its scan target. + You can see an [example](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + of this in our Auto DevOps CI YAML. -If both values are set, the `DAST_WEBSITE` value will take precedence. +If both values are set, the `DAST_WEBSITE` value takes precedence. -The included template will create a `dast` job in your CI/CD pipeline and scan +The included template creates a `dast` job in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. -The results will be saved as a -[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate) +The results are saved as a +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) that you can later download and analyze. Due to implementation limitations we always take the latest DAST artifact available. Behind the scenes, the [GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast) is used to run the tests on the specified URL and scan it for possible vulnerabilities. -By default, the DAST template will use the latest major version of the DAST Docker image. Using the `DAST_VERSION` variable, -you can choose to automatically update DAST with new features and fixes by pinning to a major version (e.g. 1), only update fixes by pinning to a minor version (e.g. 1.6) or prevent all updates by pinning to a specific version (e.g. 1.6.4). +By default, the DAST template will use the latest major version of the DAST Docker +image. Using the `DAST_VERSION` variable, you can choose how DAST updates: + +- Automatically update DAST with new features and fixes by pinning to a major version (such as `1`). +- Only update fixes by pinning to a minor version (such as `1.6`). +- Prevent all updates by pinning to a specific version (such as `1.6.4`). + Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases) page. ### When DAST scans run -When using `DAST.gitlab-ci.yml` template, the `dast` job is run last as shown in the example below. To ensure DAST is scanning the latest code, your CI pipeline should deploy changes to the web server in one of the jobs preceeding the `dast` job. +When using `DAST.gitlab-ci.yml` template, the `dast` job is run last as shown in +the example below. To ensure DAST is scanning the latest code, your CI pipeline +should deploy changes to the web server in one of the jobs preceding the `dast` job. ```yaml stages: @@ -120,12 +132,15 @@ stages: - dast ``` -Be aware that if your pipeline is configured to deploy to the same webserver in each run, running a pipeline while another is still running, could cause a race condition -where one pipeline overwrites the code from another pipeline. The site to be scanned should be excluded from changes for the duration of a DAST scan. -The only changes to the site should be from the DAST scanner. Be aware that any changes that users, scheduled tasks, database or code changes, other pipelines, or other scanners make to +Be aware that if your pipeline is configured to deploy to the same webserver in +each run, running a pipeline while another is still running could cause a race condition +where one pipeline overwrites the code from another pipeline. The site to be scanned +should be excluded from changes for the duration of a DAST scan. +The only changes to the site should be from the DAST scanner. Be aware that any +changes that users, scheduled tasks, database changes, code changes, other pipelines, or other scanners make to the site during a scan could lead to inaccurate results. -### Authenticated scan +### Authentication It's also possible to authenticate the user before performing the DAST checks: @@ -144,12 +159,15 @@ variables: ``` The results will be saved as a -[DAST report artifact](../../../ci/yaml/README.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest DAST artifact available. DANGER: **Danger:** -**DO NOT** run an authenticated scan against a production server. When an authenticated scan is run, it may perform *any* function that the authenticated user can. This includes modifying and deleting data, submitting forms, following links, and so on. Only run an authenticated scan against a test server. +**NEVER** run an authenticated scan against a production server. When an authenticated +scan is run, it may perform *any* function that the authenticated user can. This +includes actions like modifying and deleting data, submitting forms, and following links. +Only run an authenticated scan against a test server. ### Full scan @@ -170,7 +188,8 @@ The DAST job can be run anywhere, which means you can accidentally hit live web and potentially damage them. You could even take down your production environment. For that reason, you should use domain validation. -Domain validation is not required by default. It can be required by setting the [environment variable](#available-variables) `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` to true. +Domain validation is not required by default. It can be required by setting the +[environment variable](#available-variables) `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` to `"true"`. ```yaml include: @@ -181,19 +200,23 @@ variables: DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED: "true" ``` -Since ZAP full scan actively attacks the target application, DAST sends a ping to the target (normally defined in `DAST_WEBSITE` or `environment_url.txt`) beforehand. +Since ZAP full scan actively attacks the target application, DAST sends a ping +to the target (normally defined in `DAST_WEBSITE` or `environment_url.txt`) beforehand. -If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is false or unset, the scan will _proceed_ unless the response to the ping -includes a `Gitlab-DAST-Permission` header with a value of `deny`. +- If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is `false` or unset, the scan will + proceed unless the response to the ping includes a `Gitlab-DAST-Permission` + header with a value of `deny`. +- If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is `true`, the scan will exit + unless the response to the ping includes a `Gitlab-DAST-Permission` header with + a value of `allow`. -If `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` is true, the scan will _exit_ unless the response to the ping -includes a `Gitlab-DAST-Permission` header with a value of `allow`. - -Here are some examples of adding the `Gitlab-DAST-Permission` header to a response in Rails, Django, and Node (with Express). +Here are some examples of adding the `Gitlab-DAST-Permission` header to a response +in Rails, Django, and Node (with Express). ##### Ruby on Rails -Here's how you would add a [custom header in Ruby on Rails](https://guides.rubyonrails.org/action_controller_overview.html#setting-custom-headers): +Here's how you would add a +[custom header in Ruby on Rails](https://guides.rubyonrails.org/action_controller_overview.html#setting-custom-headers): ```ruby class DastWebsiteTargetController < ActionController::Base @@ -207,7 +230,8 @@ end ##### Django -Here's how you would add a [custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): +Here's how you would add a +[custom header in Django](https://docs.djangoproject.com/en/2.2/ref/request-response/#setting-header-fields): ```python class DastWebsiteTargetView(View): @@ -220,7 +244,8 @@ class DastWebsiteTargetView(View): ##### Node (with Express) -Here's how you would add a [custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append): +Here's how you would add a +[custom header in Node (with Express)](http://expressjs.com/en/5x/api.html#res.append): ```javascript app.get('/dast-website-target', function(req, res) { @@ -235,7 +260,8 @@ It's also possible to add the `Gitlab-DAST-Permission` header via a proxy. ###### NGINX -The following config allows NGINX to act as a reverse proxy and add the `Gitlab-DAST-Permission` [header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header): +The following configuration allows NGINX to act as a reverse proxy and add the +`Gitlab-DAST-Permission` [header](http://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header): ```nginx # default.conf @@ -287,9 +313,9 @@ API scans support OpenAPI V2 and OpenAPI V3 specifications. You can define these #### Import API specification from a URL If your API specification is accessible at a URL, you can pass that URL in directly as the target. -The specification doesn't have to be hosted on the same host as the API being tested. +The specification does not have to be hosted on the same host as the API being tested. -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -299,9 +325,11 @@ variables: #### Import API specification from a file -If your API specification is in your repository, you can provide the specification's filename directly as the target. The specification file is expected to be in the `/zap/wrk` directory. +If your API specification is in your repository, you can provide the specification's +filename directly as the target. The specification file is expected to be in the +`/zap/wrk` directory. -```yml +```yaml dast: script: - mkdir -p /zap/wrk @@ -314,23 +342,27 @@ dast: #### Full scan -API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` environment variable. Domain validation isn't supported for full API scans. +API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` +environment variable. Domain validation is not supported for full API scans. #### Host override -Specifications often define a host, which contains a domain name and a port. The host referenced may be different than the host of the API's review instance. -This can cause incorrect URLs to be imported, or a scan on an incorrect host. Use the `DAST_API_HOST_OVERRIDE` environment variable to override these values. +Specifications often define a host, which contains a domain name and a port. The +host referenced may be different than the host of the API's review instance. +This can cause incorrect URLs to be imported, or a scan on an incorrect host. +Use the `DAST_API_HOST_OVERRIDE` environment variable to override these values. For example, with a OpenAPI V3 specification containing: -```yml +```yaml servers: - url: https://api.host.com ``` -If the test version of the API is running at `https://api-test.host.com`, then the following DAST configuration can be used: +If the test version of the API is running at `https://api-test.host.com`, then +the following DAST configuration can be used: -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -343,9 +375,11 @@ Note that `DAST_API_HOST_OVERRIDE` is only applied to specifications imported by #### Authentication using headers -Tokens in request headers are often used as a way to authenticate API requests. You can achieve this by using the `DAST_REQUEST_HEADERS` environment variable. Headers are applied to every request DAST makes. +Tokens in request headers are often used as a way to authenticate API requests. +You can achieve this by using the `DAST_REQUEST_HEADERS` environment variable. +Headers are applied to every request DAST makes. -```yml +```yaml include: - template: DAST.gitlab-ci.yml @@ -404,7 +438,8 @@ don't forget to add `stage: dast` when you override the template job definition. DAST can be [configured](#customizing-the-dast-settings) using environment variables. | Environment variable | Required | Description | -|-----------------------------| ----------|--------------------------------------------------------------------------------| +|-----------------------------| -----------|--------------------------------------------------------------------------------| +| `SECURE_ANALYZERS_PREFIX` | no | Set the Docker registry base address from which to download the analyzer. | | `DAST_WEBSITE` | no| The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. | | `DAST_API_SPECIFICATION` | no | The API specification to import. `DAST_WEBSITE` must be specified if this is omitted. | | `DAST_AUTH_URL` | no | The authentication URL of the website to scan. Not supported for API scans. | @@ -416,14 +451,16 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_TARGET_AVAILABILITY_TIMEOUT` | no | Time limit in seconds to wait for target availability. Scan is attempted nevertheless if it runs out. Integer. Defaults to `60`. | | `DAST_FULL_SCAN_ENABLED` | no | Switches the tool to execute [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | | `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | no | Requires [domain validation](#domain-validation) when running DAST full scans. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. Not supported for API scans. | -| `DAST_AUTO_UPDATE_ADDONS` | no | Set to `false` to pin the versions of ZAProxy add-ons to those provided with the DAST image. Defaults to `true`. | +| `DAST_AUTO_UPDATE_ADDONS` | no | By default the versions of ZAP add-ons are pinned to those provided with the DAST image. Set to `true` to allow ZAP to download the latest versions. | | `DAST_API_HOST_OVERRIDE` | no | Used to override domains defined in API specification files. | | `DAST_EXCLUDE_RULES` | no | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from scans. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/master/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. | | `DAST_REQUEST_HEADERS` | no | Set to a comma-separated list of request header names and values. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | +| `DAST_ZAP_USE_AJAX_SPIDER` | no | Use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Boolean. `true`, `True`, or `1` are considered as true value, otherwise false. Defaults to `false`. | ### DAST command-line options -Not all DAST configuration is available via environment variables. To find out all possible options, run the following configuration. +Not all DAST configuration is available via environment variables. To find out all +possible options, run the following configuration. Available command-line options will be printed to the job log: ```yaml @@ -435,7 +472,8 @@ dast: - /analyze --help ``` -You must then overwrite the `script` command to pass in the appropriate argument. For example, AJAX spidering can be enabled by using `-j`, as shown in the following configuration: +You must then overwrite the `script` command to pass in the appropriate argument. +For example, debug messages can be enabled by using `-d`, as shown in the following configuration: ```yaml include: @@ -444,14 +482,16 @@ include: dast: script: - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -j -t $DAST_WEBSITE + - /analyze -d -t $DAST_WEBSITE ``` ### Custom ZAProxy configuration The ZAProxy server contains many [useful configurable values](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_245801885). -Many key/values for `-config` remain undocumented, but there is an untested list of [possible keys](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_244981023). -Note that these options are not supported by DAST, and may break the DAST scan when used. An example of how to rewrite the Authorization header value with `TOKEN` follows: +Many key/values for `-config` remain undocumented, but there is an untested list of +[possible keys](https://gitlab.com/gitlab-org/gitlab/issues/36437#note_244981023). +Note that these options are not supported by DAST, and may break the DAST scan +when used. An example of how to rewrite the Authorization header value with `TOKEN` follows: ```yaml include: @@ -479,50 +519,52 @@ successfully run. For more information, see [Offline environments](../offline_de To use DAST in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- Docker Container Registry with a locally available copy of the DAST [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). +- Docker Container Registry with a locally available copy of the DAST + [container image](https://gitlab.com/gitlab-org/security-products/dast), found in the + [DAST container registry](https://gitlab.com/gitlab-org/security-products/dast/container_registry). NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner may try to pull remote images even if a local copy is available. Set GitLab -Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) -in an offline environment if you prefer using only locally available Docker images. +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +in an offline environment if you prefer using only locally available Docker images. However, we +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab DAST analyzer images available inside your Docker registry -For DAST, import the following default DAST analyzer image from `registry.gitlab.com` to your local "offline" -registry: +For DAST, import the following default DAST analyzer image from `registry.gitlab.com` to your [local Docker container registry](../../packages/container_registry/index.md): - `registry.gitlab.com/gitlab-org/security-products/dast:latest` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. Note +that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you're able to make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on -[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), -[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). +[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), +[`docker load`](https://docs.docker.com/engine/reference/commandline/load/), +[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and +[`docker import`](https://docs.docker.com/engine/reference/commandline/import/). ### Set DAST CI job variables to use local DAST analyzers -1. Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer - to the DAST Docker image hosted on your local Docker container registry: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace `image` to refer to +the DAST Docker image hosted on your local Docker container registry: - ```yaml - include: - - template: DAST.gitlab-ci.yml - - dast: - image: registry.example.com/namespace/dast:latest - script: - - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -t $DAST_WEBSITE --auto-update-addons false -z"-silent" - ``` +```yaml +include: + - template: DAST.gitlab-ci.yml +dast: + image: registry.example.com/namespace/dast:latest +``` -The option `--auto-update-addons false` instructs ZAP not to update add-ons. +The DAST job should now use local copies of the DAST analyzers to scan your code and generate +security reports without requiring internet access. -The option `-z` passes the quoted `-silent` parameter to ZAP. The `-silent` parameter ensures ZAP -does not make any unsolicited requests including checking for updates. +Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. ## Reports @@ -530,7 +572,8 @@ The DAST job can emit various reports. ### List of URLs scanned -When DAST completes scanning, the merge request page states the number of URLs that were scanned. Click **View details** to view the web console output which includes the list of scanned URLs. +When DAST completes scanning, the merge request page states the number of URLs scanned. +Click **View details** to view the web console output which includes the list of scanned URLs.  @@ -539,9 +582,14 @@ When DAST completes scanning, the merge request page states the number of URLs t CAUTION: **Caution:** The JSON report artifacts are not a public API of DAST and their format is expected to change in the future. -The DAST tool always emits a JSON report file called `gl-dast-report.json` and sample reports can be found in the [DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect). +The DAST tool always emits a JSON report file called `gl-dast-report.json` and +sample reports can be found in the +[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect). -There are two formats of data in the JSON report that are used side by side: the proprietary ZAP format which will be eventually deprecated, and a "common" format which will be the default in the future. +There are two formats of data in the JSON report that are used side by side: + +- The proprietary ZAP format that will be eventually deprecated. +- A common format that will be the default in the future. ### Other formats @@ -574,7 +622,9 @@ vulnerabilities in your groups, projects and pipelines. Read more about the ## Bleeding-edge vulnerability definitions -ZAProxy first creates rules in the `alpha` class. After a testing period with the community, they are promoted to `beta`. DAST uses `beta` definitions by default. To request `alpha` definitions, use `-a` as shown in the following configuration: +ZAProxy first creates rules in the `alpha` class. After a testing period with the +community, they are promoted to `beta`. DAST uses `beta` definitions by default. +To request `alpha` definitions, use `-a` as shown in the following configuration: ```yaml include: @@ -608,6 +658,18 @@ Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> +## Optimizing DAST + +By default, DAST will download all artifacts defined by previous jobs in the pipeline. If +your DAST job does not rely on `environment_url.txt` to define the URL under test or any other files created +in previous jobs, we recommend you don't download artifacts. To avoid downloading +artifacts, add the following to your `gitlab-ci.yml` file: + +```json +dast: + dependencies: [] +``` + ## Troubleshooting ### Running out of memory @@ -615,14 +677,14 @@ but commented out to help encourage others to add to it in the future. --> By default, ZAProxy, which DAST relies on, is allocated memory that sums to 25% of the total memory on the host. Since it keeps most of its information in memory during a scan, -it is possible for DAST to run out of memory while scanning large applications. +it's possible for DAST to run out of memory while scanning large applications. This results in the following error: ```plaintext [zap.out] java.lang.OutOfMemoryError: Java heap space ``` -Fortunately, it is straightforward to increase the amount of memory available +Fortunately, it's straightforward to increase the amount of memory available for DAST by overwriting the `script` key in the DAST template: ```yaml diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 26352f21cfb..474f9339d0b 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -43,7 +43,7 @@ include: template: Dependency-Scanning.gitlab-ci.yml variables: - DS_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images ``` This configuration requires that your custom registry provides images for all diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning.png Binary files differdeleted file mode 100644 index 18df356f846..00000000000 --- a/doc/user/application_security/dependency_scanning/img/dependency_scanning.png +++ /dev/null diff --git a/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png Binary files differnew file mode 100644 index 00000000000..9f3990df957 --- /dev/null +++ b/doc/user/application_security/dependency_scanning/img/dependency_scanning_v13_0.png diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index cda621e61a6..53462cf232e 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -7,25 +7,24 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5105) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.7. Dependency Scanning helps to automatically find security vulnerabilities in your dependencies -while you are developing and testing your applications, for example when your +while you're developing and testing your applications, such as when your application is using an external (open source) library which is known to be vulnerable. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your dependencies for known +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your dependencies for known vulnerabilities using Dependency Scanning. All dependencies are scanned, including the transitive dependencies (also known as nested dependencies). - You can take advantage of Dependency Scanning by either [including the Dependency Scanning template](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) -that is provided by [Auto DevOps](../../../topics/autodevops/index.md). +the [Auto Dependency Scanning](../../../topics/autodevops/stages.md#auto-dependency-scanning-ultimate) +provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the Dependency Scanning report, compares the found vulnerabilities between the source and target branches, and shows the information on the merge request. - + The results are sorted by the severity of the vulnerability: @@ -38,17 +37,16 @@ The results are sorted by the severity of the vulnerability: ## Requirements -To run a Dependency Scanning job, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +To run Dependency Scanning jobs, by default, you need GitLab Runner with the +[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +If you're using the shared Runners on GitLab.com, this is enabled by default. CAUTION: **Caution:** -If you use your own Runners, make sure that the Docker version you have installed +If you use your own Runners, make sure your installed version of Docker is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. -Privileged mode is not necessary if you've [disabled Docker in Docker for Dependency Scanning](#disabling-docker-in-docker-for-dependency-scanning) +Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for Dependency Scanning](#enabling-docker-in-docker). ## Supported languages and package managers @@ -56,16 +54,16 @@ The following languages and dependency managers are supported. | Language (package managers) | Supported | Scan tool(s) | |----------------------------- | --------- | ------------ | -| Java ([Gradle](https://gradle.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | -| PHP ([Composer](https://getcomposer.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Python ([pip](https://pip.pypa.io/en/stable/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Java ([Gradle](https://gradle.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Java ([Maven](https://maven.apache.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://classic.yarnpkg.com/en/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js/) | +| PHP ([Composer](https://getcomposer.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Python ([pip](https://pip.pypa.io/en/stable/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | | Python ([Pipfile](https://pipenv.kennethreitz.org/en/latest/basics/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | | Python ([poetry](https://python-poetry.org/)) | not currently ([issue](https://gitlab.com/gitlab-org/gitlab/issues/7006 "Support Poetry in Dependency Scanning")) | not available | -| Ruby ([gem](https://rubygems.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | -| Scala ([sbt](https://www.scala-sbt.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | -| Go ([Go Modules](https://github.com/golang/go/wiki/Modules)) | yes ([alpha](https://gitlab.com/gitlab-org/gitlab/issues/7132)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Ruby ([gem](https://rubygems.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | +| Scala ([sbt](https://www.scala-sbt.org/)) | yes | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Go ([Go Modules](https://github.com/golang/go/wiki/Modules)) | yes ([alpha](https://gitlab.com/gitlab-org/gitlab/issues/7132)) | [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | ## Contribute your scanner @@ -73,7 +71,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) ## Configuration -For GitLab 11.9 and later, to enable Dependency Scanning, you must +To enable Dependency Scanning for GitLab 11.9 and later, you must [include](../../../ci/yaml/README.md#includetemplate) the [`Dependency-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) that's provided as a part of your GitLab installation. @@ -87,11 +85,10 @@ include: - template: Dependency-Scanning.gitlab-ci.yml ``` -The included template will create a `dependency_scanning` job in your CI/CD +The included template will create Dependency Scanning jobs in your CI/CD pipeline and scan your project's source code for possible vulnerabilities. - The results will be saved as a -[Dependency Scanning report artifact](../../../ci/yaml/README.md#artifactsreportsdependency_scanning-ultimate) +[Dependency Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdependency_scanning-ultimate) that you can later download and analyze. Due to implementation limitations, we always take the latest Dependency Scanning artifact available. @@ -99,7 +96,6 @@ always take the latest Dependency Scanning artifact available. The Dependency Scanning settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. - For example: ```yaml @@ -113,23 +109,24 @@ variables: Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline configuration, the last mention of the variable will take precedence. -### Overriding the Dependency Scanning template +### Overriding Dependency Scanning jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `dependency_scanning` job -after the template inclusion and specify any additional keys under it. For example: +To override a job definition (for example, to change properties like `variables` or `dependencies`), +declare a new job with the same name as the one to override. Place this new job after the template +inclusion and specify any additional keys under it. For example, this disables `DS_REMEDIATE` for +the `gemnasium` analyzer: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml -dependency_scanning: +gemnasium-dependency_scanning: variables: - CI_DEBUG_TRACE: "true" + DS_REMEDIATE: "false" ``` ### Available variables @@ -143,19 +140,20 @@ The following variables allow configuration of global dependency scanning settin | Environment variable | Description | | --------------------------------------- |------------ | -| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_PREFIX` | **DEPRECATED:** Use `SECURE_ANALYZERS_PREFIX` instead. | | `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | -| `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#disabling-docker-in-docker-for-dependency-scanning).| +| `DS_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. | #### Configuring Docker-in-Docker orchestrator -The following variables configure the Docker-in-Docker orchestrator. +The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). | Environment variable | Default | Description | | --------------------------------------- | ----------- | ----------- | -| `DS_ANALYZER_IMAGES` | | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGES` | | Comma-separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | | `DS_ANALYZER_IMAGE_TAG` | | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | | `DS_PULL_ANALYZER_IMAGES` | | Pull the images from the Docker registry (set to `0` to disable). | | `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, or `h`. For example, `300ms`, `1.5h`, or `2h45m`. | @@ -168,20 +166,20 @@ The following variables are used for configuring specific analyzers (used for a | Environment variable | Analyzer | Default | Description | | --------------------------------------- | ------------------ | ---------------------------- |------------ | -| `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local gemnasium database. | -| `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the gemnasium database. | +| `GEMNASIUM_DB_LOCAL_PATH` | `gemnasium` | `/gemnasium-db` | Path to local Gemnasium database. | +| `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | -| `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | +| `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | | `PIP_INDEX_URL` | `gemnasium-python` | `https://pypi.org/simple` | Base URL of Python Package Index. | -| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma separated. | +| `PIP_EXTRA_INDEX_URL` | `gemnasium-python` | | Array of [extra URLs](https://pip.pypa.io/en/stable/reference/pip_install/#cmdoption-extra-index-url) of package indexes to use in addition to `PIP_INDEX_URL`. Comma-separated. | | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) | | `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)| -| `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repos](../index.md#using-private-maven-repos). | -| `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | -| `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | -| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| +| `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repos). | +| `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that will be passed to `gradle` by the analyzer. | +| `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer will pass to `sbt`. | +| `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.| | `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | | `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` environment variable. | @@ -190,39 +188,31 @@ The following variables are used for configuring specific analyzers (used for a ### Using private Maven repos -If you have a private Maven repository which requires login credentials, +If your private Maven repository requires login credentials, you can use the `MAVEN_CLI_OPTS` environment variable. -Read more on [how to use private Maven repos](../index.md#using-private-maven-repos). +Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Disabling Docker in Docker for Dependency Scanning +### Enabling Docker-in-Docker > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12487) in GitLab Ultimate 12.5. -You can avoid the need for Docker in Docker by running the individual analyzers. -This does not require running the executor in privileged mode. For example: +If needed, you can enable Docker-in-Docker to restore the Dependency Scanning behavior that existed +prior to GitLab 13.0. Follow these steps to do so: -```yaml -include: - - template: Dependency-Scanning.gitlab-ci.yml +1. Configure GitLab Runner with Docker-in-Docker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). +1. Set the `DS_DISABLE_DIND` variable to `false`: -variables: - DS_DISABLE_DIND: "true" -``` + ```yaml + include: + - template: Dependency-Scanning.gitlab-ci.yml -This will create individual `<analyzer-name>-dependency_scanning` jobs for each analyzer that runs in your CI/CD pipeline. + variables: + DS_DISABLE_DIND: "false" + ``` -By removing Docker-in-Docker (DIND), GitLab relies on [Linguist](https://github.com/github/linguist) -to start relevant analyzers depending on the detected repository language(s) instead of the -[orchestrator](https://gitlab.com/gitlab-org/security-products/dependency-scanning/). However, there -are some differences in the way repository languages are detected between DIND and non-DIND. You can -observe these differences by checking both Linguist and the common library. For instance, Linguist -looks for `*.java` files to spin up the [gemnasium-maven](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) -image, while orchestrator only looks for the existence of `pom.xml` or `build.gradle`. GitLab uses -Linguist to detect new file types in the default branch. This means that when introducing files or -dependencies for a new language or package manager, the corresponding scans won't be triggered in -the MR and will only run on the default branch once the MR is merged. This will be addressed by -[#211702](https://gitlab.com/gitlab-org/gitlab/-/issues/211702). +This creates a single `dependency_scanning` job in your CI/CD pipeline instead of multiple +`<analyzer-name>-dependency_scanning` jobs. ## Interacting with the vulnerabilities @@ -232,9 +222,8 @@ Once a vulnerability is found, you can interact with it. Read more on how to ## Solutions for vulnerabilities (auto-remediation) Some vulnerabilities can be fixed by applying the solution that GitLab -automatically generates. - -Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). +automatically generates. Read more about the +[solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). ## Security Dashboard @@ -371,33 +360,33 @@ it highlighted: CAUTION: **Deprecation:** Beginning with GitLab 12.9, dependency scanning no longer reports `undefined` severity and confidence levels. -Here is the description of the report file structure nodes and their meaning. All fields are mandatory to be present in -the report JSON unless stated otherwise. Presence of optional fields depends on the underlying analyzers being used. +This table describes the report file structure nodes and their meaning. All fields are mandatory to be present in +the report JSON, unless stated otherwise. The presence of optional fields depends on the underlying analyzers used. | Report JSON node | Description | |------------------------------------------------------|-------------| | `version` | Report syntax version used to generate this JSON. | | `vulnerabilities` | Array of vulnerability objects. | | `vulnerabilities[].id` | Unique identifier of the vulnerability. | -| `vulnerabilities[].category` | Where this vulnerability belongs (SAST, Dependency Scanning etc.). For Dependency Scanning, it will always be `dependency_scanning`. | -| `vulnerabilities[].name` | Name of the vulnerability, this must not include the occurrence's specific information. Optional. | -| `vulnerabilities[].message` | A short text that describes the vulnerability, it may include occurrence's specific information. Optional. | +| `vulnerabilities[].category` | Where this vulnerability belongs, such as SAST or Dependency Scanning. For Dependency Scanning, it will always be `dependency_scanning`. | +| `vulnerabilities[].name` | Name of the vulnerability. Must not include the occurrence's specific information. Optional. | +| `vulnerabilities[].message` | A short text that describes the vulnerability. May include occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | -| `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | +| `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. Used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a `snake_case` string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | -| `vulnerabilities[].location.file` | Path to the dependencies file (e.g., `yarn.lock`). Optional. | +| `vulnerabilities[].location.file` | Path to the dependencies file (such as `yarn.lock`). Optional. | | `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | | `vulnerabilities[].location.dependency.package` | A node that provides the information on the package where the vulnerability is located. | | `vulnerabilities[].location.dependency.package.name` | Name of the package where the vulnerability is located. Optional. | | `vulnerabilities[].location.dependency.version` | Version of the vulnerable package. Optional. | | `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | -| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (e.g. `gemnasium` for [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/)). | +| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (such as `gemnasium` for [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/)). | | `vulnerabilities[].identifiers[].name` | Name of the identifier for display purpose. | | `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purpose. | | `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | @@ -406,10 +395,10 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].links[].url` | URL of the vulnerability details document. Optional. | | `remediations` | An array of objects containing information on cured vulnerabilities along with patch diffs to apply. Empty if no remediations provided by an underlying analyzer. | | `remediations[].fixes` | An array of strings that represent references to vulnerabilities fixed by this particular remediation. | -| `remediations[].fixes[].id` | The id of a fixed vulnerability. | +| `remediations[].fixes[].id` | The ID of a fixed vulnerability. | | `remediations[].fixes[].cve` | (**DEPRECATED - use `remediations[].fixes[].id` instead**) A string value that describes a fixed vulnerability in the same format as `vulnerabilities[].cve`. | | `remediations[].summary` | Overview of how the vulnerabilities have been fixed. | -| `remediations[].diff` | base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | +| `remediations[].diff` | Base64-encoded remediation code diff, compatible with [`git apply`](https://git-scm.com/docs/git-format-patch#_discussion). | ## Versioning and release process @@ -424,32 +413,33 @@ You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security ## Running Dependency Scanning in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for dependency scannings jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). +to external resources through the internet, some adjustments are required for Dependency Scanning +jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md). ### Requirements for offline Dependency Scanning Here are the requirements for using Dependency Scanning in an offline environment: -- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-dependency-scanning) +- Keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. +- Docker Container Registry with locally available copies of Dependency Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. - Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/) - _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db). - _Only if scanning npm/yarn projects_: Host an offline copy of the [retire.js](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [js](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases. NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner will try to pull Docker images from the GitLab container registry even if a local +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we -recommend keeping the pull policy setting to `always` as it will better enable updated scanners to -be utilized within your CI/CD pipelines. +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab Dependency Scanning analyzer images available inside your Docker registry -For Dependency Scanning, import docker images ([supported languages and frameworks](#supported-languages-and-package-managers)) -from `registry.gitlab.com` to your offline docker registry. The Dependency Scanning analyzer -docker images are: +For Dependency Scanning with all [supported languages and frameworks](#supported-languages-and-package-managers), +import the following default Dependency Scanning analyzer images from `registry.gitlab.com` into +your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 @@ -461,39 +451,34 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. +Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) +with new definitions, so consider if you can make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). -### Set Dependency Scanning CI config for "offline" use +### Set Dependency Scanning CI job variables to use local Dependency Scanning analyzers -Below is a general `.gitlab-ci.yml` template to configure your environment for running Dependency -Scanning offline: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace +`DS_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: Dependency-Scanning.gitlab-ci.yml variables: - DS_DISABLE_DIND: "true" DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers" + GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" + GIT_SSL_NO_VERIFY: "true" ``` See explanations of the variables above in the [configuration section](#configuration). ### Specific settings for languages and package managers -For every language and package manager, add the following to the variables section of -`.gitlab-ci.yml`: - -```yaml -GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git" -``` - -See the following sections for additional instructions on specific languages and package managers. +See the following sections for configuring specific languages and package managers. #### JavaScript (npm and yarn) projects @@ -515,10 +500,12 @@ BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git" #### Java (Maven) projects -When using a self-signed certificates, add the following to the variables section of`.gitlab-ci.yml`: +When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`: ```yaml -MAVEN_CLI_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"` +gemnasium-maven-dependency_scanning: + variables: + MAVEN_CLI_OPTS: "-s settings.xml -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true" ``` #### Java (Gradle) projects @@ -527,13 +514,12 @@ When using self-signed certificates, add the following job section to the `.gitl ```yaml gemnasium-maven-dependency_scanning: - variables: - before_script: - - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt - - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt + before_script: + - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt ``` -This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. +This adds the self-signed certificates of your Maven repository to the Java KeyStore of the analyzer's Docker image. #### Scala (sbt) projects @@ -541,41 +527,20 @@ When using self-signed certificates, add the following job section to the `.gitl ```yaml gemnasium-maven-dependency_scanning: - variables: - before_script: - - echo -n | openssl s_client -connect gitlab-airgap-test.us-west1-b.c.group-secure-a89fe7.internal:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt - - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt -``` - -This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image. - -#### Python (pip) and Python (Pipfile) projects - -Add the following `pip.conf` to your repository to define your index URL and trust its self-signed -certificate: - -```toml -[global] -index-url = https://pypi.example.com -trusted-host = pypi.example.com -``` - -Add the following job section to `.gitlab-ci.yml`: - -```yaml -gemnasium-python-dependency_scanning: before_script: - - mkdir ~/.config/pip - - cp pip.conf ~/.config/pip/pip.conf + - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt + - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt ``` +This adds the self-signed certificates of your Maven repository to the Java KeyStore of the analyzer's Docker image. + #### Python (setuptools) -When using self-signed certificates for your private PyPi repo no extra job configuration (aside +When using self-signed certificates for your private PyPi repository, no extra job configuration (aside from the template `.gitlab-ci.yml` above) is needed. However, you must update your `setup.py` to -ensure that it can reach your private repo. Here is an example configuration: +ensure that it can reach your private repository. Here is an example configuration: -1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repo for each +1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repository for each dependency in the `install_requires` list: ```python @@ -596,11 +561,27 @@ ensure that it can reach your private repo. Here is an example configuration: setuptools.ssl_support.cert_paths = ['internal.crt'] ``` +## Limitations + +### Referencing local dependencies using a path in JavaScript projects + +The [Retire.js](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) analyzer +doesn't support dependency references made with [local paths](https://docs.npmjs.com/files/package.json#local-paths) +in the `package.json` of JavaScript projects. The dependency scan outputs the following error for +such references: + +```plaintext +ERROR: Could not find dependencies: <dependency-name>. You may need to run npm install +``` + +As a workaround, remove the [`retire.js`](analyzers.md#selecting-specific-analyzers) analyzer from +[DS_DEFAULT_ANALYZERS](#configuring-dependency-scanning). + ## Troubleshooting ### Error response from daemon: error processing tar file: docker-tar: relocation error -This error occurs when the Docker version used to run the SAST job is `19.03.0`. -You are advised to update to Docker `19.03.1` or greater. Older versions are not +This error occurs when the Docker version that runs the Dependency Scanning job is `19.03.00`. +Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/issues/13830#note_211354992 "Current SAST container fails"). diff --git a/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png b/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png Binary files differnew file mode 100644 index 00000000000..385236d08f2 --- /dev/null +++ b/doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png diff --git a/doc/user/application_security/img/dismissed_info_v12_3.png b/doc/user/application_security/img/dismissed_info_v12_3.png Binary files differdeleted file mode 100644 index 92037493eaa..00000000000 --- a/doc/user/application_security/img/dismissed_info_v12_3.png +++ /dev/null diff --git a/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png b/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png Binary files differnew file mode 100644 index 00000000000..866ad74d42c --- /dev/null +++ b/doc/user/application_security/img/interacting_with_vulnerability_v13_0.png diff --git a/doc/user/application_security/img/interactive_reports.png b/doc/user/application_security/img/interactive_reports.png Binary files differdeleted file mode 100644 index 1b2ef0d3da9..00000000000 --- a/doc/user/application_security/img/interactive_reports.png +++ /dev/null diff --git a/doc/user/application_security/img/security_configuration_page_v12_9.png b/doc/user/application_security/img/security_configuration_page_v12_9.png Binary files differdeleted file mode 100644 index a81d82e03c3..00000000000 --- a/doc/user/application_security/img/security_configuration_page_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/img/security_configuration_page_v13_1.png b/doc/user/application_security/img/security_configuration_page_v13_1.png Binary files differnew file mode 100644 index 00000000000..0147084f705 --- /dev/null +++ b/doc/user/application_security/img/security_configuration_page_v13_1.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index dadff8583db..eefa52eb5c3 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -44,6 +44,12 @@ To add Container Scanning, follow the steps listed in the [Container Scanning do To further configure any of the other scanners, refer to each scanner's documentation. +### Override the default registry base address + +By default, GitLab security scanners use `registry.gitlab.com/gitlab-org/security-products/analyzers` as the +base address for Docker images. You can override this globally by setting the variable +`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once. + ## Security scanning tools GitLab uses the following tools to scan and report known vulnerabilities found in your project. @@ -54,6 +60,7 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | +| [Secret Detection](secret_detection/index.md) **(ULTIMATE)** | Analyze Git history for leaked secrets. | | [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) **(ULTIMATE)** | Analyze source code for known vulnerabilities. | @@ -69,9 +76,10 @@ The scanning tools and vulnerabilities database are updated regularly. | [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | Currently, you do not have to update GitLab to benefit from the latest vulnerabilities definitions. -The security tools are released as Docker images. The vendored job definitions to enable them use -the `x-y-stable` image tags that get overridden each time a new release of the tools is pushed. The -Docker images are updated to match the previous GitLab releases, so users automatically get the +The security tools are released as Docker images. The vendored job definitions that enable them use +major release tags according to [Semantic Versioning](https://semver.org/). Each new release of the +tools overrides these tags. +The Docker images are updated to match the previous GitLab releases, so users automatically get the latest versions of the scanning tools without having to do anything. There are some known issues with this approach, however, and there is a [plan to resolve them](https://gitlab.com/gitlab-org/gitlab/issues/9725). @@ -80,9 +88,6 @@ with this approach, however, and there is a > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.8. -CAUTION: **Warning:** -This feature is currently [Alpha](https://about.gitlab.com/handbook/product/#alpha-beta-ga) and while you can start using it, it may receive important changes in the future. - Each security vulnerability in the merge request report or the [Security Dashboard](security_dashboard/index.md) is actionable. Click an entry to view detailed information with several options: @@ -95,25 +100,27 @@ information with several options: - [Solution](#solutions-for-vulnerabilities-auto-remediation): For some vulnerabilities, a solution is provided for how to fix the vulnerability. - + ### Dismissing a vulnerability -You can dismiss vulnerabilities by clicking the **Dismiss vulnerability** button. -This will dismiss the vulnerability and re-render it to reflect its dismissed state. -If you wish to undo this dismissal, you can click the **Undo dismiss** button. +To dismiss a vulnerability, you must set its status to Dismissed. Follow these steps to do so: + +1. Select the vulnerability in the Security Dashboard. +1. Select **Dismissed** from the **Status** selector menu at the top-right. + +You can undo this action by selecting a different status from the same menu. #### Adding a dismissal reason > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. -When dismissing a vulnerability, it's often helpful to provide a reason for doing so. -If you press the comment button next to **Dismiss vulnerability** in the modal, -a text box appears for you to add a comment with your dismissal. -Once added, you can edit or delete it. This allows you to add and update -context for a vulnerability as you learn more over time. +When dismissing a vulnerability, it's often helpful to provide a reason for doing so. Upon setting a +vulnerability's status to Dismissed, a text box appears for you to add a comment with your +dismissal. Once added, you can edit or delete it. This allows you to add and update context for a +vulnerability as you learn more over time. - + #### Dismissing multiple vulnerabilities @@ -193,9 +200,19 @@ security team when a merge request would introduce one of the following security - A security vulnerability - A software license compliance violation -This threshold is defined as `high`, `critical`, or `unknown` severity. When any vulnerabilities are -present within a merge request, an approval is required from the `Vulnerability-Check` approver -group. +The security vulnerability threshold is defined as `high`, `critical`, or `unknown` severity. The +`Vulnerability-Check` approver group must approve merge requests that contain vulnerabilities. + +When GitLab can assess vulnerability severity, the rating can be one of the following: + +- `unknown` +- `low` +- `medium` +- `high` +- `critical` + +The rating `unknown` indicates that the underlying scanner doesn't contain or provide a severity +rating. ### Enabling Security Approvals within a project @@ -209,7 +226,7 @@ Any code changes cause the approvals required to reset. An approval is required when a security report: -- Contains a new vulnerability of `high`, `critical`, or `unknown` severity. +- Contains a new vulnerability of `high`, `critical`, or `unknown` severity, regardless of dismissal. - Is not generated during pipeline execution. An approval is optional when a security report: @@ -259,7 +276,7 @@ to pass a username and password. You can set it under your project's settings so that your credentials aren't exposed in `.gitlab-ci.yml`. If the username is `myuser` and the password is `verysecret` then you would -[set the following variable](../../ci/variables/README.md#via-the-ui) +[set the following variable](../../ci/variables/README.md#create-a-custom-variable-in-the-ui) under your project's settings: | Type | Key | Value | @@ -313,7 +330,8 @@ You can do it quickly by following the hyperlink given to run a new pipeline. ### Getting error message `sast job: stage parameter should be [some stage name here]` -When including a security job template like [`SAST`](sast/index.md#configuration), +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), the following error may occur, depending on your GitLab CI/CD configuration: ```plaintext @@ -326,15 +344,115 @@ This error appears when the included job's stage (named `test`) isn't declared i To fix this issue, you can either: - Add a `test` stage in your `.gitlab-ci.yml`. -- Change the default stage of the included security jobs. For example, with `SAST`: +- Change the default stage of the included security jobs. For example, with SpotBugs (SAST): ```yaml include: template: SAST.gitlab-ci.yml - sast: + spotbugs-sast: stage: unit-tests ``` -[Learn more on overriding the SAST template](sast/index.md#overriding-the-sast-template). +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). All the security scanning tools define their stage, so this error can occur with all of them. + +### Getting error message `sast job: config key may not be used with 'rules': only/except` + +When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template +like [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml), +the following error may occur, depending on your GitLab CI/CD configuration: + +```plaintext +Found errors in your .gitlab-ci.yml: + + jobs:sast config key may not be used with `rules`: only/except +``` + +This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) +with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#onlyexcept-basic) +To fix this issue, you must either: + +- [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). +- (Temporarily) [Pin your templates to the deprecated versions](#pin-your-templates-to-the-deprecated-versions) + +[Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). + +#### Transitioning your `only/except` syntax to `rules` + +When overriding the template to control job execution, previous instances of +[`only` or `except`](../../ci/yaml/README.md#onlyexcept-basic) are no longer compatible +and must be transitioned to [the `rules` syntax](../../ci/yaml/README.md#rules). + +If your override is aimed at limiting jobs to only run on `master`, the previous syntax +would look similar to: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on master or merge requests +spotbugs-sast: + only: + refs: + - master + - merge_requests +``` + +To transition the above configuration to the new `rules` syntax, the override +would be written as follows: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is only executed on master or merge requests +spotbugs-sast: + rules: + - if: $CI_COMMIT_BRANCH == "master" + - if: $CI_MERGE_REQUEST_ID +``` + +If your override is aimed at limiting jobs to only run on branches, not tags, +it would look similar to: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + except: + - tags +``` + +To transition to the new `rules` syntax, the override would be rewritten as: + +```yaml +include: + - template: SAST.gitlab-ci.yml + +# Ensure that the scanning is not executed on tags +spotbugs-sast: + rules: + - if: $CI_COMMIT_TAG == null +``` + +[Learn more on the usage of `rules`](../../ci/yaml/README.md#rules). + +#### Pin your templates to the deprecated versions + +To ensure the latest support, we **strongly** recommend that you migrate to [`rules`](../../ci/yaml/README.md#rules). + +If you're unable to immediately update your CI configuration, there are several workarounds that +involve pinning to the previous template versions, for example: + + ```yaml + include: + remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/12-10-stable-ee/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml' + ``` + +Additionally, we provide a dedicated project containing the versioned legacy templates. +This can be useful for offline setups or anyone wishing to use [Auto DevOps](../../topics/autodevops/index.md). + +Instructions are available in the [legacy template project](https://gitlab.com/gitlab-org/auto-devops-v12-10). diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 61b34901849..3deb38902f2 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -61,6 +61,14 @@ Once a vulnerability is found, you can interact with it. Read more on how to Please note that in some cases the reported vulnerabilities provide metadata that can contain external links exposed in the UI. These links might not be accessible within an offline environment. +### Suggested Solutions for vulnerabilities + +The [suggested solutions](../index.md#solutions-for-vulnerabilities-auto-remediation) feature +(auto-remediation) is available for Dependency Scanning and Container Scanning, but may not work +depending on your instance's configuration. We can only suggest solutions, which are generally more +current versions that have been patched, when we are able to access up-to-date registry services +hosting the latest versions of that dependency or image. + ### Scanner signature and rule updates When connected to the internet, some scanners will reference public databases @@ -79,3 +87,4 @@ above. You can find more information at each of the pages below: - [SAST offline directions](../sast/index.md#running-sast-in-an-offline-environment) - [DAST offline directions](../dast/index.md#running-dast-in-an-offline-environment) - [License Compliance offline directions](../../compliance/license_compliance/index.md#running-license-compliance-in-an-offline-environment) +- [Dependency Scanning offline directions](../dependency_scanning/index.md#running-dependency-scanning-in-an-offline-environment) diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 9010a7cae0b..08078a66719 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -52,7 +52,7 @@ include: - template: SAST.gitlab-ci.yml variables: - SAST_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images + SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images ``` This configuration requires that your custom registry provides images for all @@ -149,7 +149,7 @@ The [Security Scanner Integration](../../../development/integrations/secure.md) | End line | ✓ | ✓ | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | | Start column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | ✓ | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | ✓ | | End column | ✓ | 𐄂 | 𐄂 | ✓ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | -| External id (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g. CVE) | 𐄂 | 𐄂 | ⚠ | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | URLs | ✓ | 𐄂 | ✓ | 𐄂 | ⚠ | 𐄂 | ⚠ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | | Internal doc/explanation | ✓ | ⚠ | ✓ | 𐄂 | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | ✓ | 𐄂 | | Solution | ✓ | 𐄂 | 𐄂 | 𐄂 | ⚠ | ✓ | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | 𐄂 | diff --git a/doc/user/application_security/sast/img/sast_v12_9.png b/doc/user/application_security/sast/img/sast_v12_9.png Binary files differdeleted file mode 100644 index 3c6ee7a276b..00000000000 --- a/doc/user/application_security/sast/img/sast_v12_9.png +++ /dev/null diff --git a/doc/user/application_security/sast/img/sast_v13_0.png b/doc/user/application_security/sast/img/sast_v13_0.png Binary files differnew file mode 100644 index 00000000000..b4aea6ea466 --- /dev/null +++ b/doc/user/application_security/sast/img/sast_v13_0.png diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index a6457d58fe2..370c6d0e8e7 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -13,7 +13,7 @@ to learn how to protect your organization. ## Overview -If you are using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known +If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your source code for known vulnerabilities using Static Application Security Testing (SAST). You can take advantage of SAST by doing one of the following: @@ -25,7 +25,7 @@ You can take advantage of SAST by doing one of the following: GitLab checks the SAST report, compares the found vulnerabilities between the source and target branches, and shows the information right on the merge request. - + The results are sorted by the priority of the vulnerability: @@ -36,6 +36,9 @@ The results are sorted by the priority of the vulnerability: 1. Unknown 1. Everything else +NOTE: **Note:** +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard won't show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard won't show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Use cases - Your code has a potentially dangerous attribute in a class, or unsafe code @@ -45,19 +48,17 @@ The results are sorted by the priority of the vulnerability: ## Requirements -To run a SAST job, by default, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +To run SAST jobs, by default, you need GitLab Runner with the +[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or +[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +If you're using the shared Runners on GitLab.com, this is enabled by default. -Privileged mode is not necessary if you've [disabled Docker in Docker -for SAST](#disabling-docker-in-docker-for-sast) +Beginning with GitLab 13.0, Docker privileged mode is necessary only if you've [enabled Docker-in-Docker for SAST](#enabling-docker-in-docker). CAUTION: **Caution:** Our SAST jobs currently expect a Linux container type. Windows containers are not yet supported. CAUTION: **Caution:** -If you use your own Runners, make sure that the Docker version you have installed +If you use your own Runners, make sure the Docker version installed is **not** `19.03.0`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details. ## Supported languages and frameworks @@ -66,9 +67,10 @@ The following table shows which languages, package managers and frameworks are s | Language (package managers) / framework | Scan tool | Introduced in GitLab Version | |-----------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------------------------| -| .NET | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Core | [Security Code Scan](https://security-code-scan.github.io) | 11.0 | +| .NET Framework | [Security Code Scan](https://security-code-scan.github.io) | 13.0 | | Any | [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) | 11.9 | -| Apex (Salesforce) | [pmd](https://pmd.github.io/pmd/index.html) | 12.1 | +| Apex (Salesforce) | [PMD](https://pmd.github.io/pmd/index.html) | 12.1 | | C/C++ | [Flawfinder](https://dwheeler.com/flawfinder/) | 10.7 | | Elixir (Phoenix) | [Sobelow](https://github.com/nccgroup/sobelow) | 11.10 | | Go | [Gosec](https://github.com/securego/gosec) | 10.7 | @@ -82,7 +84,7 @@ The following table shows which languages, package managers and frameworks are s | React | [ESLint react plugin](https://github.com/yannickcr/eslint-plugin-react) | 12.5 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | -| TypeScript | [TSLint config security](https://github.com/webschik/tslint-config-security/) | 11.9 | +| TypeScript | [`tslint-config-security`](https://github.com/webschik/tslint-config-security/) | 11.9 | NOTE: **Note:** The Java analyzers can also be used for variants like the @@ -101,7 +103,7 @@ provided by [Auto DevOps](../../../topics/autodevops/index.md). For GitLab 11.9 and later, to enable SAST you must [include](../../../ci/yaml/README.md#includetemplate) the [`SAST.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) -that is provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you +provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined that template. Add the following to your `.gitlab-ci.yml` file: @@ -111,22 +113,19 @@ include: - template: SAST.gitlab-ci.yml ``` -The included template will create a `sast` job in your CI/CD pipeline and scan +The included template will create SAST jobs in your CI/CD pipeline and scan your project's source code for possible vulnerabilities. The results will be saved as a -[SAST report artifact](../../../ci/yaml/README.md#artifactsreportssast-ultimate) +[SAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportssast-ultimate) that you can later download and analyze. Due to implementation limitations, we -always take the latest SAST artifact available. Behind the scenes, the -[GitLab SAST Docker image](https://gitlab.com/gitlab-org/security-products/sast) -is used to detect the languages/frameworks and in turn runs the matching scan tools. +always take the latest SAST artifact available. ### Customizing the SAST settings The SAST settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. - In the following example, we include the SAST template and at the same time we set the `SAST_GOSEC_LEVEL` variable to `2`: @@ -139,25 +138,26 @@ variables: ``` Because the template is [evaluated before](../../../ci/yaml/README.md#include) -the pipeline configuration, the last mention of the variable will take precedence. +the pipeline configuration, the last mention of the variable takes precedence. -### Overriding the SAST template +### Overriding SAST jobs CAUTION: **Deprecation:** Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. -If you want to override the job definition (for example, change properties like -`variables` or `dependencies`), you need to declare a `sast` job after the -template inclusion and specify any additional keys under it. For example: +To override a job definition, (for example, change properties like `variables` or `dependencies`), +declare a job with the same name as the SAST job to override. Place this new job after the template +inclusion and specify any additional keys under it. For example, this enables `FAIL_NEVER` for the +`spotbugs` analyzer: ```yaml include: - template: SAST.gitlab-ci.yml -sast: +spotbugs-sast: variables: - CI_DEBUG_TRACE: "true" + FAIL_NEVER: 1 ``` ### Using environment variables to pass credentials for private repositories @@ -170,57 +170,42 @@ it via [custom environment variables](#custom-environment-variables). #### Using a variable to pass username and password to a private Maven repository -If you have a private Maven repository which requires login credentials, +If your private Maven repository requires login credentials, you can use the `MAVEN_CLI_OPTS` environment variable. -Read more on [how to use private Maven repos](../index.md#using-private-maven-repos). +Read more on [how to use private Maven repositories](../index.md#using-private-maven-repos). -### Disabling Docker in Docker for SAST +### Enabling Docker-in-Docker -You can avoid the need for Docker in Docker by running the individual analyzers. -This does not require running the executor in privileged mode. For example: +If needed, you can enable Docker-in-Docker to restore the SAST behavior that existed prior to GitLab +13.0. Follow these steps to do so: -```yaml -include: - - template: SAST.gitlab-ci.yml +1. Configure GitLab Runner with Docker-inDocker in [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). +1. Set the variable `SAST_DISABLE_DIND` set to `false`: -variables: - SAST_DISABLE_DIND: "true" -``` + ```yaml + include: + - template: SAST.gitlab-ci.yml -This will create individual `<analyzer-name>-sast` jobs for each analyzer that runs in your CI/CD pipeline. + variables: + SAST_DISABLE_DIND: "false" + ``` -By removing Docker-in-Docker (DIND), GitLab relies on [Linguist](https://github.com/github/linguist) -to start relevant analyzers depending on the detected repository language(s) instead of the -[orchestrator](https://gitlab.com/gitlab-org/security-products/dependency-scanning/). However, there -are some differences in the way repository languages are detected between DIND and non-DIND. You can -observe these differences by checking both Linguist and the common library. For instance, Linguist -looks for `*.java` files to spin up the [spotbugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) -image, while orchestrator only looks for the existence of `pom.xml`, `build.xml`, `gradlew`, -`grailsw`, or `mvnw`. GitLab uses Linguist to detect new file types in the default branch. This -means that when introducing files or dependencies for a new language or package manager, the -corresponding scans won't be triggered in the MR and will only run on the default branch once the -MR is merged. This will be addressed by [#211702](https://gitlab.com/gitlab-org/gitlab/-/issues/211702). - -NOTE: **Note:** -With the current language detection logic, any new languages or frameworks introduced within the -context of a merge request don't trigger a corresponding scan. These scans only occur once the code -is committed to the default branch. +This creates a single `sast` job in your CI/CD pipeline instead of multiple `<analyzer-name>-sast` +jobs. -#### Enabling kubesec analyzer +#### Enabling Kubesec analyzer > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12752) in GitLab Ultimate 12.6. -When [Docker in Docker is disabled](#disabling-docker-in-docker-for-sast), -you will need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the -kubesec analyzer. In `.gitlab-ci.yml`, define: +You need to set `SCAN_KUBERNETES_MANIFESTS` to `"true"` to enable the +Kubesec analyzer. In `.gitlab-ci.yml`, define: ```yaml include: - template: SAST.gitlab-ci.yml variables: - SAST_DISABLE_DIND: "true" SCAN_KUBERNETES_MANIFESTS: "true" ``` @@ -246,9 +231,6 @@ stages: include: - template: SAST.gitlab-ci.yml -variables: - SAST_DISABLE_DIND: "true" - build: stage: build script: @@ -291,10 +273,11 @@ The following are Docker image-related variables. | Environment variable | Description | |------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `SAST_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | +| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `SAST_ANALYZER_IMAGE_PREFIX` | **DEPRECATED**: Use `SECURE_ANALYZERS_PREFIX` instead. | +| `SAST_ANALYZER_IMAGE_TAG` | **DEPRECATED:** Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). | | `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). | -| `SAST_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-sast). | +| `SAST_DISABLE_DIND` | Disable Docker-in-Docker and run analyzers [individually](#enabling-docker-in-docker). This variable is `true` by default. | #### Vulnerability filters @@ -307,25 +290,22 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre | `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | | `SAST_FLAWFINDER_LEVEL` | 1 | Ignore Flawfinder vulnerabilities under given risk level. Integer, 0=No risk, 5=High risk. | | `SAST_GITLEAKS_ENTROPY_LEVEL` | 8.0 | Minimum entropy for secret detection. Float, 0.0 = low, 8.0 = high. | -| `SAST_GOSEC_LEVEL` | 0 | Ignore gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | -| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a gitleaks scan starts at. | -| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a gitleaks scan ends at. | -| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic gitleaks scan. | +| `SAST_GOSEC_LEVEL` | 0 | Ignore Gosec vulnerabilities under given confidence level. Integer, 0=Undefined, 1=Low, 2=Medium, 3=High. | +| `SAST_GITLEAKS_COMMIT_FROM` | - | The commit a Gitleaks scan starts at. | +| `SAST_GITLEAKS_COMMIT_TO` | - | The commit a Gitleaks scan ends at. | +| `SAST_GITLEAKS_HISTORIC_SCAN` | false | Flag to enable a historic Gitleaks scan. | #### Docker-in-Docker orchestrator -The following variables configure the Docker-in-Docker orchestrator. +The following variables configure the Docker-in-Docker orchestrator, and therefore are only used when the Docker-in-Docker mode is [enabled](#enabling-docker-in-docker). | Environment variable | Default value | Description | |------------------------------------------|---------------|-------------| -| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | -| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). Not available when [Docker-in-Docker is disabled](#disabling-docker-in-docker-for-sast). | -| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". | -| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m". | -| `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". For example, "300ms", "1.5h" or "2h45m".| - -NOTE: **Note:** -Timeout variables are not applicable for setups with [disabled Docker In Docker](index.md#disabling-docker-in-docker-for-sast). +| `SAST_ANALYZER_IMAGES` | | Comma-separated list of custom images. Default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `SAST_PULL_ANALYZER_IMAGES` | 1 | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). | +| `SAST_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | 2m | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | +| `SAST_PULL_ANALYZER_IMAGE_TIMEOUT` | 5m | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`. | +| `SAST_RUN_ANALYZER_TIMEOUT` | 20m | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h` or `2h45m`.| #### Analyzer settings @@ -333,25 +313,26 @@ Some analyzers can be customized with environment variables. | Environment variable | Analyzer | Description | |-----------------------------|----------|-------------| -| `SCAN_KUBERNETES_MANIFESTS` | kubesec | Set to `"true"` to scan Kubernetes manifests when [Docker in Docker](#disabling-docker-in-docker-for-sast) is disabled. | -| `ANT_HOME` | spotbugs | The `ANT_HOME` environment variable. | -| `ANT_PATH` | spotbugs | Path to the `ant` executable. | -| `GRADLE_PATH` | spotbugs | Path to the `gradle` executable. | -| `JAVA_OPTS` | spotbugs | Additional arguments for the `java` executable. | -| `JAVA_PATH` | spotbugs | Path to the `java` executable. | -| `SAST_JAVA_VERSION` | spotbugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | -| `MAVEN_CLI_OPTS` | spotbugs | Additional arguments for the `mvn` or `mvnw` executable. | -| `MAVEN_PATH` | spotbugs | Path to the `mvn` executable. | -| `MAVEN_REPO_PATH` | spotbugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | -| `SBT_PATH` | spotbugs | Path to the `sbt` executable. | -| `FAIL_NEVER` | spotbugs | Set to `1` to ignore compilation failure. | +| `SCAN_KUBERNETES_MANIFESTS` | Kubesec | Set to `"true"` to scan Kubernetes manifests. | +| `ANT_HOME` | SpotBugs | The `ANT_HOME` environment variable. | +| `ANT_PATH` | SpotBugs | Path to the `ant` executable. | +| `GRADLE_PATH` | SpotBugs | Path to the `gradle` executable. | +| `JAVA_OPTS` | SpotBugs | Additional arguments for the `java` executable. | +| `JAVA_PATH` | SpotBugs | Path to the `java` executable. | +| `SAST_JAVA_VERSION` | SpotBugs | Which Java version to use. Supported versions are `8` and `11`. Defaults to `8`. | +| `MAVEN_CLI_OPTS` | SpotBugs | Additional arguments for the `mvn` or `mvnw` executable. | +| `MAVEN_PATH` | SpotBugs | Path to the `mvn` executable. | +| `MAVEN_REPO_PATH` | SpotBugs | Path to the Maven local repository (shortcut for the `maven.repo.local` property). | +| `SBT_PATH` | SpotBugs | Path to the `sbt` executable. | +| `FAIL_NEVER` | SpotBugs | Set to `1` to ignore compilation failure. | +| `SAST_GOSEC_CONFIG` | Gosec | Path to configuration for Gosec (optional). | #### Custom environment variables > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18193) in GitLab Ultimate 12.5. In addition to the aforementioned SAST configuration variables, -all [custom environment variables](../../../ci/variables/README.md#creating-a-custom-environment-variable) are propagated +all [custom environment variables](../../../ci/variables/README.md#custom-environment-variables) are propagated to the underlying SAST analyzer images if [the SAST vendored template](#configuration) is used. @@ -451,16 +432,16 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `version` | Report syntax version used to generate this JSON. | | `vulnerabilities` | Array of vulnerability objects. | | `vulnerabilities[].id` | Unique identifier of the vulnerability. | -| `vulnerabilities[].category` | Where this vulnerability belongs (SAST, Dependency Scanning etc.). For SAST, it will always be `sast`. | -| `vulnerabilities[].name` | Name of the vulnerability, this must not include the occurrence's specific information. Optional. | +| `vulnerabilities[].category` | Where this vulnerability belongs (such as SAST, Dependency Scanning). For SAST, it will always be `sast`. | +| `vulnerabilities[].name` | Name of the vulnerability. Must not include the occurrence's specific information. Optional. | | `vulnerabilities[].message` | A short text that describes the vulnerability, it may include the occurrence's specific information. Optional. | | `vulnerabilities[].description` | A long text that describes the vulnerability. Optional. | | `vulnerabilities[].cve` | (**DEPRECATED - use `vulnerabilities[].id` instead**) A fingerprint string value that represents a concrete occurrence of the vulnerability. It's used to determine whether two vulnerability occurrences are same or different. May not be 100% accurate. **This is NOT a [CVE](https://cve.mitre.org/)**. | -| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Undefined` (an analyzer has not provided this information), `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | -| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Undefined` (an analyzer has not provided this information), `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | +| `vulnerabilities[].severity` | How much the vulnerability impacts the software. Possible values: `Info`, `Unknown`, `Low`, `Medium`, `High`, `Critical`. | +| `vulnerabilities[].confidence` | How reliable the vulnerability's assessment is. Possible values: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, `Confirmed`. | | `vulnerabilities[].solution` | Explanation of how to fix the vulnerability. Optional. | | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | -| `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | +| `vulnerabilities[].scanner.id` | ID of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | | `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.file` | Path to the file where the vulnerability is located. Optional. | @@ -468,29 +449,15 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].location.end_line` | The last line of the code affected by the vulnerability. Optional. | | `vulnerabilities[].location.class` | If specified, provides the name of the class where the vulnerability is located. Optional. | | `vulnerabilities[].location.method` | If specified, provides the name of the method where the vulnerability is located. Optional. | -| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external DBs. | -| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (e.g., `bandit_test_id` for [Bandit analyzer](https://wiki.openstack.org/wiki/Security/Projects/Bandit)). | +| `vulnerabilities[].identifiers` | An ordered array of references that identify a vulnerability on internal or external databases. | +| `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (like `bandit_test_id` for [Bandit analyzer](https://wiki.openstack.org/wiki/Security/Projects/Bandit)). | | `vulnerabilities[].identifiers[].name` | Name of the identifier for display purposes. | | `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purposes. | | `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | ## Secret detection -GitLab is also able to detect secrets and credentials that have been unintentionally pushed to the -repository (for example, an API key that allows write access to third-party deployment -environments). - -This check is performed by a specific analyzer during the `sast` job. It runs regardless of the programming -language of your app, and you don't need to change anything to your -CI/CD configuration file to turn it on. Results are available in the SAST report. - -GitLab currently includes [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) checks. - -NOTE: **Note:** -The secrets analyzer will ignore "Password in URL" vulnerabilities if the password begins -with a dollar sign (`$`) as this likely indicates the password being used is an environment -variable. For example, `https://username:$password@example.com/path/to/repo` will not be -detected, whereas `https://username:password@example.com/path/to/repo` would be detected. +Learn more about [Secret Detection](../secret_detection). ## Security Dashboard @@ -503,7 +470,12 @@ vulnerabilities in your groups, projects and pipelines. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). -## Vulnerabilities database update +## Vulnerabilities database + +Vulnerabilities contained within the vulnerability database can be searched +and viewed at the [GitLab vulnerability advisory database](https://advisories.gitlab.com). + +### Vulnerabilities database update For more information about the vulnerabilities database update, check the [maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). @@ -512,29 +484,29 @@ For more information about the vulnerabilities database update, check the For self-managed GitLab instances in an environment with limited, restricted, or intermittent access to external resources through the internet, some adjustments are required for the SAST job to -successfully run. For more information, see [Offline environments](../offline_deployments/index.md). +run successfully. For more information, see [Offline environments](../offline_deployments/index.md). ### Requirements for offline SAST To use SAST in an offline environment, you need: -- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-sast) +- To keep Docker-In-Docker disabled (default). - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). - Docker Container Registry with locally available copies of SAST [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images. NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the runner will try to pull Docker images from the GitLab container registry even if a local +meaning the Runner tries to pull Docker images from the GitLab container registry even if a local copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we -recommend keeping the pull policy setting to `always` as it will better enable updated scanners to -be utilized within your CI/CD pipelines. +recommend keeping the pull policy setting to `always` if not in an offline environment, as this +enables the use of updated scanners in your CI/CD pipelines. ### Make GitLab SAST analyzer images available inside your Docker registry For SAST with all [supported languages and frameworks](#supported-languages-and-frameworks), -import the following default SAST analyzer images from `registry.gitlab.com` to your local "offline" -registry: +import the following default SAST analyzer images from `registry.gitlab.com` into your +[local Docker container registry](../../packages/container_registry/index.md): ```plaintext registry.gitlab.com/gitlab-org/security-products/analyzers/bandit:2 @@ -557,7 +529,7 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/tslint:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +with new definitions, so consider if you're able to make periodic updates yourself. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -565,18 +537,15 @@ For details on saving and transporting Docker images as a file, see Docker's doc ### Set SAST CI job variables to use local SAST analyzers -[Override SAST environment variables](#customizing-the-sast-settings) to use to your [local container registry](./analyzers.md#using-a-custom-docker-mirror) -as the source for SAST analyzer images. - -For example, assuming a local Docker registry repository of `localhost:5000/analyzers`: +Add the following configuration to your `.gitlab-ci.yml` file. You must replace +`SAST_ANALYZER_IMAGE_PREFIX` to refer to your local Docker container registry: ```yaml include: - template: SAST.gitlab-ci.yml variables: - SAST_ANALYZER_IMAGE_PREFIX: "localhost:5000/analyzers" - SAST_DISABLE_DIND: "true" + SECURE_ANALYZERS_PREFIX: "localhost:5000/analyzers" ``` The SAST job should now use local copies of the SAST analyzers to scan your code and generate @@ -586,7 +555,7 @@ security reports without requiring internet access. ### Error response from daemon: error processing tar file: docker-tar: relocation error -This error occurs when the Docker version used to run the SAST job is `19.03.0`. -You are advised to update to Docker `19.03.1` or greater. Older versions are not +This error occurs when the Docker version that runs the SAST job is `19.03.0`. +Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/issues/13830#note_211354992 "Current SAST container fails"). diff --git a/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png Binary files differnew file mode 100644 index 00000000000..17893610f10 --- /dev/null +++ b/doc/user/application_security/secret_detection/img/secret-detection-merge-request-ui.png diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md new file mode 100644 index 00000000000..74dd3c89984 --- /dev/null +++ b/doc/user/application_security/secret_detection/index.md @@ -0,0 +1,67 @@ +--- +type: reference, howto +--- + +# Secret Detection **(ULTIMATE)** + +> [Introduced](https://about.gitlab.com/releases/2019/03/22/gitlab-11-9-released/#detect-secrets-and-credentials-in-the-repository) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9. + +## Overview + +A recurring problem when developing applications is that developers may unintentionally commit +secrets and credentials to their remote repositories. If other people have access to the source, +or if the project is public, the sensitive information is then exposed and can be leveraged by +malicious users to gain access to resources like deployment environments. + +GitLab 11.9 includes a new check called Secret Detection. It scans the content of the repository +to find API keys and other information that should not be there. + +GitLab displays identified secrets as part of the SAST reports visibly in a few places: + +- [Security Dashboard](../security_dashboard/) +- Pipelines' **Security** tab +- Report in the merge request widget + + + +## Use cases + +- Detecting accidental commit of secrets like keys, passwords, and API tokens. +- Performing a single or recurring scan of the full history of your repository for secrets. + +## Configuration + +If you already have SAST enabled for your app, you don’t need to take any action to benefit from this +new feature. It is also included in the Auto DevOps default configuration. + +Secret Detection is performed by a [specific analyzer](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L180) +during the `sast` job. It runs regardless of the programming +language of your app, and you don't need to change your +CI/CD configuration file to enable it. Results are available in the SAST report. + +The Secret Detection analyzer includes [Gitleaks](https://github.com/zricethezav/gitleaks) and [TruffleHog](https://github.com/dxa4481/truffleHog) checks. + +NOTE: **Note:** +The Secret Detection analyzer will ignore "Password in URL" vulnerabilities if the password begins +with a dollar sign (`$`) as this likely indicates the password being used is an environment +variable. For example, `https://username:$password@example.com/path/to/repo` won't be +detected, whereas `https://username:password@example.com/path/to/repo` would be detected. + +## Full History Secret Scan + +GitLab 12.11 introduced support for scanning the full history of a reposity. This new functionality +is particularly useful when you are enabling Secret Detection in a repository for the first time and you +want to perform a full secret scan. Running a secret scan on the full history can take a long time, +especially for larger repositories with lengthy Git histories. We recommend not setting this variable +as part of your normal job defintion. + +A new configuration variable ([`SAST_GITLEAKS_HISTORIC_SCAN`](../sast/#vulnerability-filters)) +can be set to change the behavior of the GitLab Secret Detection scan to run on the entire Git history of a repository. + +We have created a [short video walkthrough](https://youtu.be/wDtc_K00Y0A) showcasing how you can perform a full history secret scan. +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=wDtc_K00Y0A">Walkthrough of historical secret scan</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/wDtc_K00Y0A" frameborder="0" allowfullscreen="true"> </iframe> +</figure> diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png Binary files differdeleted file mode 100644 index c93a3ce8c35..00000000000 --- a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v12_6.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png Binary files differnew file mode 100644 index 00000000000..c788e2165ad --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/group_security_dashboard_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png Binary files differnew file mode 100644 index 00000000000..77e75551bd9 --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png Binary files differdeleted file mode 100644 index fd0548d0b34..00000000000 --- a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v12_8.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png Binary files differnew file mode 100644 index 00000000000..a500f186c2b --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_0.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12_10.png Binary files differindex 07b41b471d4..07b41b471d4 100644 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12_10.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png Binary files differdeleted file mode 100644 index 51e80bdb50d..00000000000 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v12_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png Binary files differnew file mode 100644 index 00000000000..bb88b7f371c --- /dev/null +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_0.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 42b28b7b9f2..2988b3642ef 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -30,7 +30,7 @@ To use the instance, group, project, or pipeline security dashboard: 1. At least one project inside a group must be configured with at least one of the [supported reports](#supported-reports). -1. The configured jobs must use the [new `reports` syntax](../../../ci/yaml/README.md#artifactsreports). +1. The configured jobs must use the [new `reports` syntax](../../../ci/pipelines/job_artifacts.md#artifactsreports). 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used. If you're using the shared Runners on GitLab.com, this is already the case. @@ -44,15 +44,17 @@ Visit the page for any pipeline which has run any of the [supported reports](#su  +NOTE: **Note:** +A pipeline consists of multiple jobs, including SAST and DAST scanning. If any job fails to finish for any reason, the security dashboard will not show SAST scanner output. For example, if the SAST job finishes but the DAST job fails, the security dashboard will not show SAST results. The analyzer will output an [exit code](../../../development/integrations/secure.md#exit-code) on failure. + ## Project Security Dashboard > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6165) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.1. -At the project level, the Security Dashboard displays the latest security reports -for your project from the last successful pipeline. Use it to find and fix vulnerabilities affecting the -[default branch](../../project/repository/branches/index.md#default-branch). +At the project level, the Security Dashboard displays the latest security reports for your project. +Use it to find and fix vulnerabilities. - + ### Export vulnerabilities @@ -64,7 +66,7 @@ NOTE: **Note:** It may take several minutes for the download to start if your project consists of thousands of vulnerabilities. Do not close the page until the download finishes. - + ## Group Security Dashboard @@ -78,32 +80,27 @@ First, navigate to the Security Dashboard found under your group's Once you're on the dashboard, at the top you should see a series of filters for: +- Status - Severity -- Confidence - Report type -- Project - -To the right of the filters, you should see a **Hide dismissed** toggle button. NOTE: **Note:** -The dashboard only shows projects with [security reports](#supported-reports) enabled in a group -according to the last successful projects' pipelines. +The dashboard only shows projects with [security reports](#supported-reports) enabled in a group. - + -Selecting one or more filters will filter the results in this page. Disabling the **Hide dismissed** -toggle button will let you also see vulnerabilities that have been dismissed. +Selecting one or more filters will filter the results in this page. The main section is a list of all the vulnerabilities in the group, sorted by severity. In that list, you can see the severity of the vulnerability, its name, its confidence (likelihood of the vulnerability to be a positive one), and the project it's from. -If you hover over a row, there will appear some actions you can take: +If you hover over a row, the following actions appear: -- "More info" -- "Create issue" -- "Dismiss vulnerability" +- More info +- Create issue +- Dismiss vulnerability Next to the list is a timeline chart that shows how many open vulnerabilities your projects had at various points in time. You can filter among 30, 60, and @@ -147,7 +144,23 @@ To add projects to the dashboard: Once added, the dashboard will display the vulnerabilities found in your chosen projects. - + + +### Export vulnerabilities + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213014) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. + +You can export all your vulnerabilities as CSV by clicking the **{upload}** **Export** +button located at top right of the **Instance Security Dashboard**. After the report +is built, the CSV report downloads to your local machine. The report contains all +vulnerabilities for the projects defined in the **Instance Security Dashboard**, +as filters don't apply to the export function. + +NOTE: **Note:** +It may take several minutes for the download to start if your project contains +thousands of vulnerabilities. Do not close the page until the download finishes. + + ## Keeping the dashboards up to date diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index 482fceea680..7bd148edd15 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -21,7 +21,7 @@ The Web Application Firewall section provides metrics for the NGINX Ingress controller and ModSecurity firewall. This section has the following prerequisites: -- Project has to have at least one [environment](../../../ci/environments.md). +- Project has to have at least one [environment](../../../ci/environments/index.md). - [Web Application Firewall](../../clusters/applications.md#web-application-firewall-modsecurity) has to be enabled. - [Elastic Stack](../../clusters/applications.md#web-application-firewall-modsecurity) has to be installed. @@ -38,7 +38,7 @@ about your Ingress traffic: If a significant percentage of traffic is anomalous, you should investigate it for potential threats by -[examining the application logs](../../clusters/applications.md#web-application-firewall-modsecurity). +[examining the Web Application Firewall logs](../../clusters/applications.md#web-application-firewall-modsecurity). ## Container Network Policy @@ -48,7 +48,7 @@ The **Container Network Policy** section provides packet flow metrics for your application's Kubernetes namespace. This section has the following prerequisites: -- Your project contains at least one [environment](../../../ci/environments.md) +- Your project contains at least one [environment](../../../ci/environments/index.md) - You've [installed Cilium](../../clusters/applications.md#install-cilium-using-gitlab-cicd) - You've configured the [Prometheus service](../../project/integrations/prometheus.md#enabling-prometheus-integration) diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 5cb4f16e0d8..b691a97fc32 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -4,11 +4,7 @@ type: reference, howto # Standalone Vulnerability pages -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. - -CAUTION: **Warning:** -This feature is currently [Alpha](https://about.gitlab.com/handbook/product/#alpha-beta-ga). -You can begin using it, but it may receive important changes in the future. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13561) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.0. Each security vulnerability in the [Vulnerability List](../dependency_list/index.md) has its own standalone page. |
