diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-05-19 15:44:42 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-05-19 15:44:42 +0000 |
| commit | 4555e1b21c365ed8303ffb7a3325d773c9b8bf31 (patch) | |
| tree | 5423a1c7516cffe36384133ade12572cf709398d /doc/user/application_security | |
| parent | e570267f2f6b326480d284e0164a6464ba4081bc (diff) | |
| download | gitlab-ce-4555e1b21c365ed8303ffb7a3325d773c9b8bf31.tar.gz | |
Add latest changes from gitlab-org/gitlab@13-12-stable-eev13.12.0-rc42
Diffstat (limited to 'doc/user/application_security')
33 files changed, 2580 insertions, 619 deletions
diff --git a/doc/user/application_security/api_fuzzing/create_har_files.md b/doc/user/application_security/api_fuzzing/create_har_files.md new file mode 100644 index 00000000000..220d00adc7b --- /dev/null +++ b/doc/user/application_security/api_fuzzing/create_har_files.md @@ -0,0 +1,234 @@ +--- +stage: Secure +group: Fuzz Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: howto +--- + +# HTTP Archive format + +HTTP Archive (HAR) format files are an industry standard for exchanging information about HTTP +requests and HTTP responses. A HAR file's content is JSON formatted, containing browser interactions +with a web site. The file extension `.har` is commonly used. + +The HAR files can be used to perform [web API Fuzz Testing](index.md#http-archive-har) as part of +your [GitLab CI/CD](../../../ci/README.md) pipelines. + +WARNING: +**DANGER** A HAR file stores information exchanged between web client and web server. It could also +store sensitive information such as authentication tokens, API keys, and session cookies. We +recommend that you review the HAR file contents before adding them to a repository. + +## HAR file creation + +You can create HAR files manually or by using a specialized tool for recording web sessions. We +recommend using a specialized tool. However, it is important to make sure files created by these +tools do not expose sensitive information, and can be safely used. + +The following tools can be used generate a HAR file based on your network activity. They +automatically record your network activity and generate the HAR file: + +1. [GitLab HAR Recorder](#gitlab-har-recorder). +1. [Insomnia API Client](#insomnia-api-client). +1. [Fiddler debugging proxy](#fiddler-debugging-proxy). +1. [Safari web browser](#safari-web-browser). +1. [Chrome web browser](#chrome-web-browser). +1. [Firefox web browser](#firefox-web-browser). + +WARNING: +**DANGER** HAR files may contain sensitive information such as authentication tokens, API keys, and +session cookies. We recommend that you review the HAR file contents before adding them to a +repository. + +### GitLab HAR Recorder + +[GitLab HAR Recorder](https://gitlab.com/gitlab-org/security-products/har-recorder) is a command +line tool for recording HTTP messages and saving them to HTTP Archive (HAR) files. For more details +about the GitLab HAR Recorder, see the [homepage](https://gitlab.com/gitlab-org/security-products/har-recorder). + +#### Install GitLab HAR Recorder + +Prerequisites: + +- Install Python 3.6 or greater. +- For Microsoft Windows, you must also install `Microsoft Visual C++ 14.0`. It's included with + *Build Tools for Visual Studio* from [Visual Studio Downloads page](https://visualstudio.microsoft.com/downloads/). +- Install HAR Recorder. + +Install GitLab HAR Recorder: + + ```shell + pip install gitlab-har-recorder --extra-index-url https://gitlab.com/api/v4/projects/22441624/packages/pypi/simple + ``` + +#### Create a HAR file with GitLab HAR Recorder + +1. Start recorder with the proxy port and HAR filename. +1. Complete the browser actions, using the proxy. + 1. Make sure proxy is used! +1. Stop the recorder. + +To verify the HAR contains all requests, use the [HAR Viewer (online)](http://www.softwareishard.com/har/viewer/). +[Google Admin Toolbox HAR Analyzer](https://toolbox.googleapps.com/apps/har_analyzer/) + +### Insomnia API Client + +[Insomnia API Client](https://insomnia.rest/) is an API design tool that among many uses, helps +you to design, describe, and test your API. You can also use it to generate HAR files that can be +used in [Web API Fuzz Testing](index.md#http-archive-har). + +#### Create a HAR file with the Insomnia API Client + +1. Define or import your API. + - Postman v2. + - Curl. + - OpenAPI v2, v3. +1. Verify each API call works. + - If you imported an OpenAPI specification, go through and add working data. +1. Select **API > Import/Export**. +1. Select **Export Data > Current Workspace**. +1. Select requests to include in the HAR file. +1. Select **Export**. +1. In the **Select Export Type** dropdown select **HAR -- HTTP Archive Format**. +1. Select **Done**. +1. Enter a location and filename for the HAR file. + +### Fiddler debugging proxy + +[Fiddler](https://www.telerik.com/fiddler) is a web debugger tool. It captures HTTP and HTTP(S) +network traffic and allows you to examine each request. It also lets you export the requests and +responses in HAR format. + +#### Create a HAR file with Fiddler + +1. Go to the [Fiddler home page](https://www.telerik.com/fiddler) and sign in. If you don't already +have an account, first create an account. +1. Browse pages that call an API. Fiddler automatically captures the requests. +1. Select one or more requests, then from the context menu, select **Export > Selected Sessions**. +1. In the **Choose Format** dropdown select **HTTPArchive v1.2**. +1. Enter a filename and select **Save**. + +Fiddler shows a popup message confirming the export has succeeded. + +### Safari web browser + +[Safari](https://www.apple.com/safari/) is a web browser maintained by Apple. As web development +evolves, browsers support new capabilities. With Safari you can explore network traffic and +export it as a HAR file. + +#### Create a HAR file with Safari + +Prerequisites: + +- Enable the `Develop` menu item. + 1. Open Safari's preferences. Press <kbd>Command</kbd>+<kbd>,</kbd> or from the menu, select + **Safari > Preferences...**. + 1. Select **Advanced** tab, then select `Show Develop menu item in menu bar`. + 1. Close the **Preferences** window. + +1. Open the **Web Inspector**. Press <kbd>Option</kbd>+<kbd>Command</kbd>+<kbd>i</kbd>, or from the + menu, select **Develop > Show Web Inspector**. +1. Select the **Network** tab, and select **Preserve Log**. +1. Browse pages that call the API. +1. Open the **Web Inspector** and select the **Network** tab +1. Right-click on the request to export and select **Export HAR**. +1. Enter a filename and select **Save**. + +### Chrome web browser + +[Chrome](https://www.google.com/chrome/) is a web browser maintained by Google. As web development +evolves, browsers support new capabilities. With Chrome you can explore network traffic and +export it as a HAR file. + +#### Create a HAR file with Chrome + +1. From the Chrome context menu, select **Inspect**. +1. Select the **Network** tab. +1. Select **Preserve log**. +1. Browse pages that call the API. +1. Select one or more requests. +1. Right click and select **Save all as HAR with content**. +1. Enter a filename and select **Save**. +1. To append additional requests, select and save them to the same file. + +### Firefox Web Browser + +[Firefox](https://www.mozilla.org/en-US/firefox/new/) is a web browser maintained by Mozilla. As web +development evolves, browsers support new capabilities. With Firefox you can explore network traffic +and export it as a HAR file. + +#### Create a HAR file with Firefox + +1. From the Firefox context menu, select **Inspect**. +1. Select the **Network** tab. +1. Browse pages that call the API. +1. Check the **Network** tab and confirm requests are being recorded. If there is a message + `Perform a request or Reload the page to see detailed information about network activity`, + select **Reload** to start recording requests. +1. Select one or more requests. +1. Right click and select **Save All As HAR**. +1. Enter a filename and select **Save**. +1. To append additional requests, select and save them to the same file. + +## HAR verification + +Before using HAR files it's important to make sure they don't expose any sensitive information. + +For each HAR file you should: + +- View the HAR file's content +- Review the HAR file for sensitive information +- Edit or remove sensitive information + +### View HAR file contents + +We recommend viewing a HAR file's content in a tool that can present its content in a structured +way. Several HAR file viewers are available online. If you would prefer not to upload the HAR file, +you can use a tool installed on your computer. HAR files used JSON format, so can also be viewed in +a text editor. + +Tools recommended for viewing HAR files include: + +- [HAR Viewer](http://www.softwareishard.com/har/viewer/) - (online) +- [Google Admin Toolbox HAR Analyzer](https://toolbox.googleapps.com/apps/har_analyzer/) - (online) +- [Fiddler](https://www.telerik.com/fiddler) - local +- [Insomnia API Client](https://insomnia.rest/) - local + +## Review HAR file content + +Review the HAR file for any of the following: + +- Information that could help to grant access to your application, for example: authentication + tokens, authentication tokens, cookies, API keys. +- [Personally Identifiable Information (PII)](https://en.wikipedia.org/wiki/Personal_data). + +We strongly recommended that you [edit or remove it](#edit-or-remove-sensitive-information) any +sensitive information. + +Use the following as a checklist to start with. Note that it's not an exhaustive list. + +- Look for secrets. For example: if your application requires authentication, check common locations + or authentication information: + - Authentication related headers. For example: cookies, authorization. These headers could contain + valid information. + - A request related to authentication. The body of these requests might contain information such + as user credentials or tokens. + - Session tokens. Session tokens could grant access to your application. The location of these + token could vary. They could be in headers, query parameters or body. +- Look for Personally Identifiable Information + - For example, if your application retrieves a list of users and their personal data: phones, + names, emails. + - Authentication information might also contain personal information. + +## Edit or remove sensitive information + +Edit or remove sensitive information found during the [HAR file content review](#review-har-file-content). +HAR files are JSON files and can be edited in any text editor. + +After editing the HAR file, open it in a HAR file viewer to verify its formatting and structure are +intact. + +The following example demonstrates use of [Visual Studio Code](https://code.visualstudio.com/) text +editor to edit an Authorization token found in a header. + + diff --git a/doc/user/application_security/api_fuzzing/img/vscode_har_edit_auth_header.png b/doc/user/application_security/api_fuzzing/img/vscode_har_edit_auth_header.png Binary files differnew file mode 100644 index 00000000000..a6bc45e67d3 --- /dev/null +++ b/doc/user/application_security/api_fuzzing/img/vscode_har_edit_auth_header.png diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index dfb7e12a8be..8511c919c14 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -7,17 +7,51 @@ type: reference, howto # Web API Fuzz Testing **(ULTIMATE)** -You can add web API fuzzing to your [GitLab CI/CD](../../../ci/README.md) -pipelines. This helps you discover bugs and potential security issues that other QA processes may -miss. API fuzzing performs fuzz testing of API operation parameters. Fuzz testing sets operation +Web API fuzzing performs fuzz testing of API operation parameters. Fuzz testing sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API -backend. +backend. This helps you discover bugs and potential security issues that other QA processes may +miss. We recommend that you use fuzz testing in addition to [GitLab Secure](../index.md)'s other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), you can run fuzz tests as part your CI/CD workflow. -## Requirements +## When Web API fuzzing runs + +Web API fuzzing runs in the `fuzz` stage of the CI/CD pipeline. To ensure API fuzzing scans the +latest code, your CI/CD pipeline should deploy changes to a test environment in one of the stages +preceding the `fuzz` stage. + +Note the following changes have been made to the API fuzzing template: + +- In GitLab 14.0 and later, you must define a `fuzz` stage in your `.gitlab-ci.yml` file. +- In GitLab 13.12 and earlier, the API fuzzing template defines `build`, `test`, `deploy`, and + `fuzz` stages. The `fuzz` stage runs last by default. The predefined stages were deprecated, and removed from the `API-Fuzzing.latest.gitlab-ci.yml` template. They will be removed in a future GitLab + version. + +If your pipeline is configured to deploy to the same web server on each run, running a +pipeline while another is still running could cause a race condition in which one pipeline +overwrites the code from another. The API to scan should be excluded from changes for the duration +of a fuzzing scan. The only changes to the API should be from the fuzzing scanner. Any changes made +to the API (for example, by users, scheduled tasks, database changes, code changes, other pipelines, +or other scanners) during a scan could cause inaccurate results. + +You can run a Web API fuzzing scan using the following methods: + +- [OpenAPI Specification](#openapi-specification) - version 2.0 or 3.0 +- [HTTP Archive](#http-archive-har) (HAR) +- [Postman Collection](#postman-collection) - version 2.0 or 2.1 + +Example projects using these methods are available: + +- [Example OpenAPI v2 Specification project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/openapi) +- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har) +- [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/postman-api-fuzzing-example) +- [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/graphql-api-fuzzing-example) + +## Enable Web API fuzzing + +Requirements: - One of the following web API types: - REST API @@ -29,49 +63,26 @@ you can run fuzz tests as part your CI/CD workflow. - HTTP Archive (HAR) of API requests to test - Postman Collection v2.0 or v2.1 -## When fuzzing scans run - -When using the `API-Fuzzing.gitlab-ci.yml` template, the `fuzz` job runs last, as shown here. To -ensure API fuzzing scans the latest code, your CI pipeline should deploy changes to a test -environment in one of the jobs preceding the `fuzz` job: - -```yaml -stages: - - build - - test - - deploy - - fuzz -``` - -Note that if your pipeline is configured to deploy to the same web server on each run, running a -pipeline while another is still running could cause a race condition in which one pipeline -overwrites the code from another. The API to scan should be excluded from changes for the duration -of a fuzzing scan. The only changes to the API should be from the fuzzing scanner. Be aware that -any changes made to the API (for example, by users, scheduled tasks, database changes, code -changes, other pipelines, or other scanners) during a scan could cause inaccurate results. - -## Configuration + WARNING: + **NEVER** run fuzz testing against a production server. Not only can it perform *any* function that + the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting + data. Only run fuzzing against a test server. -There are three ways to perform scans. See the configuration section for the one you wish to use: +To enable Web API fuzzing: -- [OpenAPI v2 or v3 specification](#openapi-specification) -- [HTTP Archive (HAR)](#http-archive-har) -- [Postman Collection v2.0 or v2.1](#postman-collection) +- Include the API fuzzing template in your `.gitlab-ci.yml` file. +- From GitLab 13.10 and later, use the Web API fuzzing configuration form. -Examples of both configurations can be found here: +- For manual configuration instructions, see the respective section, depending on the API type: + - [OpenAPI Specification](#openapi-specification) + - [HTTP Archive (HAR)](#http-archive-har) + - [Postman Collection](#postman-collection) +- Otherwise, see [Web API fuzzing configuration form](#web-api-fuzzing-configuration-form). -- [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/openapi) -- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har) -- [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/postman-api-fuzzing-example) -- [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/graphql-api-fuzzing-example) - -WARNING: -GitLab 14.0 will require that you place API fuzzing configuration files (for example, -`gitlab-api-fuzzing-config.yml`) in your repository's `.gitlab` directory instead of your -repository's root. You can continue using your existing configuration files as they are, but -starting in GitLab 14.0, GitLab will not check your repository's root for configuration files. +In GitLab 14.0 and later, API fuzzing configuration files must be in your repository's +`.gitlab` directory instead of your repository's root. -### Configuration form +### Web API fuzzing configuration form > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299234) in GitLab 13.10. @@ -82,6 +93,8 @@ The API fuzzing configuration form helps you create or modify your project's API configuration. The form lets you choose values for the most common API fuzzing options and builds a YAML snippet that you can paste in your GitLab CI/CD configuration. +#### Configure Web API fuzzing with the configuration form + To generate an API Fuzzing configuration snippet: 1. From your project's home page, go to **Security & Compliance > Configuration** in the left @@ -89,25 +102,23 @@ To generate an API Fuzzing configuration snippet: 1. Select **Configure** in the **API Fuzzing** row. 1. Complete the form as needed. Read below for more information on available configuration options. 1. Select **Generate code snippet**. - -A modal opens with the YAML snippet corresponding to the options you've selected in the form. - - - -Select **Copy code and open `.gitlab-ci.yml` file** to copy the snippet to your clipboard and be redirected -to your project's `.gitlab-ci.yml` file where you can paste the YAML configuration. - -Select **Copy code only** to copy the snippet to your clipboard and close the modal. + A modal opens with the YAML snippet corresponding to the options you've selected in the form. +1. Choose one of the following actions: + 1. Select **Copy code and open `.gitlab-ci.yml` file** to copy the snippet to your clipboard and + be redirected to your project's `.gitlab-ci.yml` file where you can paste the YAML + configuration. + 1. Select **Copy code only** to copy the snippet to your clipboard and close the modal. ### OpenAPI Specification -> Support for OpenAPI Specification v3 was +> Support for OpenAPI Specification using YAML format was +> [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0. +> Support for OpenAPI Specification v3.0 was > [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228652) in GitLab 13.9. -The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an -API description format for REST APIs. This section shows you how to configure API fuzzing by using -an OpenAPI specification to provide information about the target API to test. OpenAPI specifications -are provided as a file system resource or URL. +The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs. +This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test. +OpenAPI Specifications are provided as a file system resource or URL. Both JSON and YAML OpenAPI formats are supported. API fuzzing uses an OpenAPI document to generate the request body. When a request body is required, the body generation is limited to these body types: @@ -116,59 +127,42 @@ the body generation is limited to these body types: - `multipart/form-data` - `application/json` -Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: +#### Configure Web API fuzzing with an OpenAPI Specification -1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) - the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) - that's provided as part of your GitLab installation. To do so, add the following to your - `.gitlab-ci.yml` file: - - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - ``` +To configure API fuzzing in GitLab with an OpenAPI Specification: -1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. +1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. The [configuration file](#configuration-files) has several testing profiles defined with varying - amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this - profile completes quickly, allowing for easier configuration validation. +1. [Include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + in your `.gitlab-ci.yml` file. - Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, - substituting `Quick-10` for the profile you choose: +1. Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file. + The profile specifies how many tests are run. Substitute `Quick-10` for the profile you choose. For more details, see [API fuzzing profiles](#api-fuzzing-profiles). ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - variables: FUZZAPI_PROFILE: Quick-10 ``` -1. Provide the location of the OpenAPI specification. You can provide the specification as a file - or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable: +1. Provide the location of the OpenAPI Specification. You can provide the specification as a file + or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable. - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - - variables: - FUZZAPI_PROFILE: Quick-10 - FUZZAPI_OPENAPI: test-api-specification.json - ``` - -1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` - variable or an `environment_url.txt` file. +1. Provide the target API instance's base URL. Use either the `FUZZAPI_TARGET_URL` variable or an + `environment_url.txt` file. Adding the URL in an `environment_url.txt` file at your project's root is great for testing in - dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD - pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing - automatically parses that file to find its scan target. You can see an - [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + dynamic environments. To run API fuzzing against an application dynamically created during a + GitLab CI/CD pipeline, have the application persist its URL in an `environment_url.txt` file. + API fuzzing automatically parses that file to find its scan target. You can see an + example of this in the [Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). - Here's an example of using `FUZZAPI_TARGET_URL`: +Example `.gitlab-ci.yml` file using an OpenAPI Specification: ```yaml + stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -184,10 +178,7 @@ This is a minimal configuration for API Fuzzing. From here you can: - [Add authentication](#authentication). - Learn how to [handle false positives](#handling-false-positives). -WARNING: -**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that -the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting -data. Only run fuzzing against a test server. +For details of API fuzzing configuration options, see [Available CI/CD variables](#available-cicd-variables). ### HTTP Archive (HAR) @@ -196,59 +187,33 @@ is an archive file format for logging HTTP transactions. When used with the GitL must contain records of calling the web API to test. The API fuzzer extracts all the requests and uses them to perform testing. -You can use various tools to generate HAR files: - -- [Fiddler](https://www.telerik.com/fiddler): Web debugging proxy -- [Insomnia Core](https://insomnia.rest/): API client -- [Chrome](https://www.google.com/chrome/): Browser -- [Firefox](https://www.mozilla.org/en-US/firefox/): Browser -- [GitLab HAR Recorder](https://gitlab.com/gitlab-org/security-products/har-recorder): Command line +For more details, including how to create a HAR file, see [HTTP Archive format](create_har_files.md). WARNING: HAR files may contain sensitive information such as authentication tokens, API keys, and session cookies. We recommend that you review the HAR file contents before adding them to a repository. -Follow these steps to configure API fuzzing to use a HAR file that provides information about the -target API to test: +#### Configure Web API fuzzing with a HAR file -1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) - the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) - that's provided as part of your GitLab installation. To do so, add the following to your - `.gitlab-ci.yml` file: +To configure API fuzzing to use a HAR file: - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - ``` +1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. - -1. The [configuration file](#configuration-files) has several testing profiles defined with varying - amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this - profile completes quickly, allowing for easier configuration validation. +1. [Include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + in your `.gitlab-ci.yml` file. - Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, - substituting `Quick-10` for the profile you choose: +1. Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file. + The profile specifies how many tests are run. Substitute `Quick-10` for the profile you choose. For more details, see [API fuzzing profiles](#api-fuzzing-profiles). ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - variables: FUZZAPI_PROFILE: Quick-10 ``` 1. Provide the location of the HAR specification. You can provide the specification as a file - or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `FUZZAPI_HAR` variable: - - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - - variables: - FUZZAPI_PROFILE: Quick-10 - FUZZAPI_HAR: test-api-recording.har - ``` + or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) + in GitLab 13.10 and later. Specify the location by adding the `FUZZAPI_HAR` variable. 1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` variable or an `environment_url.txt` file. @@ -259,9 +224,12 @@ target API to test: automatically parses that file to find its scan target. You can see an [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). - Here's an example of using `FUZZAPI_TARGET_URL`: +Example `.gitlab-ci.yml` file using a HAR file: ```yaml + stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -271,16 +239,13 @@ target API to test: FUZZAPI_TARGET_URL: http://test-deployment/ ``` -This is a minimal configuration for API Fuzzing. From here you can: +This is a minimal configuration for API fuzzing. From here you can: - [Run your first scan](#running-your-first-scan). - [Add authentication](#authentication). - Learn how to [handle false positives](#handling-false-positives). -WARNING: -**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that -the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting -data. Only run fuzzing against a test server. +For details of API fuzzing configuration options, see [Available CI/CD variables](#available-cicd-variables). ### Postman Collection @@ -299,51 +264,31 @@ Postman Collection files may contain sensitive information such as authenticatio and session cookies. We recommend that you review the Postman Collection file contents before adding them to a repository. -Follow these steps to configure API fuzzing to use a Postman Collection file that provides -information about the target API to test: +#### Configure Web API fuzzing with a Postman Collection file -1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) - the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) - that's provided as part of your GitLab installation. To do so, add the following to your - `.gitlab-ci.yml` file: +To configure API fuzzing to use a Postman Collection file: - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - ``` +1. Add the `fuzz` stage to your `.gitlab-ci.yml` file. -1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) - to your repository's root as `.gitlab-api-fuzzing.yml`. - -1. The [configuration file](#configuration-files) has several testing profiles defined with varying - amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this - profile completes quickly, allowing for easier configuration validation. +1. [Include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + in your `.gitlab-ci.yml` file. - Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, - substituting `Quick-10` for the profile you choose: +1. Provide the profile by adding the `FUZZAPI_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file. + The profile specifies how many tests are run. Substitute `Quick-10` for the profile you choose. For more details, see [API fuzzing profiles](#api-fuzzing-profiles). ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - variables: FUZZAPI_PROFILE: Quick-10 ``` -1. Provide the location of the Postman Collection specification. You can provide the specification as a file - or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `FUZZAPI_POSTMAN_COLLECTION` variable: - - ```yaml - include: - - template: API-Fuzzing.gitlab-ci.yml - - variables: - FUZZAPI_PROFILE: Quick-10 - FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json - ``` +1. Provide the location of the Postman Collection specification. You can provide the specification + as a file or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) + in GitLab 13.10 and later. Specify the location by adding the `FUZZAPI_POSTMAN_COLLECTION` + variable. -1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` - variable or an `environment_url.txt` file. +1. Provide the target API instance's base URL. Use either the `FUZZAPI_TARGET_URL` variable or an + `environment_url.txt` file. Adding the URL in an `environment_url.txt` file at your project's root is great for testing in dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD @@ -351,9 +296,12 @@ information about the target API to test: automatically parses that file to find its scan target. You can see an [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). - Here's an example of using `FUZZAPI_TARGET_URL`: +Example `.gitlab-ci.yml` file using a Postman Collection file: ```yaml + stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -369,15 +317,13 @@ This is a minimal configuration for API Fuzzing. From here you can: - [Add authentication](#authentication). - Learn how to [handle false positives](#handling-false-positives). -WARNING: -**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that -the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting -data. Only run fuzzing against a test server. +For details of API fuzzing configuration options, see [Available CI/CD variables](#available-cicd-variables). #### Postman variables Postman allows the developer to define placeholders that can be used in different parts of the -requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/). +requests. These placeholders are called variables, as explained in the Postman documentation, +[Using variables](https://learning.postman.com/docs/sending-requests/variables/). You can use variables to store and reuse values in your requests and scripts. For example, you can edit the collection to add variables to the document: @@ -398,7 +344,7 @@ Postman file. For example, Postman does not export environment-scoped variables file. By default, the API fuzzer uses the Postman file to resolve Postman variable values. If a JSON file -is set in a GitLab CI environment variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON +is set in a GitLab CI/CD variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON file takes precedence to get Postman variable values. Although Postman can export environment variables into a JSON file, the format is not compatible @@ -407,6 +353,9 @@ with the JSON expected by `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`. Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -428,6 +377,13 @@ values. For example: } ``` +## API fuzzing configuration + +The API fuzzing behavior can be changed through CI/CD variables. + +From GitLab 13.12 and later, the default API fuzzing configuration file is `.gitlab/gitlab-api-fuzzing-config.yml`. In GitLab 14.0 and later, API fuzzing configuration files must be in your repository's +`.gitlab` directory instead of your repository's root. + ### Authentication Authentication is handled by providing the authentication token as a header or cookie. You can @@ -449,6 +405,9 @@ GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use as the value for `FUZZAPI_HTTP_PASSWORD`: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -487,6 +446,9 @@ Follow these steps to provide the bearer token with `FUZZAPI_OVERRIDES_ENV`: 1. In your `.gitlab-ci.yml` file, set `FUZZAPI_OVERRIDES_ENV` to the variable you just created: ```yaml + stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -522,6 +484,9 @@ This file can be generated by a prior stage and provided to API fuzzing through Set `FUZZAPI_OVERRIDES_FILE` in your `.gitlab-ci.yml` file: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -561,6 +526,9 @@ You must provide three CI/CD variables, each set for correct operation: For example: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -576,16 +544,15 @@ variables: To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and the test API's application logs. -### Configuration files +### API fuzzing profiles -To get you started quickly, GitLab provides the configuration file +GitLab provides the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml). -This file has several testing profiles that perform various numbers of tests. The run time of each -profile increases as the test numbers go up. To use a configuration file, add it to your -repository's root as `.gitlab-api-fuzzing.yml`. +It contains several testing profiles that perform a specific numbers of tests. The runtime of each +profile increases as the number of tests increases. -| Profile | Fuzz Tests (per parameter) | -|:---------|:-----------| +| Profile | Fuzz Tests (per parameter) | +|:----------|:---------------------------| | Quick-10 | 10 | | Medium-20 | 20 | | Medium-50 | 50 | @@ -593,35 +560,23 @@ repository's root as `.gitlab-api-fuzzing.yml`. ### Available CI/CD variables -| CI/CD variable | Description | -|------------------------------------------------------|--------------------| -| `FUZZAPI_VERSION` | Specify API Fuzzing container version. Defaults to `latest`. | -| `FUZZAPI_TARGET_URL` | Base URL of API testing target. | -|[`FUZZAPI_CONFIG`](#configuration-files) | API Fuzzing configuration file. Defaults to `.gitlab-apifuzzer.yml`. | -|[`FUZZAPI_PROFILE`](#configuration-files) | Configuration profile to use during testing. Defaults to `Quick`. | -| `FUZZAPI_REPORT` | Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. | -|[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. | -|[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | -|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | -|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. | -|[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | -|[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | -|[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | -|[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | -|[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | -|[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | - -<!--|[`FUZZAPI_D_TARGET_IMAGE`](#target-container) |API target docker image | -|[`FUZZAPI_D_TARGET_ENV`](#target-container) |Docker environment options | -|[`FUZZAPI_D_TARGET_VOLUME`](#target-container) | Docker volume options | -|[`FUZZAPI_D_TARGET_PORTS`](#target-container) |Docker port options | -| `FUZZAPI_D_WORKER_IMAGE` |Custom worker docker image | -| `FUZZAPI_D_WORKER_ENV` |Custom worker docker environment options | -| `FUZZAPI_D_WORKER_VOLUME` |Custom worker docker volume options | -| `FUZZAPI_D_WORKER_PORTS` |Custom worker docker port options | -| `FUZZAPI_D_NETWORK` |Name of docker network, defaults to "testing-net"| -| `FUZZAPI_D_PRE_SCRIPT` |Pre script runs after docker login and docker network create, but before starting the scanning image container.| -| `FUZZAPI_D_POST_SCRIPT` |Post script runs after scanning image container is started. This is the place to start your target(s) and kick off scanning when using an advanced configuration.| --> +| CI/CD variable | Description | +|-------------------------------------------------------------|-------------| +| `SECURE_ANALYZERS_PREFIX` | Specify the Docker registry base address from which to download the analyzer. | +| `FUZZAPI_VERSION` | Specify API Fuzzing container version. Defaults to `latest`. | +| `FUZZAPI_TARGET_URL` | Base URL of API testing target. | +| `FUZZAPI_CONFIG` | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/276395) in GitLab 13.12, replaced with default `.gitlab/gitlab-api-fuzzing-config.yml`. API Fuzzing configuration file. | +|[`FUZZAPI_PROFILE`](#api-fuzzing-profiles) | Configuration profile to use during testing. Defaults to `Quick-10`. | +|[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI Specification file or URL. | +|[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | +|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | +|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract Postman variable values. | +|[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | +|[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | +|[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | +|[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | +|[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | +|[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | ### Overrides @@ -788,6 +743,9 @@ To provide the overrides JSON as a file, the `FUZZAPI_OVERRIDES_FILE` CI/CD vari Here's an example `.gitlab-ci.yml`: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -806,6 +764,9 @@ This allows you to place the JSON as variables that can be masked and protected. In this example `.gitlab-ci.yml`, the `FUZZAPI_OVERRIDES_ENV` variable is set directly to the JSON: ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -820,6 +781,9 @@ In this example `.gitlab-ci.yml`, the `SECRET_OVERRIDES` variable provides the J [group or instance level CI/CD variable defined in the UI](../../../ci/variables/README.md#instance-cicd-variables): ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -845,6 +809,9 @@ You must provide three CI/CD variables, each set for correct operation: - `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. ```yaml +stages: + - fuzz + include: - template: API-Fuzzing.gitlab-ci.yml @@ -1009,7 +976,7 @@ pipelines. For more information, see the [Security Dashboard documentation](../s Fuzzing faults show up as vulnerabilities with a severity of Unknown. Once a fault is found, you can interact with it. Read more on how to -[address the vulnerabilities](../index.md#addressing-vulnerabilities). +[address the vulnerabilities](../vulnerabilities/index.md). ## Handling False Positives @@ -1026,7 +993,7 @@ False positives can be handled in two ways: ### Turn off a Check Checks perform testing of a specific type and can be turned on and off for specific configuration -profiles. The provided [configuration files](#configuration-files) define several profiles that you +profiles. The default configuration file defines several profiles that you can use. The profile definition in the configuration file lists all the checks that are active during a scan. To turn off a specific check, remove it from the profile definition in the configuration file. The profiles are defined in the `Profiles` section of the configuration file. @@ -1143,13 +1110,19 @@ Profiles: UnicodeFuzzing: true ``` +## Running API fuzzing in an offline environment + +For self-managed GitLab instances in an environment with limited, restricted, or intermittent access +to external resources through the internet, some adjustments are required for the Web API Fuzz testing job to +successfully run. For more information, see [Offline environments](../offline_deployments/index.md). + ## Troubleshooting ### Error, the OpenAPI document is not valid. Errors were found during validation of the document using the published OpenAPI schema -At the start of an API Fuzzing job the OpenAPI specification is validated against the [published schema](https://github.com/OAI/OpenAPI-Specification/tree/master/schemas). This error is shown when the provided OpenAPI specification has validation errors. Errors can be introduced when creating an OpenAPI specification manually, and also when the schema is generated. +At the start of an API Fuzzing job the OpenAPI Specification is validated against the [published schema](https://github.com/OAI/OpenAPI-Specification/tree/master/schemas). This error is shown when the provided OpenAPI Specification has validation errors. Errors can be introduced when creating an OpenAPI Specification manually, and also when the schema is generated. -For OpenAPI specifications that are generated automatically validation errors are often the result of missing code annotations. +For OpenAPI Specifications that are generated automatically validation errors are often the result of missing code annotations. **Error message** @@ -1157,9 +1130,9 @@ For OpenAPI specifications that are generated automatically validation errors ar - `OpenAPI 2.0 schema validation error ...` - `OpenAPI 3.0.x schema validation error ...` -**Solution** +**Solution** -**For generated OpenAPI specifications** +**For generated OpenAPI Specifications** 1. Identify the validation errors. 1. Use the [Swagger Editor](https://editor.swagger.io/) to identify validation problems in your specification. The visual nature of the Swagger Editor makes it easier to understand what needs to change. @@ -1170,7 +1143,7 @@ For OpenAPI specifications that are generated automatically validation errors ar **For manually created OpenAPI Specifications** 1. Identify the validation errors. - 1. The simplest solution is to use a visual tool to edit and validate the OpenAPI document. For example the [Swagger Editor](https://editor.swagger.io/) will point out schema errors and possible solutions. + 1. The simplest solution is to use a visual tool to edit and validate the OpenAPI document. For example the [Swagger Editor](https://editor.swagger.io/) highlights schema errors and possible solutions. 1. Alternatively, you can check the log output and look for schema validation warnings. They are prefixed with messages such as `OpenAPI 2.0 schema validation error` or `OpenAPI 3.0.x schema validation error`. Each failed validation provides extra information about `location` and `description`. Correct each of the validation failures and then resubmit the OpenAPI doc. Note that JSON Schema validation message might not be easy to understand. This is why we recommend the use of editors to validate document. 1. Once the validation issues are resolved, re-run your pipeline. @@ -1183,11 +1156,61 @@ The API Fuzzing engine outputs an error message when it cannot establish a conne - In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/323939), `Failed to start scanner session (version header not found).` - In GitLab 13.10 and earlier, `API Security version header not found. Are you sure that you are connecting to the API Security server?`. -**Solution** +**Solution** - Remove the `FUZZAPI_API` variable from the `.gitlab-ci.yml` file. The value will be inherited from the API Fuzzing CI/CD template. We recommend this method instead of manually setting a value. - If removing the variable is not possible, check to see if this value has changed in the latest version of the [API Fuzzing CI/CD template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml). If so, update the value in the `.gitlab-ci.yml` file. +### Application cannot determine the base URL for the target API + +The API Fuzzing analyzer outputs an error message when it cannot determine the target API after inspecting the OpenAPI document. This error message is shown when the target API has not been set in the `.gitlab-ci.yml`file, it is not available in the `environment_url.txt` file, and it could not be computed using the OpenAPI document. + +There is an order of precedence in which the API Fuzzing analyzer tries to get the target API when checking the different sources. First, it will try to use the `FUZZAPI_TARGET_URL`. If the environment variable has not been set, then the API Fuzzing analyzer will attempt to use the `environment_url.txt` file. If there is no file `environment_url.txt`, the API Fuzzing analyzer will then use the OpenAPI document contents and the URL provided in `FUZZAPI_OPENAPI` (if a URL is provided) to try to compute the target API. + +The best-suited solution will depend on whether or not your target API changes for each deployment. In static environments, the target API is the same for each deployment, in this case please refer to the [static environment solution](#static-environment-solution). If the target API changes for each deployment a [dynamic environment solution](#dynamic-environment-solutions) should be applied. + +#### Static environment solution + +This solution is for pipelines in which the target API URL doesn't change (is static). + +**Add environmental variable** + +For environments where the target API remains the same, we recommend you specify the target URL by using the `FUZZAPI_TARGET_URL` environment variable. In your `.gitlab-ci.yml` file, add a variable `FUZZAPI_TARGET_URL`. The variable must be set to the base URL of API testing target. For example: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OPENAPI: test-api-specification.json +``` + +#### Dynamic environment solutions + +In a dynamic environment your target API changes for each different deployment. In this case, there is more than one possible solution, we recommend to use the `environment_url.txt` file when dealing with dynamic environments. + +**Use environment_url.txt** + +To support dynamic environments in which the target API URL changes during each pipeline, API Fuzzing supports the use of an `environment_url.txt` file that contains the URL to use. This file is not checked into the repository, instead it's created during the pipeline by the job that deploys the test target and collected as an artifact that can be used by later jobs in the pipeline. The job that creates the `environment_url.txt` file must run before the API Fuzzing job. + +1. Modify the test target deployment job adding the base URL in an `environment_url.txt` file at the root of your project. +1. Modify the test target deployment job collecting the `environment_url.txt` as an artifact. + +Example: + +```yaml +deploy-test-target: + script: + # Perform deployment steps + # Create environment_url.txt (example) + - echo http://${CI_PROJECT_ID}-${CI_ENVIRONMENT_SLUG}.example.org > environment_url.txt + + artifacts: + paths: + - environment_url.txt +``` + <!-- ### Target Container diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 4d5e3529762..8c34303ca00 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. **(ULTIMATE)** > - SAST configuration was [enabled](https://gitlab.com/groups/gitlab-org/-/epics/3659) in 13.3 and [improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in 13.4. **(ULTIMATE)** > - DAST Profiles feature was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40474) in 13.4. **(ULTIMATE)** -> - A simplified version was made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/294076) in GitLab 13.9. +> - A simplified version was made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/294076) in GitLab 13.10. WARNING: This feature might not be available to you. Check the **version history** note above for details. @@ -49,3 +49,6 @@ You can configure the following security controls: - Click either **Enable** or **Configure** to use SAST for the current project. For more details, see [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). - DAST Profiles - Click **Manage** to manage the available DAST profiles used for on-demand scans. For more details, see [DAST on-demand scans](../dast/index.md#on-demand-scans). +- Secret Detection + - Select **Configure via Merge Request** to create a merge request with the changes required to + enable Secret Detection. For more details, see [Enable Secret Detection via an automatic merge request](../secret_detection/index.md#enable-secret-detection-via-an-automatic-merge-request). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index fcf5c81db74..5ab56634d29 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -11,22 +11,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: GitLab 14.0 will replace its container scanning engine with Trivy. Currently, GitLab uses the open -source Clair engine for container scanning. GitLab 13.9 deprecates Clair. This is not a hard -breaking change, as customers who wish to continue to use Clair can do so by setting the -`CS_MAJOR_VERSION` CI/CD variable to version 3 (or earlier) in their `gitlab-ci.yaml` file. Since Clair is -deprecated, however, note that GitLab will no longer update or maintain that scanning engine -beginning in the 14.0 release. We advise customers to use the new default of Trivy beginning in -GitLab 14.0 for regular updates and the latest features. +source Clair engine for container scanning. GitLab 13.9 deprecates Clair. Until GitLab 14.0, this is +not a hard breaking change. Beginning in GitLab 14.0, GitLab will no longer update or maintain +Clair. To ensure that you get regular updates and the latest features, you must use the Trivy +container scanning engine beginning in GitLab 14.0. See the following sections for instructions on +moving from Clair to Trivy. Your application's Docker image may itself be based on Docker images that contain known vulnerabilities. By including an extra job in your pipeline that scans for those vulnerabilities and displays them in a merge request, you can use GitLab to audit your Docker-based apps. -By default, container scanning in GitLab is based on [Clair](https://github.com/quay/clair) and -[Klar](https://github.com/optiopay/klar), which are open-source tools for vulnerability static analysis in -containers. The GitLab [Klar analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar/) -scans the containers and serves as a wrapper for Clair. -To integrate security scanners other than Clair and Klar into GitLab, see +GitLab provides integration with two different open-source tools for vulnerability static analysis +in containers: + +- [Clair](https://github.com/quay/claircore) +- [Trivy](https://github.com/aquasecurity/trivy) + +To integrate GitLab with security scanners other than those listed here, see [Security scanner integration](../../../development/integrations/secure.md). You can enable container scanning by doing one of the following: @@ -52,7 +53,13 @@ To enable container scanning in your pipeline, you need the following: or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. - Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the shared runners on GitLab.com, then this is already the case. -- An image matching [Clair's list of supported distributions](https://quay.github.io/claircore/). +- An image matching the following supported distributions (depending on the analyzer being used): + + | Scanning Engine | Supported distributions | + | --- | --- | + | [Clair](https://github.com/quay/claircore) | [Supported operating systems and languages](https://quay.github.io/claircore/) | + | [Trivy](https://github.com/aquasecurity/trivy) | Supported [operating systems](https://aquasecurity.github.io/trivy/latest/vuln-detection/os/) and [languages](https://aquasecurity.github.io/trivy/latest/vuln-detection/library/) | + - [Build and push](../../packages/container_registry/index.md#build-and-push-by-using-gitlab-cicd) your Docker image to your project's container registry. The name of the Docker image should use the following [predefined CI/CD variables](../../../ci/variables/predefined_variables.md): @@ -86,7 +93,19 @@ How you enable container scanning depends on your GitLab version: that comes with your GitLab installation. - GitLab versions earlier than 11.9: Copy and use the job from the [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml). -- GitLab 13.6 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263482) better support for [FIPS](https://csrc.nist.gov/publications/detail/fips/140/2/final) by upgrading the `CS_MAJOR_VERSION` from `2` to `3`. +- GitLab 13.6 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263482) better support for + [FIPS](https://csrc.nist.gov/publications/detail/fips/140/2/final) by upgrading the + `CS_MAJOR_VERSION` from `2` to `3`. Version `3` of the `container_scanning` Docker image uses + [`centos:centos8`](https://hub.docker.com/_/centos) + as the new base. It also removes the use of the [start.sh](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/merge_requests/77) + script and instead executes the analyzer by default. Any customizations made to the + `container_scanning` job's [`before_script`](../../../ci/yaml/README.md#before_script) + and [`after_script`](../../../ci/yaml/README.md#after_script) + blocks may not work with the new version. To roll back to the previous [`alpine:3.11.3`](https://hub.docker.com/_/alpine)-based + Docker image, you can specify the major version through the [`CS_MAJOR_VERSION`](#available-variables) + variable. +- GitLab 13.9 [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322656) integration with + [Trivy](https://github.com/aquasecurity/trivy) by upgrading `CS_MAJOR_VERSION` from `3` to `4`. To include the `Container-Scanning.gitlab-ci.yml` template (GitLab 11.9 and later), add the following to your `.gitlab-ci.yml` file: @@ -138,31 +157,34 @@ include: ### Customizing the container scanning settings There may be cases where you want to customize how GitLab scans your containers. For example, you -may want to enable more verbose output from Clair or Klar, access a Docker registry that requires +may want to enable more verbose output, access a Docker registry that requires authentication, and more. To change such settings, use the [`variables`](../../../ci/yaml/README.md#variables) parameter in your `.gitlab-ci.yml` to set [CI/CD variables](#available-variables). The variables you set in your `.gitlab-ci.yml` overwrite those in `Container-Scanning.gitlab-ci.yml`. This example [includes](../../../ci/yaml/README.md#include) the container scanning template and -enables verbose output from Clair by setting the `CLAIR_OUTPUT` variable to `High`: +enables verbose output for both analyzers: + +Clair: ```yaml include: - template: Container-Scanning.gitlab-ci.yml variables: - CLAIR_OUTPUT: High + CLAIR_TRACE: true ``` -Version `3` of the `container_scanning` Docker image uses [`centos:centos8`](https://hub.docker.com/_/centos) -as the new base. It also removes the use of the [start.sh](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/merge_requests/77) -script and instead executes the analyzer by default. Any customizations made to the -`container_scanning` job's [`before_script`](../../../ci/yaml/README.md#before_script) -and [`after_script`](../../../ci/yaml/README.md#after_script) -blocks may not work with the new version. To roll back to the previous [`alpine:3.11.3`](https://hub.docker.com/_/alpine)-based -Docker image, you can specify the major version through the [`CS_MAJOR_VERSION`](#available-variables) -variable. +Trivy: + +```yaml +include: + - template: Container-Scanning.gitlab-ci.yml + +variables: + TRIVY_DEBUG: true +``` This example [includes](../../../ci/yaml/README.md#include) the container scanning template and enables version `2` of the analyzer: @@ -181,57 +203,109 @@ variables: #### Available variables -You can [configure](#customizing-the-container-scanning-settings) container -scanning by using the following CI/CD variables: - -| CI/CD Variable | Default | Description | -| ------------------------------ | ------------- | ----------- | -| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | -| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | -| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | -| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | -| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | -| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the Clair server process. | -| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. | -| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | -| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | -| `CS_MAJOR_VERSION` | `3` | The major version of the Docker image tag. | -| `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | -| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | -| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | -| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | -| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | -| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from Klar. | -| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | -| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | -| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | +You can [configure](#customizing-the-container-scanning-settings) both analyzers by using the following CI/CD variables: + +| CI/CD Variable | Default | Description | Supported by| +| ------------------------------ | ------------- | ----------- | ------------ | +| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | Both | +| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerability definitions](https://hub.docker.com/r/arminc/clair-db) database. **Do not change this** unless you're running the image locally as described in [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool). The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | Clair | +| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerability definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version (for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerability database for an on-premise offline installation). | Clair | +| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerability definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version (for example, to provide a consistent set of vulnerabilities for integration testing purposes). | Clair | +| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are output. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical`, and `Defcon1`. | Clair | +| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the Clair server process. | Clair | +| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerability definitions](https://hub.docker.com/r/arminc/clair-db) is running on. **Do not change this** unless you're running the image locally as described in [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool). | Clair | +| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | Both | +| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | Both | +| `CS_ANALYZER_IMAGE` | `$SECURE_ANALYZERS_PREFIX/$CS_PROJECT:$CS_MAJOR_VERSION` | Docker image of the analyzer. | Both | +| `CS_MAJOR_VERSION` | `3` | The major version of the Docker image tag. | Both | +| `CS_PROJECT` | Depends on `$CS_MAJOR_VERSION`. `klar` if `$CS_MAJOR_VERSION` is set to `1`, `2` or `3`, and `container-scanning` otherwise. | Analyzer project to be used. | Both | +| `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | Both | +| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | Clair | +| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | Clair | +| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | Clair | +| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to use for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project. You should configure this variable only if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | Both | +| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from Klar. | Clair | +| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | Clair | +| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | Both | +| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | Both | +| `TRIVY_DEBUG` | `"false"` | Set to true to enable more verbose output from the Trivy process. | Trivy | ### Overriding the container scanning template If you want to override the job definition (for example, to change properties like `variables`), you -must declare a `container_scanning` job after the template inclusion, and then -specify any additional keys. For example: +must declare and override a job after the template inclusion, and then +specify any additional keys. + +This example sets `GIT_STRATEGY` to `fetch` to be considered by both Clair and Trivy: ```yaml include: - template: Container-Scanning.gitlab-ci.yml -container_scanning: +.cs_common: variables: GIT_STRATEGY: fetch ``` +This example sets `KLAR_TRACE` to `true`, which is specific to Clair: + +```yaml +include: + - template: Container-Scanning.gitlab-ci.yml + +container_scanning: + variables: + CLAIR_TRACE: true +``` + +This example sets `TRIVY_DEBUG` to `true`, which is specific to Trivy: + +```yaml +include: + - template: Container-Scanning.gitlab-ci.yml + +container_scanning_new: + variables: + TRIVY_DEBUG: true +``` + WARNING: -GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic). +GitLab 13.0 and later doesn't support [`only` and `except`](../../../ci/yaml/README.md#only--except). When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. +### Migrating from Clair to Trivy + +If you are currently using Clair and want to migrate to Trivy before GitLab 14.0, you can do so by +taking the following steps: + +1. Take the following actions in your CI file: + + - Set the variable `CS_MAJOR_VERSION` to `4`. The job scope is global variables, or under `.cs_common`. + - Remove the variable `CS_PROJECT` from your CI file. The job scope is `container_scanning_new`. + Setting this variable to `container-scanning` under the correct scope has the same effect as + removing it from your CI file. + - Remove the `CS_ANALYZER_IMAGE` variable from your CI file. The job scope is `.cs_common`. Note + that instead of overriding this variable, you can use `CS_MAJOR_VERSION`. + +1. Remove any variables that are only applicable to Clair. For a complete list of these variables, + see the [available variables](#available-variables). +1. Make any [necessary customizations](#customizing-the-container-scanning-settings) to the + `Trivy` scanner. We strongly recommended that you minimize customizations, as they + might require changes in future GitLab major releases. + +**Troubleshooting** + +Prior to the GitLab 14.0 release, any variable defined under the scope `container_scanning` is not +considered for the Trivy scanner. Verify that all variables for Trivy are +either defined as a global variable, or under `.cs_common` and `container_scanning_new`. + ### Using a custom SSL CA certificate authority You can use the `ADDITIONAL_CA_CERT_BUNDLE` CI/CD variable to configure a custom SSL CA certificate authority, which is used to verify the peer when fetching Docker images from a registry which uses HTTPS. The `ADDITIONAL_CA_CERT_BUNDLE` value should contain the [text representation of the X.509 PEM public-key certificate](https://tools.ietf.org/html/rfc7468#section-5.1). For example, to configure this value in the `.gitlab-ci.yml` file, use the following: ```yaml -container_scanning: +.cs_common: variables: ADDITIONAL_CA_CERT_BUNDLE: | -----BEGIN CERTIFICATE----- @@ -250,7 +324,7 @@ To allowlist specific vulnerabilities, follow these steps: 1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in [overriding the container scanning template](#overriding-the-container-scanning-template). 1. Define the allowlisted vulnerabilities in a YAML file named `vulnerability-allowlist.yml`. This must use - the format described in [vulnerability-allowlist.yml data format](#vulnerability-allowlistyml-data-format). + the format described in [`vulnerability-allowlist.yml` data format](#vulnerability-allowlistyml-data-format). 1. Add the `vulnerability-allowlist.yml` file to the root folder of your project's Git repository. #### vulnerability-allowlist.yml data format @@ -291,9 +365,9 @@ This example excludes from `gl-container-scanning-report.json`: You can specify container image in multiple ways: - - as image name only (ie. `centos`). - - as full image name with registry hostname (ie. `your.private.registry:5000/centos`). - - as full image name with registry hostname and sha256 label (ie. `registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256`). + - as image name only (such as `centos`). + - as full image name with registry hostname (such as `your.private.registry:5000/centos`). + - as full image name with registry hostname and sha256 label (such as `registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256`). NOTE: The string after CVE ID (`cups` and `libxml2` in the previous example) is an optional comment format. It has **no impact** on the handling of vulnerabilities. You can include comments to describe the vulnerability. @@ -341,7 +415,13 @@ successfully run. For more information, see [Offline environments](../offline_de To use container scanning in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- To configure a local Docker container registry with copies of the container scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [container scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). +- To configure a local Docker container registry with copies of the container scanning images. You + can find these images in their respective registries: + +| GitLab Analyzer | Container Registry | +| --- | --- | +| [Klar](https://gitlab.com/gitlab-org/security-products/analyzers/klar/) (used to run Clair) | [Klar container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry) | +| [Container-Scanning](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning) (used to run Trivy) | [Container-Scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/container_registry/1741162) | Note that GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), meaning the runner tries to pull Docker images from the GitLab container registry even if a local @@ -354,25 +434,34 @@ enables the use of updated scanners in your CI/CD pipelines. Support for custom certificate authorities was introduced in the following versions: -| Analyzer | Version | +| Scanner | Version | | -------- | ------- | -| `klar` | [v2.3.0](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/releases/v2.3.0) | +| `Clair` | [v2.3.0](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/releases/v2.3.0) | +| `Trivy` | [4.0.0](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/releases/4.0.0) | #### Make GitLab container scanning analyzer images available inside your Docker registry For container scanning, import the following default images from `registry.gitlab.com` into your [local Docker container registry](../../packages/container_registry/index.md): +Clair: + ```plaintext registry.gitlab.com/gitlab-org/security-products/analyzers/klar https://hub.docker.com/r/arminc/clair-db ``` +Trivy: + +```plaintext +registry.gitlab.com/gitlab-org/security-products/analyzers/container-scanning +``` + The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which you can import or temporarily access external resources. Note that these scanners -are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you are able to make periodic updates yourself. +process by which you can import or temporarily access external resources. These scanners +are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance), +and you may be able to make occasional updates on your own. For more information, see [the specific steps on how to update an image with a pipeline](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). @@ -384,49 +473,72 @@ For details on saving and transporting Docker images as a file, see Docker's doc 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: + Clair: + ```yaml include: - template: Container-Scanning.gitlab-ci.yml - container_scanning: + .cs_common: image: $CI_REGISTRY/namespace/gitlab-klar-analyzer variables: CLAIR_DB_IMAGE: $CI_REGISTRY/namespace/clair-vulnerabilities-db ``` + Trivy: + + ```yaml + include: + - template: Container-Scanning.gitlab-ci.yml + + .cs_common: + image: $CI_REGISTRY/namespace/gitlab-container-scanning + ``` + 1. If your local Docker container registry is running securely over `HTTPS`, but you're using a self-signed certificate, then you must set `DOCKER_INSECURE: "true"` in the above - `container_scanning` section of your `.gitlab-ci.yml`. + `container_scanning` section of your `.gitlab-ci.yml`. This only applies to Clair. #### Automating container scanning vulnerability database updates with a pipeline -It can be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to -build a new version of the vulnerabilities database on a preset schedule. Automating -this with a pipeline means you do not have to do it manually each time. You can use the following -`.gitlab-yml.ci` as a template: +We recommend that you set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) +to fetch the latest vulnerabilities database on a preset schedule. Because the Clair scanner is +deprecated, the latest vulnerabilities are currently only available for the Trivy scanner. +Automating this with a pipeline means you do not have to do it manually each time. You can use the +following `.gitlab-yml.ci` example as a template. ```yaml -image: docker:stable +variables: + # If using Clair, uncomment the following 2 lines and comment the Trivy lines below + # SOURCE_IMAGE: arminc/clair-db:latest + # TARGET_IMAGE: $CI_REGISTRY/$CI_PROJECT_PATH/clair-vulnerabilities-db -stages: - - build + # If using Trivy, uncomment the following 3 lines and comment the Clair lines above + CS_MAJOR_VERSION: 4 # ensure that this value matches the one you use in your scanning jobs + SOURCE_IMAGE: registry.gitlab.com/gitlab-org/security-products/analyzers/container-scanning:$CS_MAJOR_VERSION + TARGET_IMAGE: $CI_REGISTRY/$CI_PROJECT_PATH/gitlab-container-scanning -build_latest_vulnerabilities: - stage: build +image: docker:stable + +update-vulnerabilities-db: services: - - docker:19.03.12-dind + - docker:19-dind script: - - docker pull arminc/clair-db:latest - - docker tag arminc/clair-db:latest $CI_REGISTRY/namespace/clair-vulnerabilities-db - - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY - - docker push $CI_REGISTRY/namespace/clair-vulnerabilities-db + - docker pull $SOURCE_IMAGE + - docker tag $SOURCE_IMAGE $TARGET_IMAGE + - echo "$CI_REGISTRY_PASSWORD" | docker login $CI_REGISTRY --username $CI_REGISTRY_USER --password-stdin + - docker push $TARGET_IMAGE ``` -The above template works for a GitLab Docker registry running on a local installation, however, if you're using a non-GitLab Docker registry, you need to change the `$CI_REGISTRY` value and the `docker login` credentials to match the details of your local registry. +The above template works for a GitLab Docker registry running on a local installation. However, if +you're using a non-GitLab Docker registry, you must change the `$CI_REGISTRY` value and the +`docker login` credentials to match your local registry's details. ## Running the standalone container scanning tool -It's possible to run the [GitLab container scanning tool](https://gitlab.com/gitlab-org/security-products/analyzers/klar) +### Clair + +It's possible to run [Klar](https://gitlab.com/gitlab-org/security-products/analyzers/klar) against a Docker container without needing to run it within the context of a CI job. To scan an image directly, follow these steps: @@ -458,6 +570,30 @@ image directly, follow these steps: The results are stored in `gl-container-scanning-report.json`. +### Trivy + +It's possible to run the [GitLab container scanning tool](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning) +against a Docker container without needing to run it within the context of a CI job. To scan an +image directly, follow these steps: + +1. Run [Docker Desktop](https://www.docker.com/products/docker-desktop) + or [Docker Machine](https://github.com/docker/machine). + +1. Run the analyzer's Docker image, passing the image and tag you want to analyze in the + `CI_APPLICATION_REPOSITORY` and `CI_APPLICATION_TAG` variables: + + ```shell + docker run \ + --interactive --rm \ + --volume "$PWD":/tmp/app \ + -e CI_PROJECT_DIR=/tmp/app \ + -e CI_APPLICATION_REPOSITORY=registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256 \ + -e CI_APPLICATION_TAG=bc09fe2e0721dfaeee79364115aeedf2174cce0947b9ae5fe7c33312ee019a4e \ + registry.gitlab.com/gitlab-org/security-products/analyzers/container-scanning + ``` + +The results are stored in `gl-container-scanning-report.json`. + ## Reports JSON format The container scanning tool emits a JSON report file. For more information, see the @@ -467,19 +603,19 @@ Here's an example container scanning report: ```json-doc { - "version": "2.3", + "version": "3.0.0", "vulnerabilities": [ { - "id": "ac0997ad-1006-4c81-81fb-ee2bbe6e78e3", + "id": "df52bc8ce9a2ae56bbcb0c4ecda62123fbd6f69b", "category": "container_scanning", - "message": "CVE-2019-3462 in apt", + "message": "CVE-2019-3462 in apt-1.4.8", "description": "Incorrect sanitation of the 302 redirect field in HTTP transport method of apt versions 1.4.8 and earlier can lead to content injection by a MITM attacker, potentially leading to remote code execution on the target machine.", "severity": "High", "confidence": "Unknown", "solution": "Upgrade apt from 1.4.8 to 1.4.9", "scanner": { - "id": "klar", - "name": "klar" + "id": "trivy", + "name": "trivy" }, "location": { "dependency": { @@ -488,7 +624,7 @@ Here's an example container scanning report: }, "version": "1.4.8" }, - "operating_system": "debian:9", + "operating_system": "debian:9.4", "image": "registry.gitlab.com/gitlab-org/security-products/dast/webgoat-8.0@sha256:bc09fe2e0721dfaeee79364115aeedf2174cce0947b9ae5fe7c33312ee019a4e" }, "identifiers": [ @@ -496,27 +632,62 @@ Here's an example container scanning report: "type": "cve", "name": "CVE-2019-3462", "value": "CVE-2019-3462", - "url": "https://security-tracker.debian.org/tracker/CVE-2019-3462" + "url": "http://www.securityfocus.com/bid/106690" } ], "links": [ { - "url": "https://security-tracker.debian.org/tracker/CVE-2019-3462" + "url": "http://www.securityfocus.com/bid/106690" + }, + { + "url": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-3462" + }, + { + "url": "https://lists.apache.org/thread.html/8338a0f605bdbb3a6098bb76f666a95fc2b2f53f37fa1ecc89f1146f@%3Cdevnull.infra.apache.org%3E" + }, + { + "url": "https://lists.debian.org/debian-lts-announce/2019/01/msg00013.html" + }, + { + "url": "https://lists.debian.org/debian-lts-announce/2019/01/msg00014.html" + }, + { + "url": "https://security.netapp.com/advisory/ntap-20190125-0002/" + }, + { + "url": "https://usn.ubuntu.com/3863-1/" + }, + { + "url": "https://usn.ubuntu.com/3863-2/" + }, + { + "url": "https://usn.ubuntu.com/usn/usn-3863-1" + }, + { + "url": "https://usn.ubuntu.com/usn/usn-3863-2" + }, + { + "url": "https://www.debian.org/security/2019/dsa-4371" } ] } ], - "remediations": [ - { - "fixes": [ - { - "id": "c0997ad-1006-4c81-81fb-ee2bbe6e78e3" - } - ], - "summary": "Upgrade apt from 1.4.8 to 1.4.9", - "diff": "YXB0LWdldCB1cGRhdGUgJiYgYXB0LWdldCB1cGdyYWRlIC15IGFwdA==" - } - ] + "remediations": [] + "scan": { + "scanner": { + "id": "trivy", + "name": "Trivy", + "url": "https://github.com/aquasecurity/trivy/", + "vendor": { + "name": "GitLab" + }, + "version": "0.16.0" + }, + "type": "container_scanning", + "start_time": "2021-04-14T19:45:58", + "end_time": "2021-04-14T19:46:18", + "status": "success" + } } ``` @@ -527,12 +698,12 @@ the security vulnerabilities in your groups, projects and pipelines. ## Vulnerabilities database update -For more information about the vulnerabilities database update, check the -[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). +If you're using Klar and want more information about the vulnerabilities database update, see the +[maintenance table](../vulnerabilities/index.md#vulnerability-scanner-maintenance). ## Interacting with the vulnerabilities -After a vulnerability is found, you can [address it](../index.md#addressing-vulnerabilities). +After a vulnerability is found, you can [address it](../vulnerabilities/index.md). ## Solutions for vulnerabilities (auto-remediation) @@ -546,7 +717,7 @@ file, it's necessary to set [`GIT_STRATEGY: fetch`](../../../ci/runners/README.m your `.gitlab-ci.yml` file by following the instructions described in this document's [overriding the container scanning template](#overriding-the-container-scanning-template) section. -Read more about the [solutions for vulnerabilities](../index.md#apply-an-automatic-remediation-for-a-vulnerability). +Read more about the [solutions for vulnerabilities](../vulnerabilities/index.md#remediate-a-vulnerability-automatically). ## Troubleshooting diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index e9097836d83..8b0a84eae4b 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -237,7 +237,7 @@ The `covfuzz-ci.yml` is the same as that in the [original synchronous example](h ## Interacting with the vulnerabilities -After a vulnerability is found, you can [address it](../index.md#addressing-vulnerabilities). +After a vulnerability is found, you can [address it](../vulnerabilities/index.md). The merge request widget lists the vulnerability and contains a button for downloading the fuzzing artifacts. By clicking one of the detected vulnerabilities, you can see its details. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md new file mode 100644 index 00000000000..b8c3529225c --- /dev/null +++ b/doc/user/application_security/dast/browser_based.md @@ -0,0 +1,148 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, howto +--- + +# DAST browser-based crawler **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323423) in GitLab 13.12. + +WARNING: +This product is an early access and is considered a [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) feature. + +GitLab DAST's new browser-based crawler is a crawl engine built by GitLab to test Single Page Applications (SPAs) and traditional web applications. +Due to the reliance of modern web applications on JavaScript, handling SPAs or applications that are dependent on JavaScript is paramount to ensuring proper coverage of an application for Dynamic Application Security Testing (DAST). + +The browser-based crawler works by loading the target application into a specially-instrumented Chromium browser. A snapshot of the page is taken prior to a search to find any actions that a user might perform, +such as clicking on a link or filling in a form. For each action found, the crawler will execute it, take a new snapshot and determine what in the page changed from the previous snapshot. +Crawling continues by taking more snapshots and finding subsequent actions. + +The benefit of crawling by following user actions in a browser is that the crawler can interact with the target application much like a real user would, identifying complex flows that traditional web crawlers don’t understand. This results in better coverage of the website. + +Scanning a web application with both the browser-based crawler and GitLab DAST should provide greater coverage, compared with only GitLab DAST. The new crawler does not support API scanning or the DAST AJAX crawler. + +## Enable browser-based crawler + +The browser-based crawler is an extension to the GitLab DAST product. DAST should be included in the CI/CD configuration and the browser-based crawler enabled using CI/CD variables: + +1. Install the DAST [prerequisites](index.md#prerequisite). +1. Include the [DAST CI template](index.md#include-the-dast-template). +1. Set the target website using the `DAST_WEBSITE` CI/CD variable. +1. Set the CI/CD variable `DAST_BROWSER_SCAN` to `true`. + +An example configuration might look like the following: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +variables: + DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" +``` + +### Available variables + +The browser-based crawler can be configured using CI/CD variables. + +| CI/CD variable | Type | Example | Description | +|--------------------------------------| ----------------| --------------------------------- | ------------| +| `DAST_WEBSITE` | URL | `http://www.site.com` | The URL of the website to scan. | +| `DAST_BROWSER_SCAN` | boolean | `true` | Configures DAST to use the browser-based crawler engine. | +| `DAST_BROWSER_ALLOWED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered in scope when crawled. By default the `DAST_WEBSITE` hostname is included in the allowed hosts list. | +| `DAST_BROWSER_EXCLUDED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are considered excluded and connections are forcibly dropped. | +| `DAST_BROWSER_IGNORED_HOSTS` | List of strings | `site.com,another.com` | Hostnames included in this variable are accessed but not reported against. | +| `DAST_BROWSER_MAX_ACTIONS` | number | `10000` | The maximum number of actions that the crawler performs. For example, clicking a link, or filling a form. | +| `DAST_BROWSER_MAX_DEPTH` | number | `10` | The maximum number of chained actions that the crawler takes. For example, `Click -> Form Fill -> Click` is a depth of three. | +| `DAST_BROWSER_NUMBER_OF_BROWSERS` | number | `3` | The maximum number of concurrent browser instances to use. For shared runners on GitLab.com we recommended a maximum of three. Private runners with more resources may benefit from a higher number, but will likely produce little benefit after five to seven instances. | +| `DAST_BROWSER_COOKIES` | dictionary | `abtesting_group:3,region:locked` | A cookie name and value to be added to every request. | +| `DAST_BROWSER_LOG` | List of strings | `brows:debug,auth:debug` | A list of modules and their intended log level. | +| `DAST_AUTH_URL` | string | `https://example.com/sign-in` | The URL of page that hosts the sign-in form. | +| `DAST_USERNAME` | string | `user123` | The username to enter into the username field on the sign-in HTML form. | +| `DAST_PASSWORD` | string | `p@55w0rd` | The password to enter into the password field on the sign-in HTML form. | +| `DAST_USERNAME_FIELD` | selector | `id:user` | A selector describing the username field on the sign-in HTML form. | +| `DAST_PASSWORD_FIELD` | selector | `css:.password-field` | A selector describing the password field on the sign-in HTML form. | +| `DAST_SUBMIT_FIELD` | selector | `xpath://input[@value='Login']` | A selector describing the element that when clicked submits the login form or the password form of a multi-page login process. | +| `DAST_FIRST_SUBMIT_FIELD` | selector | `.submit` | A selector describing the element that when clicked submits the username form of a multi-page login process. | + +The [DAST variables](index.md#available-variables) `SECURE_ANALYZERS_PREFIX`, `DAST_FULL_SCAN_ENABLED`, `DAST_AUTO_UPDATE_ADDONS`, `DAST_EXCLUDE_RULES`, `DAST_REQUEST_HEADERS`, `DAST_HTML_REPORT`, `DAST_MARKDOWN_REPORT`, `DAST_XML_REPORT`, +`DAST_INCLUDE_ALPHA_VULNERABILITIES`, `DAST_PATHS_FILE`, `DAST_PATHS`, `DAST_ZAP_CLI_OPTIONS`, and `DAST_ZAP_LOG_CONFIGURATION` are also compatible with browser-based crawler scans. + +#### Selectors + +Selectors are used by CI/CD variables to specify the location of an element displayed on a page in a browser. +Selectors have the format `type`:`search string`. The crawler will search for the selector using the search string based on the type. + +| Selector type | Example | Description | +| ------------- | ------------------------------ | ----------- | +| `css` | `css:.password-field` | Searches for a HTML element having the supplied CSS selector. Selectors should be as specific as possible for performance reasons. | +| `id` | `id:element` | Searches for an HTML element with the provided element ID. | +| `name` | `name:element` | Searches for an HTML element with the provided element name. | +| `xpath` | `xpath://*[@id="my-button"]/a` | Searches for a HTML element with the provided XPath. Note that XPath searches are expected to be less performant than other searches. | +| None provided | `a.click-me` | Defaults to searching using a CSS selector. | + +## Vulnerability detection + +While the browser-based crawler crawls modern web applications efficiently, vulnerability detection is still managed by the standard DAST/Zed Attack Proxy (ZAP) solution. + +The crawler runs the target website in a browser with DAST/ZAP configured as the proxy server. This ensures that all requests and responses made by the browser are passively scanned by DAST/ZAP. +When running a full scan, active vulnerability checks executed by DAST/ZAP do not use a browser. This difference in how vulnerabilities are checked can cause issues that require certain features of the target website to be disabled to ensure the scan works as intended. + +For example, for a target website that contains forms with Anti-CSRF tokens, a passive scan will scan as intended because the browser displays pages/forms as if a user is viewing the page. +However, active vulnerability checks run in a full scan will not be able to submit forms containing Anti-CSRF tokens. In such cases we recommend you disable Anti-CSRF tokens when running a full scan. + +## Managing scan time + +It is expected that running the browser-based crawler will result in better coverage for many web applications, when compared to the normal GitLab DAST solution. +This can come at a cost of increased scan time. + +You can manage the trade-off between coverage and scan time with the following measures: + +- Limit the number of actions executed by the browser with the [variable](#available-variables) `DAST_BROWSER_MAX_ACTIONS`. The default is `10,000`. +- Limit the page depth that the browser-based crawler will check coverage on with the [variable](#available-variables) `DAST_BROWSER_MAX_DEPTH`. The crawler uses a breadth-first search strategy, so pages with smaller depth are crawled first. The default is `10`. +- Vertically scaling the runner and using a higher number of browsers with [variable](#available-variables) `DAST_BROWSER_NUMBER_OF_BROWSERS`. The default is `3`. + +## Debugging scans using logging + +Logging can be used to help you troubleshoot a scan. + +The CI/CD variable `DAST_BROWSER_LOG` configures the logging level for particular modules of the crawler. Each module represents a component of the browser-based crawler and is separated so that debug logs can be configured just for the area of the crawler that requires further inspection. For more details, see [Crawler modules](#crawler-modules). + +For example, the following job definition enables the browsing module and the authentication module to be logged in debug-mode: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +variables: + DAST_WEBSITE: "https://my.site.com" + DAST_BROWSER_SCAN: "true" + DAST_BROWSER_LOG: "brows:debug,auth:debug" +``` + +### Log message format + +Log messages have the format `[time] [log level] [log module] [message] [additional properties]`. For example, the following log entry has level `INFO`, is part of the `CRAWL` log module, and has the message `Crawled path`. + +```txt +2021-04-21T00:34:04.000 INF CRAWL Crawled path nav_id=0cc7fd path="LoadURL [https://my.site.com:8090]" +``` + +### Crawler modules + +The modules that can be configured for logging are as follows: + +| Log module | Component overview | +| ---------- | ----------- | +| `AUTH` | Used for creating an authenticated scan. | +| `BROWS` | Used for querying the state/page of the browser. | +| `BPOOL` | The set of browsers that are leased out for crawling. | +| `CRAWL` | Used for the core crawler algorithm. | +| `DATAB` | Used for persisting data to the internal database. | +| `LEASE` | Used to create browsers to add them to the browser pool. | +| `MAIN` | Used for the flow of the main event loop of the crawler. | +| `NAVDB` | Used for persistence mechanisms to store navigation entries. | +| `REPT` | Used for generating reports. | +| `STAT` | Used for general statistics while running the scan. | diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 65ddece1bde..1093e7cfabd 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -144,6 +144,14 @@ image. Using the `DAST_VERSION` variable, you can choose how DAST updates: Find the latest DAST versions on the [Releases](https://gitlab.com/gitlab-org/security-products/dast/-/releases) page. +#### Crawling web applications dependent on JavaScript + +GitLab has released a new browser-based crawler, an add-on to DAST that uses a browser to crawl web applications for content. This crawler replaces the standard DAST Spider and Ajax Crawler. + +The browser-based crawler crawls websites by browsing web pages as a user would. This approach works well with web applications that make heavy use of JavaScript, such as Single Page Applications. + +For more details, including setup instructions, see [DAST browser-based crawler](browser_based.md). + ## Deployment options Depending on the complexity of the target application, there are a few options as to how to deploy and configure @@ -452,11 +460,14 @@ configured to act as a remote proxy and add the `Gitlab-DAST-Permission` header. ### API scan -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10928) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10928) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. +> - A new DAST API scanning engine was introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.10. Using an API specification as a scan's target is a useful way to seed URLs for scanning an API. Vulnerability rules in an API scan are different than those in a normal website scan. +A new DAST API scanning engine is available in GitLab 13.12 and later. For more details, see [DAST API scanning engine](../dast_api). The new scanning engine supports REST, SOAP, GraphQL, and generic APIs using forms, XML, and JSON. Testing can be performed using OpenAPI, Postman Collections, and HTTP Archive (HAR) documents. + #### Specification format API scans support OpenAPI V2 and OpenAPI V3 specifications. You can define these specifications using `JSON` or `YAML`. @@ -471,7 +482,7 @@ include: - template: DAST.gitlab-ci.yml variables: - DAST_API_SPECIFICATION: http://my.api/api-specification.yml + DAST_API_OPENAPI: http://my.api/api-specification.yml ``` #### Import API specification from a file @@ -486,7 +497,7 @@ dast: - cp api-specification.yml /zap/wrk/api-specification.yml variables: GIT_STRATEGY: fetch - DAST_API_SPECIFICATION: api-specification.yml + DAST_API_OPENAPI: api-specification.yml ``` #### Full API scan @@ -522,7 +533,7 @@ include: - template: DAST.gitlab-ci.yml variables: - DAST_API_SPECIFICATION: http://api-test.host.com/api-specification.yml + DAST_API_OPENAPI: http://api-test.host.com/api-specification.yml DAST_API_HOST_OVERRIDE: api-test.host.com ``` @@ -537,7 +548,7 @@ include: - template: DAST.gitlab-ci.yml variables: - DAST_API_SPECIFICATION: http://api-test.api.com/api-specification.yml + DAST_API_OPENAPI: http://api-test.api.com/api-specification.yml DAST_REQUEST_HEADERS: "Authorization: Bearer my.token" ``` @@ -610,10 +621,42 @@ When using `DAST_PATHS` and `DAST_PATHS_FILE`, note the following: To perform a [full scan](#full-scan) on the listed paths, use the `DAST_FULL_SCAN_ENABLED` CI/CD variable. +### View details of a vulnerability detected by DAST + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. + +Vulnerabilities detected by DAST occur in the live web application. Addressing these types of +vulnerabilities requires specific information. DAST provides the information required to +investigate and rectify the underlying cause. + +To view details of vulnerabilities detected by DAST: + +1. To see all vulnerabilities detected, either: + - Go to your project and select **Security & Compliance**. + - Go to the merge request and select the **Security** tab. + +1. Select a vulnerability's description. The following details are provided: + + | Field | Description | + |:-----------------|:------------------------------------------------------------------ | + | Description | Description of the vulnerability. | + | Project | Namespace and project in which the vulnerability was detected. | + | Method | HTTP method used to detect the vulnerability. | + | URL | URL at which the vulnerability was detected. | + | Request Headers | Headers of the request. | + | Response Status | Response status received from the application. | + | Response Headers | Headers of the response received from the application. | + | Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. | + | Identifiers | Identifiers of the vulnerability. | + | Severity | Severity of the vulnerability. | + | Scanner Type | Type of vulnerability report. | + | Links | Links to further details of the detected vulnerability. | + | Solution | Details of a recommended solution to the vulnerability (optional). | + ### Customizing the DAST settings WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. The DAST settings can be changed through CI/CD variables by using the @@ -641,8 +684,9 @@ DAST can be [configured](#customizing-the-dast-settings) using CI/CD variables. | CI/CD variable | Type | Description | |------------------------------| --------|-------------| | `SECURE_ANALYZERS_PREFIX` | URL | Set the Docker registry base address from which to download the analyzer. | -| `DAST_WEBSITE` | URL | The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. | -| `DAST_API_SPECIFICATION` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_WEBSITE` | URL | The URL of the website to scan. `DAST_API_OPENAPI` must be specified if this is omitted. | +| `DAST_API_OPENAPI` | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | +| `DAST_API_SPECIFICATION` | URL or string | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290241) in GitLab 13.12 and replaced by `DAST_API_OPENAPI`. To be removed in GitLab 15.0. The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. | | `DAST_SPIDER_START_AT_HOST` | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` is reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. | | `DAST_AUTH_URL` | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. | | `DAST_AUTH_VERIFICATION_URL` | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST will exit if it cannot access the URL. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. @@ -658,6 +702,7 @@ DAST can be [configured](#customizing-the-dast-settings) using CI/CD variables. | `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | | `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` | | `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_ONLY_INCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to configure the scan to run only them. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250651) in GitLab 13.12. | | `DAST_REQUEST_HEADERS` | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | | `DAST_DEBUG` | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | | `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | @@ -794,9 +839,9 @@ For DAST, import the following default DAST analyzer image from `registry.gitlab The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note -that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you're able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. +These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), @@ -944,15 +989,18 @@ required for an on-demand DAST scan. A site profile contains the following: - **Profile name**: A name you assign to the site to be scanned. +- **Site type**: The type of target to be scanned, either website or API scan. - **Target URL**: The URL that DAST runs against. - **Excluded URLs**: A comma-separated list of URLs to exclude from the scan. - **Request headers**: A comma-separated list of HTTP request headers, including names and values. These headers are added to every request made by DAST. - **Authentication**: - - **Authenticated URL**: The URL of the page containing the sign-in HTML form on the target website. The username and password are submitted with the login form to create an authenticated scan. + - **Authenticated URL**: The URL of the page containing the sign-in HTML form on the target website. The username and password are submitted with the login form to create an authenticated scan. - **Username**: The username used to authenticate to the website. - **Password**: The password used to authenticate to the website. - - **Username form field**: The name of username field at the sign-in HTML form. - - **Password form field**: The name of password field at the sign-in HTML form. + - **Username form field**: The name of username field at the sign-in HTML form. + - **Password form field**: The name of password field at the sign-in HTML form. + +When an API site type is selected, a [host override](#host-override) is used to ensure the API being scanned is on the same host as the target. This is done to reduce the risk of running an active scan against the wrong API. #### Site profile validation diff --git a/doc/user/application_security/dast_api/img/dast_api_postman_collection_edit_variable.png b/doc/user/application_security/dast_api/img/dast_api_postman_collection_edit_variable.png Binary files differnew file mode 100644 index 00000000000..3a2799ecdc1 --- /dev/null +++ b/doc/user/application_security/dast_api/img/dast_api_postman_collection_edit_variable.png diff --git a/doc/user/application_security/dast_api/img/dast_api_postman_environment_edit_variable.png b/doc/user/application_security/dast_api/img/dast_api_postman_environment_edit_variable.png Binary files differnew file mode 100644 index 00000000000..656ba7652cd --- /dev/null +++ b/doc/user/application_security/dast_api/img/dast_api_postman_environment_edit_variable.png diff --git a/doc/user/application_security/dast_api/img/dast_api_postman_request_edit.png b/doc/user/application_security/dast_api/img/dast_api_postman_request_edit.png Binary files differnew file mode 100644 index 00000000000..3750af8f455 --- /dev/null +++ b/doc/user/application_security/dast_api/img/dast_api_postman_request_edit.png diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md new file mode 100644 index 00000000000..5e47f545ef9 --- /dev/null +++ b/doc/user/application_security/dast_api/index.md @@ -0,0 +1,1140 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, howto +--- + +# DAST API **(ULTIMATE)** + +You can add dynamic application security testing of web APIs to your [GitLab CI/CD](../../../ci/README.md) pipelines. +This helps you discover bugs and potential security issues that other QA processes may miss. + +We recommend that you use DAST API testing in addition to [GitLab Secure](../index.md)'s +other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +you can run DAST API tests as part your CI/CD workflow. + +## Requirements + +- One of the following web API types: + - REST API + - SOAP + - GraphQL + - Form bodies, JSON, or XML +- One of the following assets to provide APIs to test: + - OpenAPI v2 or v3 API definition + - Postman Collection v2.0 or v2.1 + - HTTP Archive (HAR) of API requests to test + +## When DAST API scans run + +When using the `DAST-API.gitlab-ci.yml` template, the defined jobs use the `dast` stage by default. To enable your `.gitlab-ci.yml` file must include the `dast` stage in your `stages` definition. To ensure DAST API scans the latest code, your CI pipeline should deploy changes to a test environment in a stage before the `dast` stage: + +```yaml +stages: + - build + - test + - deploy + - dast +``` + +Note that if your pipeline is configured to deploy to the same web server on each run, running a +pipeline while another is still running could cause a race condition in which one pipeline +overwrites the code from another. The API to scan should be excluded from changes for the duration +of a DAST API scan. The only changes to the API should be from the DAST API scanner. Be aware that +any changes made to the API (for example, by users, scheduled tasks, database changes, code +changes, other pipelines, or other scanners) during a scan could cause inaccurate results. + +## Enable DAST API scanning + +There are three ways to perform scans. See the configuration section for the one you wish to use: + +- [OpenAPI v2 or v3 specification](#openapi-specification) +- [HTTP Archive (HAR)](#http-archive-har) +- [Postman Collection v2.0 or v2.1](#postman-collection) + +Examples of various configurations can be found here: + +- [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/openapi-example) +- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/har-example) +- [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/postman-example) +- [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/graphql-example) + +WARNING: +GitLab 14.0 will require that you place DAST API configuration files (for example, +`gitlab-dast-api-config.yml`) in your repository's `.gitlab` directory instead of your +repository's root. You can continue using your existing configuration files as they are, but +starting in GitLab 14.0, GitLab will not check your repository's root for configuration files. + +### OpenAPI Specification + +> Support for OpenAPI Specification using YAML format was +> [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0. + +The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs. +This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test. +OpenAPI Specifications are provided as a file system resource or URL. Both JSON and YAML OpenAPI formats are supported. + +DAST API uses an OpenAPI document to generate the request body. When a request body is required, +the body generation is limited to these body types: + +- `application/x-www-form-urlencoded` +- `multipart/form-data` +- `application/json` + +Follow these steps to configure DAST API in GitLab with an OpenAPI specification: + +1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) + that's provided as part of your GitLab installation. Add the following to your + `.gitlab-ci.yml` file: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + ``` + +1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. + Testing with this profile completes faster, allowing for easier configuration validation. + + Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, + substituting `Quick` for the profile you choose: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + ``` + +1. Provide the location of the OpenAPI specification. You can provide the specification as a file + or URL. Specify the location by adding the `DAST_API_OPENAPI` variable: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + ``` + +1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run DAST API against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its URL in an `environment_url.txt` file. DAST API + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `DAST_API_TARGET_URL`: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for DAST API. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +WARNING: +**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting data. Only run DAST API scanning against a test server. + +### HTTP Archive (HAR) + +The [HTTP Archive format (HAR)](http://www.softwareishard.com/blog/har-12-spec/) +is an archive file format for logging HTTP transactions. When used with the GitLab DAST API scanner, HAR must contain records of calling the web API to test. The DAST API scanner extracts all the requests and +uses them to perform testing. + +You can use various tools to generate HAR files: + +- [Insomnia Core](https://insomnia.rest/): API client +- [Chrome](https://www.google.com/chrome/): Browser +- [Firefox](https://www.mozilla.org/en-US/firefox/): Browser +- [Fiddler](https://www.telerik.com/fiddler): Web debugging proxy +- [GitLab HAR Recorder](https://gitlab.com/gitlab-org/security-products/har-recorder): Command line + +WARNING: +HAR files may contain sensitive information such as authentication tokens, API keys, and session +cookies. We recommend that you review the HAR file contents before adding them to a repository. + +Follow these steps to configure DAST API to use a HAR file that provides information about the +target API to test: + +1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + ``` + +1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. + Testing with this profile completes faster, allowing for easier configuration validation. + + Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, + substituting `Quick` for the profile you choose: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + ``` + +1. Provide the location of the HAR specification. You can provide the specification as a file + or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `DAST_API_HAR` variable: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_HAR: test-api-recording.har + ``` + +1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run DAST API against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its URL in an `environment_url.txt` file. DAST API + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `DAST_API_TARGET_URL`: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_HAR: test-api-recording.har + DAST_API_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for DAST API. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +WARNING: +**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run DAST API against a test server. + +### Postman Collection + +The [Postman API Client](https://www.postman.com/product/api-client/) is a popular tool that +developers and testers use to call various types of APIs. The API definitions +[can be exported as a Postman Collection file](https://learning.postman.com/docs/getting-started/importing-and-exporting-data/#exporting-postman-data) +for use with DAST API. When exporting, make sure to select a supported version of Postman +Collection: v2.0 or v2.1. + +When used with the GitLab DAST API scanner, Postman Collections must contain definitions of the web API to +test with valid data. The DAST API scanner extracts all the API definitions and uses them to perform +testing. + +WARNING: +Postman Collection files may contain sensitive information such as authentication tokens, API keys, +and session cookies. We recommend that you review the Postman Collection file contents before adding +them to a repository. + +Follow these steps to configure DAST API to use a Postman Collection file that provides +information about the target API to test: + +1. To use DAST API, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + ``` + +1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. + Testing with this profile completes faster, allowing for easier configuration validation. + + Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, + substituting `Quick` for the profile you choose: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + ``` + +1. Provide the location of the Postman Collection specification. You can provide the specification as a file or URL. Specify the location by adding the `DAST_API_POSTMAN_COLLECTION` variable: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json + ``` + +1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run DAST API against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its URL in an `environment_url.txt` file. DAST API + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `DAST_API_TARGET_URL`: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json + DAST_API_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for DAST API. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +WARNING: +**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run DAST API against a test server. + +#### Postman variables + +Postman allows the developer to define placeholders that can be used in different parts of the +requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/). +You can use variables to store and reuse values in your requests and scripts. For example, you can +edit the collection to add variables to the document: + + + +You can then use the variables in sections such as URL, headers, and others: + + + +Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) +(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at +the Environment scope: + + + +When you export a Postman collection, only Postman collection variables are exported into the +Postman file. For example, Postman does not export environment-scoped variables into the Postman +file. + +By default, the DAST API scanner uses the Postman file to resolve Postman variable values. If a JSON file +is set in a GitLab CI environment variable `DAST_API_POSTMAN_COLLECTION_VARIABLES`, then the JSON +file takes precedence to get Postman variable values. + +Although Postman can export environment variables into a JSON file, the format is not compatible +with the JSON expected by `DAST_API_POSTMAN_COLLECTION_VARIABLES`. + +Here is an example of using `DAST_API_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json + DAST_API_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json + DAST_API_TARGET_URL: http://test-deployment/ +``` + +The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with +key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + + ```json + { + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" + } + ``` + +### Authentication + +Authentication is handled by providing the authentication token as a header or cookie. You can +provide a script that performs an authentication flow or calculates the token. + +#### HTTP Basic Authentication + +[HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) +is an authentication method built in to the HTTP protocol and used in conjunction with +[transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). +To use HTTP basic authentication, two CI/CD variables are added to your `.gitlab-ci.yml` file: + +- `DAST_API_HTTP_USERNAME`: The username for authentication. +- `DAST_API_HTTP_PASSWORD`: The password for authentication. + +For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables) +(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the +GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable +as the value for `DAST_API_HTTP_PASSWORD`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_HAR: test-api-recording.har + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_HTTP_USERNAME: testuser + DAST_API_HTTP_PASSWORD: $TEST_API_PASSWORD +``` + +#### Bearer Tokens + +Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web +Tokens (JWT). Bearer tokens are transmitted using the `Authorization` HTTP header. To use bearer +tokens with DAST API, you need one of the following: + +- A token that doesn't expire +- A way to generate a token that lasts the length of testing +- A Python script that DAST API can call to generate the token + +##### Token doesn't expire + +If the bearer token doesn't expire, use the `DAST_API_OVERRIDES_ENV` variable to provide it. This +variable's content is a JSON snippet that provides headers and cookies to add to DAST API's +outgoing HTTP requests. + +Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: + +1. [Create a CI/CD variable](../../../ci/variables/README.md#custom-cicd-variables), + for example `TEST_API_BEARERAUTH`, with the value + `{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You + can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the + **Variables** section. + +1. In your `.gitlab-ci.yml` file, set `DAST_API_OVERRIDES_ENV` to the variable you just created: + + ```yaml + stages: + - dast + + include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_ENV: $TEST_API_BEARERAUTH + ``` + +1. To validate that authentication is working, run an DAST API test and review the job logs + and the test API's application logs. + +##### Token generated at test runtime + +If the bearer token must be generated and doesn't expire during testing, you can provide to DAST API a file containing the token. A prior stage and job, or part of the DAST API job, can +generate this file. + +DAST API expects to receive a JSON file with the following structure: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +This file can be generated by a prior stage and provided to DAST API through the +`DAST_API_OVERRIDES_FILE` CI/CD variable. + +Set `DAST_API_OVERRIDES_FILE` in your `.gitlab-ci.yml` file: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json +``` + +To validate that authentication is working, run an DAST API test and review the job logs and +the test API's application logs. + +##### Token has short expiration + +If the bearer token must be generated and expires prior to the scan's completion, you can provide a +program or script for the DAST API scanner to execute on a provided interval. The provided script runs in +an Alpine Linux container that has Python 3 and Bash installed. If the Python script requires +additional packages, it must detect this and install the packages at runtime. + +The script must create a JSON file containing the bearer token in a specific format: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +You must provide three CI/CD variables, each set for correct operation: + +- `DAST_API_OVERRIDES_FILE`: JSON file the provided command generates. +- `DAST_API_OVERRIDES_CMD`: Command that generates the JSON file. +- `DAST_API_OVERRIDES_INTERVAL`: Interval (in seconds) to run command. + +For example: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_CMD: renew_token.py + DAST_API_OVERRIDES_INTERVAL: 300 +``` + +To validate that authentication is working, run an DAST API test and review the job logs and +the test API's application logs. + +### Configuration files + +To get you started quickly, GitLab provides the configuration file +[`gitlab-dast-api-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/dast/-/blob/master/config/gitlab-dast-api-config.yml). +This file has several testing profiles that perform various numbers of tests. The run time of each +profile increases as the test numbers go up. To use a configuration file, add it to your +repository's root as `.gitlab/gitlab-dast-api-config.yml`. + +#### Profiles + +The following profiles are pre-defined in the default configuration file. Profiles +can be added, removed, and modified by creating a custom configuration. + +##### Quick + +- Application Information Check +- Cleartext Authentication Check +- FrameworkDebugModeCheck +- HTML Injection Check +- Insecure Http Methods Check +- JSON Hijacking Check +- JSON Injection Check +- Sensitive Information Check +- Session Cookie Check +- SQL Injection Check +- Token Check +- XML Injection Check + +##### Full + +- Application Information Check +- Cleartext AuthenticationCheck +- CORS Check +- DNS Rebinding Check +- Framework Debug Mode Check +- HTML Injection Check +- Insecure Http Methods Check +- JSON Hijacking Check +- JSON Injection Check +- Open Redirect Check +- Sensitive File Check +- Sensitive Information Check +- Session Cookie Check +- SQL Injection Check +- TLS Configuration Check +- Token Check +- XML Injection Check + +### Available CI/CD variables + +| CI/CD variable | Description | +|------------------------------------------------------|--------------------| +| `DAST_API_VERSION` | Specify DAST API container version. Defaults to `latest`. | +| `DAST_API_TARGET_URL` | Base URL of API testing target. | +|[`DAST_API_CONFIG`](#configuration-files) | DAST API configuration file. Defaults to `.gitlab-dast-api.yml`. | +|[`DAST_API_PROFILE`](#configuration-files) | Configuration profile to use during testing. Defaults to `Quick`. | +|[`DAST_API_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. | +|[`DAST_API_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | +|[`DAST_API_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | +|[`DAST_API_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. | +|[`DAST_API_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | +|[`DAST_API_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | +|[`DAST_API_OVERRIDES_CMD`](#overrides) | Overrides command. | +|[`DAST_API_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | +|[`DAST_API_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | +|[`DAST_API_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | +|`DAST_API_SERVICE_START_TIMEOUT` | How long to wait for target API to become available in seconds. Default is 300 seconds. | +|`DAST_API_TIMEOUT` | How long to wait for API responses in seconds. Default is 30 seconds. | + +### Overrides + +DAST API provides a method to add or override specific items in your request, for example: + +- Headers +- Cookies +- Query string +- Form data +- JSON nodes +- XML nodes + +You can use this to inject semantic version headers, authentication, and so on. The +[authentication section](#authentication) includes examples of using overrides for that purpose. + +Overrides use a JSON document, where each type of override is represented by a JSON object: + +```json +{ + "headers": { + "header1": "value", + "header2": "value" + }, + "cookies": { + "cookie1": "value", + "cookie2": "value" + }, + "query": { + "query-string1": "value", + "query-string2": "value" + }, + "body-form": { + "form-param1": "value", + "form-param1": "value", + }, + "body-json": { + "json-path1": "value", + "json-path2": "value", + }, + "body-xml" : { + "xpath1": "value", + "xpath2": "value", + } +} +``` + +Example of setting a single header: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +Example of setting both a header and cookie: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + }, + "cookies": { + "flags": "677" + } +} +``` + +Example usage for setting a `body-form` override: + +```json +{ + "body-form": { + "username": "john.doe" + } +} +``` + +The override engine uses `body-form` when the request body has only form-data content. + +Example usage for setting a `body-json` override: + +```json +{ + "body-json": { + "$.credentials.access-token": "iddqd!42.$" + } +} +``` + +Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/) +expression. The JSON Path expression `$.credentials.access-token` identifies the node to be +overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body +has only [JSON](https://www.json.org/json-en.html) content. + +For example, if the body is set to the following JSON: + +```json +{ + "credentials" : { + "username" :"john.doe", + "access-token" : "non-valid-password" + } +} +``` + +It is changed to: + +```json +{ + "credentials" : { + "username" :"john.doe", + "access-token" : "iddqd!42.$" + } +} +``` + +Here's an example for setting a `body-xml` override. The first entry overrides an XML attribute and +the second entry overrides an XML element: + +```json +{ + "body-xml" : { + "/credentials/@isEnabled": "true", + "/credentials/access-token/text()" : "iddqd!42.$" + } +} +``` + +Note that each JSON property name in the object `body-xml` is set to an +[XPath v2](https://www.w3.org/TR/xpath20/) +expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override +with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the +element node to override with the value `iddqd!42.$`. The override engine uses `body-xml` when the +request body has only [XML](https://www.w3.org/XML/) +content. + +For example, if the body is set to the following XML: + +```xml +<credentials isEnabled="false"> + <username>john.doe</username> + <access-token>non-valid-password</access-token> +</credentials> +``` + +It is changed to: + +```xml +<credentials isEnabled="true"> + <username>john.doe</username> + <access-token>iddqd!42.$</access-token> +</credentials> +``` + +You can provide this JSON document as a file or environment variable. You may also provide a command +to generate the JSON document. The command can run at intervals to support values that expire. + +#### Using a file + +To provide the overrides JSON as a file, the `DAST_API_OVERRIDES_FILE` CI/CD variable is set. The path is relative to the job current working directory. + +Here's an example `.gitlab-ci.yml`: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json +``` + +#### Using a CI/CD variable + +To provide the overrides JSON as a CI/CD variable, use the `DAST_API_OVERRIDES_ENV` variable. +This allows you to place the JSON as variables that can be masked and protected. + +In this example `.gitlab-ci.yml`, the `DAST_API_OVERRIDES_ENV` variable is set directly to the JSON: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_ENV: '{"headers":{"X-API-Version":"2"}}' +``` + +In this example `.gitlab-ci.yml`, the `SECRET_OVERRIDES` variable provides the JSON. This is a +[group or instance level CI/CD variable defined in the UI](../../../ci/variables/README.md#instance-cicd-variables): + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_ENV: $SECRET_OVERRIDES +``` + +#### Using a command + +If the value must be generated or regenerated on expiration, you can provide a program or script for +the DAST API scanner to execute on a specified interval. The provided script runs in an Alpine Linux +container that has Python 3 and Bash installed. If the Python script requires additional packages, +it must detect this and install the packages at runtime. The script creates the overrides JSON file +as defined above. + +You must provide three CI/CD variables, each set for correct operation: + +- `DAST_API_OVERRIDES_FILE`: File generated by the provided command. +- `DAST_API_OVERRIDES_CMD`: Command to generate JSON file. +- `DAST_API_OVERRIDES_INTERVAL`: Interval in seconds to run command. + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_CMD: renew_token.py + DAST_API_OVERRIDES_INTERVAL: 300 +``` + +## Running your first scan + +When configured correctly, a CI/CD pipeline contains a `dast` stage and an `dast_api` job. The job only fails when an invalid configuration is provided. During normal operation, the job always succeeds even if vulnerabilities are identified during testing. + +Vulnerabilities are displayed on the **Security** pipeline tab with the suite name. When testing against the repositories default branch, the DAST API vulnerabilities are also shown on the Security & Compliance's Vulnerability Report page. + +To prevent an excessive number of reported vulnerabilities, the DAST API scanner limits the number of vulnerabilities it reports per operation. + +## Viewing DAST API vulnerabilities + +The DAST API analyzer produces a JSON report that is collected and used +[to populate the vulnerabilities into GitLab vulnerability screens](#view-details-of-a-dast-api-vulnerability). + +See [handling false positives](#handling-false-positives) for information about configuration changes you can make to limit the number of false positives reported. + +### View details of a DAST API vulnerability + +Follow these steps to view details of a vulnerability: + +1. You can view vulnerabilities in a project, or a merge request: + + - In a project, go to the project's **{shield}** **Security & Compliance > Vulnerability Report** + page. This page shows all vulnerabilities from the default branch only. + - In a merge request, go the merge request's **Security** section and click the **Expand** + button. DAST API vulnerabilities are available in a section labeled + **DAST detected N potential vulnerabilities**. Click the title to display the vulnerability + details. + +1. Click the vulnerabilities title to display the details. The table below describes these details. + + | Field | Description | + |:--------------------|:----------------------------------------------------------------------------------------| + | Description | Description of the vulnerability including what was modified. | + | Project | Namespace and project in which the vulnerability was detected. | + | Method | HTTP method used to detect the vulnerability. | + | URL | URL at which the vulnerability was detected. | + | Request | The HTTP request that caused the vulnerability. | + | Unmodified Response | Response from an unmodified request. This is what a normal working response looks like. | + | Actual Response | Response received from test request. | + | Evidence | How we determined a vulnerability occurred. | + | Identifiers | The DAST API check used to find this vulnerability. | + | Severity | Severity of the vulnerability. | + | Scanner Type | Scanner used to perform testing. | + +### Security Dashboard + +The Security Dashboard is a good place to get an overview of all the security vulnerabilities in your groups, projects and +pipelines. For more information, see the [Security Dashboard documentation](../security_dashboard/index.md). + +### Interacting with the vulnerabilities + +Once a vulnerability is found, you can interact with it. Read more on how to +[address the vulnerabilities](../vulnerabilities/index.md). + +## Handling False Positives + +False positives can be handled in several ways: + +- Dismiss the vulnerability. +- Some checks have several methods of detecting when a vulnerability is identified, called _Assertions_. + Assertions can also be turned off and configured. For example, the DAST API scanner by default uses HTTP + status codes to help identify when something is a real issue. If an API returns a 500 error during + testing, this creates a vulnerability. This isn't always desired, as some frameworks return 500 errors often. +- Turn off the Check producing the false positive. This prevents the check from generating any + vulnerabilities. Example checks are the SQL Injection Check, and JSON Hijacking Check. + +### Turn off a Check + +Checks perform testing of a specific type and can be turned on and off for specific configuration +profiles. The provided [configuration files](#configuration-files) define several profiles that you +can use. The profile definition in the configuration file lists all the checks that are active +during a scan. To turn off a specific check, remove it from the profile definition in the +configuration file. The profiles are defined in the `Profiles` section of the configuration file. + +Example profile definition: + +```yaml +Profiles: + - Name: Quick + DefaultProfile: Empty + Routes: + - Route: *Route0 + Checks: + - Name: ApplicationInformationCheck + - Name: CleartextAuthenticationCheck + - Name: FrameworkDebugModeCheck + - Name: HtmlInjectionCheck + - Name: InsecureHttpMethodsCheck + - Name: JsonHijackingCheck + - Name: JsonInjectionCheck + - Name: SensitiveInformationCheck + - Name: SessionCookieCheck + - Name: SqlInjectionCheck + - Name: TokenCheck + - Name: XmlInjectionCheck +``` + +To turn off the JSON Hijacking Check you can remove these lines: + +```yaml + - Name: JsonHijackingCheck +``` + +This results in the following YAML: + +```yaml +- Name: Quick + DefaultProfile: Empty + Routes: + - Route: *Route0 + Checks: + - Name: ApplicationInformationCheck + - Name: CleartextAuthenticationCheck + - Name: FrameworkDebugModeCheck + - Name: HtmlInjectionCheck + - Name: InsecureHttpMethodsCheck + - Name: JsonInjectionCheck + - Name: SensitiveInformationCheck + - Name: SessionCookieCheck + - Name: SqlInjectionCheck + - Name: TokenCheck + - Name: XmlInjectionCheck +``` + +### Turn off an Assertion for a Check + +Assertions detect vulnerabilities in tests produced by checks. Many checks support multiple Assertions such as Log Analysis, Response Analysis, and Status Code. When a vulnerability is found, the Assertion used is provided. To identify which Assertions are on by default, see the Checks default configuration in the configuration file. The section is called `Checks`. + +This example shows the SQL Injection Check: + +```yaml +- Name: SqlInjectionCheck + Configuration: + UserInjections: [] + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: StatusCodeAssertion +``` + +Here you can see three Assertions are on by default. A common source of false positives is +`StatusCodeAssertion`. To turn it off, modify its configuration in the `Profiles` section. This +example provides only the other two Assertions (`LogAnalysisAssertion`, +`ResponseAnalysisAssertion`). This prevents `SqlInjectionCheck` from using `StatusCodeAssertion`: + +```yaml +Profiles: + - Name: Quick + DefaultProfile: Empty + Routes: + - Route: *Route0 + Checks: + - Name: ApplicationInformationCheck + - Name: CleartextAuthenticationCheck + - Name: FrameworkDebugModeCheck + - Name: HtmlInjectionCheck + - Name: InsecureHttpMethodsCheck + - Name: JsonHijackingCheck + - Name: JsonInjectionCheck + - Name: SensitiveInformationCheck + - Name: SessionCookieCheck + - Name: SqlInjectionCheck + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: TokenCheck + - Name: XmlInjectionCheck +``` + +## Troubleshooting + +### Failed to start scanner session (version header not found) + +The DAST API engine outputs an error message when it cannot establish a connection with the scanner application component. The error message is shown in the job output window of the `dast_api` job. A common cause of this issue is changing the `DAST_API_API` variable from its default. + +**Error message** + +- In [GitLab 13.11 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/323939), `Failed to start scanner session (version header not found).` +- In GitLab 13.10 and earlier, `API Security version header not found. Are you sure that you are connecting to the API Security server?`. + +**Solution** + +- Remove the `DAST_API_API` variable from the `.gitlab-ci.yml` file. The value will be inherited from the DAST API CI/CD template. We recommend this method instead of manually setting a value. +- If removing the variable is not possible, check to see if this value has changed in the latest version of the [DAST API CI/CD template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml). If so, update the value in the `.gitlab-ci.yml` file. + +### Application cannot determine the base URL for the target API + +The DAST API engine outputs an error message when it cannot determine the target API after inspecting the OpenAPI document. This error message is shown when the target API has not been set in the `.gitlab-ci.yml` file, it is not available in the `environment_url.txt` file, and it could not be computed using the OpenAPI document. + +There is a order of precedence in which the DAST API engine tries to get the target API when checking the different sources. First, it will try to use the `DAST_API_TARGET_URL`. If the environment variable has not been set, then the DAST API engine will attempt to use the `environment_url.txt` file. If there is no file `environment_url.txt`, then the DAST API engine will use the OpenAPI document contents and the URL provided in `DAST_API_OPENAPI` (if a URL is provided) to try to compute the target API. + +The best-suited solution will depend on whether or not your target API changes for each deployment. In static environments, the target API is the same for each deployment, in this case please refer to the [static environment solution](#static-environment-solution). If the target API changes for each deployment a [dynamic environment solution](#dynamic-environment-solutions) should be applied. + +#### Static environment solution + +This solution is for pipelines in which the target API URL doesn't change (is static). + +**Add environmental variable** + +For environments where the target API remains the same, we recommend you specify the target URL by using the `DAST_API_TARGET_URL` environment variable. In your `.gitlab-ci.yml`, add a variable `DAST_API_TARGET_URL`. The variable must be set to the base URL of API testing target. For example: + +```yaml +include: + - template: DAST-API.gitlab-ci.yml + + variables: + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_OPENAPI: test-api-specification.json +``` + +#### Dynamic environment solutions + +In a dynamic environment your target API changes for each different deployment. In this case, there is more than one possible solution, we recommend you use the `environment_url.txt` file when dealing with dynamic environments. + +**Use environment_url.txt** + +To support dynamic environments in which the target API URL changes during each pipeline, DAST API engine supports the use of an `environment_url.txt` file that contains the URL to use. This file is not checked into the repository, instead it's created during the pipeline by the job that deploys the test target and collected as an artifact that can be used by later jobs in the pipeline. The job that creates the `environment_url.txt` file must run before the DAST API engine job. + +1. Modify the test target deployment job adding the base URL in an `environment_url.txt` file at the root of your project. +1. Modify the test target deployment job collecting the `environment_url.txt` as an artifact. + +Example: + +```yaml +deploy-test-target: + script: + # Perform deployment steps + # Create environment_url.txt (example) + - echo http://${CI_PROJECT_ID}-${CI_ENVIRONMENT_SLUG}.example.org > environment_url.txt + + artifacts: + paths: + - environment_url.txt +``` + +## Glossary + +- Assert: Assertions are detection modules used by checks to trigger a vulnerability. Many assertions have + configurations. A check can use multiple Assertions. For example, Log Analysis, Response Analysis, + and Status Code are common Assertions used together by checks. Checks with multiple Assertions + allow them to be turned on and off. +- Check: Performs a specific type of test, or performed a check for a type of vulnerability. For + example, the SQL Injection Check performs DAST testing for SQL Injection vulnerabilities. The DAST API scanner is comprised of several checks. Checks can be turned on and off in a profile. +- Profile: A configuration file has one or more testing profiles, or sub-configurations. You may + have a profile for feature branches and another with extra testing for a main branch. diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index 25b7615a8ae..fcefba943ad 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -10,10 +10,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10075) in GitLab Ultimate 12.0. Use the dependency list to review your project's dependencies and key -details about those dependencies, including their known vulnerabilities. To see the dependency list, -in your project, go to **Security & Compliance > Dependency List**. +details about those dependencies, including their known vulnerabilities. It is a collection of dependencies in your project, including existing and new findings. To see the dependency list, go to your project and select **Security & Compliance > Dependency List**. This information is sometimes referred to as a Software Bill of Materials or SBoM / BOM. +The dependency list only shows the results of the last successful pipeline to run on the default branch. This is why we recommend not changing the default behavior of allowing the secure jobs to fail. + ## Prerequisites To view your project's dependencies, ensure you meet the following requirements: diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 53d91bfcd78..0faa33e0123 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -56,10 +56,10 @@ variables: This configuration requires that your custom registry provides images for all the official analyzers. -### Selecting specific analyzers +### Disable specific analyzers -You can select the official analyzers you want to run. Here's how to enable -`bundler-audit` and `gemnasium` while disabling all the other default ones. +You can select the official analyzers you don't want to run. Here's how to disable +`bundler-audit` and `gemnasium` analyzers. In `.gitlab-ci.yml` define: ```yaml @@ -67,26 +67,23 @@ include: template: Dependency-Scanning.gitlab-ci.yml variables: - DS_DEFAULT_ANALYZERS: "bundler-audit,gemnasium" + DS_EXCLUDED_ANALYZERS: "bundler-audit, gemnasium" ``` -`bundler-audit` runs first. When merging the reports, Dependency Scanning -removes the duplicates and keeps the `bundler-audit` entries. - ### Disabling default analyzers -Setting `DS_DEFAULT_ANALYZERS` to an empty string disables all the official -default analyzers. In `.gitlab-ci.yml` define: +Setting `DS_EXCLUDED_ANALYZERS` to a list of the official analyzers disables them. +In `.gitlab-ci.yml` define: ```yaml include: template: Dependency-Scanning.gitlab-ci.yml variables: - DS_DEFAULT_ANALYZERS: "" + DS_EXCLUDED_ANALYZERS: "gemnasium, gemansium-maven, gemnasium-python, bundler-audit, retire.js" ``` -That's needed when one totally relies on [custom analyzers](#custom-analyzers). +This is used when one totally relies on [custom analyzers](#custom-analyzers). ## Custom analyzers diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 53387acefef..8e23db89dfd 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -77,6 +77,7 @@ The following languages and dependency managers are supported: 1. Support for [sbt](https://www.scala-sbt.org/) 1.3 and above was added in GitLab 13.9. Plans are underway for supporting the following languages, dependency managers, and dependency files. For details, see the issue link for each. +For workarounds, see the [Troubleshooting section](#troubleshooting) | Package Managers | Languages | Supported files | Scan tools | Issue | | ------------------- | --------- | --------------- | ---------- | ----- | @@ -129,7 +130,7 @@ configuration, the last mention of the variable takes precedence. ### Overriding dependency scanning jobs WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. To override a job definition (for example, to change properties like `variables` or `dependencies`), @@ -168,7 +169,8 @@ The following variables allow configuration of global dependency scanning settin | CI/CD variables | Description | | ----------------------------|------------ | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | -| `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | +| `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). | +| `DS_DEFAULT_ANALYZERS` | ([**DEPRECATED - use `DS_EXCLUDED_ANALYZERS` instead**](https://gitlab.com/gitlab-org/gitlab/-/issues/287691)) Override the names of the official default images. For more information, see [Dependency Scanning Analyzers](analyzers.md). | | `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. | | `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | | `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info`. | @@ -227,13 +229,13 @@ Read more on [how to use private Maven repositories](../index.md#using-private-m ## Interacting with the vulnerabilities Once a vulnerability is found, you can interact with it. Read more on how to -[address the vulnerabilities](../index.md#addressing-vulnerabilities). +[address the vulnerabilities](../vulnerabilities/index.md). ## Solutions for vulnerabilities (auto-remediation) Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. Read more about the -[solutions for vulnerabilities](../index.md#apply-an-automatic-remediation-for-a-vulnerability). +[solutions for vulnerabilities](../vulnerabilities/index.md#remediate-a-vulnerability-automatically). ## Security Dashboard @@ -243,8 +245,8 @@ vulnerabilities in your groups, projects and pipelines. Read more about the ## Vulnerabilities database update -For more information about the vulnerabilities database update, check the -[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). +For more information about the vulnerabilities database update, see the +[maintenance table](../vulnerabilities/index.md#vulnerability-scanner-maintenance). ## Dependency List @@ -420,8 +422,8 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. -Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you can make periodic updates yourself. +These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -508,7 +510,7 @@ ensure that it can reach your private repository. Here is an example configurati ## Hosting a copy of the gemnasium_db advisory database -The [gemnasium_db](https://gitlab.com/gitlab-org/security-products/gemnasium-db) Git repository is +The [`gemnasium_db`](https://gitlab.com/gitlab-org/security-products/gemnasium-db) Git repository is used by `gemnasium`, `gemnasium-maven`, and `gemnasium-python` as the source of vulnerability data. This repository updates at scan time to fetch the latest advisories. However, due to a restricted networking environment, running this update is sometimes not possible. In this case, a user can do @@ -563,11 +565,58 @@ such references: ERROR: Could not find dependencies: <dependency-name>. You may need to run npm install ``` -As a workaround, remove the [`retire.js`](analyzers.md#selecting-specific-analyzers) analyzer from -[DS_DEFAULT_ANALYZERS](#configuring-dependency-scanning). +As a workaround, add the [`retire.js`](analyzers.md) analyzer to +[`DS_EXCLUDED_ANALYZERS`](#configuring-dependency-scanning). ## Troubleshooting +### Working around missing support for certain languages or package managers + +As noted in the ["Supported languages" section](#supported-languages-and-package-managers) +some dependency definition files are not yet supported. +However, Dependency Scanning can be achieved if +the language, a package manager, or a third-party tool +can convert the definition file +into a supported format. + +Generally, the approach is the following: + +1. Define a dedicated converter job in your `.gitlab-ci.yml` file. + Use a suitable Docker image, script, or both to facilitate the conversion. +1. Let that job upload the converted, supported file as an artifact. +1. Add [`dependencies: [<your-converter-job>]`](../../../ci/yaml/README.md#dependencies) + to your `dependency_scanning` job to make use of the converted definitions files. + +For example, the currently unsupported `poetry.lock` file can be +[converted](https://python-poetry.org/docs/cli/#export) +to the supported `requirements.txt` as follows. + +```yaml +include: + - template: Dependency-Scanning.gitlab-ci.yml + +stages: + - .pre + - test + +variables: + PIP_REQUIREMENTS_FILE: "requirements-converted.txt" + +convert-poetry: + stage: .pre + image: python:3-slim + script: + - pip install poetry # Or via another method: https://python-poetry.org/docs/#installation + - poetry export --output "$PIP_REQUIREMENTS_FILE" + artifacts: + paths: + - "$PIP_REQUIREMENTS_FILE" + +dependency_scanning: + stage: test + dependencies: ["convert-poetry"] +``` + ### `Error response from daemon: error processing tar file: docker-tar: relocation error` This error occurs when the Docker version that runs the dependency scanning job is `19.03.0`. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 1ba2161362c..82a018c0ae9 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -7,9 +7,16 @@ type: reference, howto # Application security **(ULTIMATE)** -GitLab can check your application for security vulnerabilities that may lead to unauthorized access, -data leaks, denial of services, and more. GitLab reports vulnerabilities in the merge request so you -can fix them before you merge. +GitLab can check your application for security vulnerabilities including: + +- Unauthorized access. +- Data leaks. +- Denial of service attacks. + +Statistics and details on vulnerabilities are included in the merge request. Providing +actionable information _before_ changes are merged enables you to be proactive. + +GitLab also provides high-level statistics of vulnerabilities across projects and groups: - The [Security Dashboard](security_dashboard/index.md) provides a high-level view of vulnerabilities detected in your projects, pipeline, and groups. @@ -18,50 +25,7 @@ can fix them before you merge. you can immediately begin risk analysis and remediation. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview of GitLab application security, see -[Security Deep Dive](https://www.youtube.com/watch?v=k4vEJnGYy84). - -## Quick start - -Get started quickly with Dependency Scanning, License Scanning, Static Application Security -Testing (SAST), and Secret Detection by adding the following to your [`.gitlab-ci.yml`](../../ci/yaml/README.md): - -```yaml -include: - - template: Security/Dependency-Scanning.gitlab-ci.yml - - template: Security/License-Scanning.gitlab-ci.yml - - template: Security/SAST.gitlab-ci.yml - - template: Security/Secret-Detection.gitlab-ci.yml -``` - -To add Dynamic Application Security Testing (DAST) scanning, add the following to your -`.gitlab-ci.yml` and replace `https://staging.example.com` with a staging server's web address: - -```yaml -include: - - template: Security/DAST.gitlab-ci.yml - -variables: - DAST_WEBSITE: https://staging.example.com -``` - -To ensure the DAST scanner runs *after* deploying the application to the staging server, review the [DAST full documentation](dast/index.md). - -To add Container Scanning, follow the steps listed in the [Container Scanning documentation](container_scanning/index.md#requirements). - -To further configure any of the other scanners, refer to each scanner's documentation. - -### SAST configuration - -You can set up and configure Static Application Security Testing -(SAST) for your project, without opening a text editor. For more details, -see [configure SAST in the UI](sast/index.md#configure-sast-in-the-ui). - -### Override the default registry base address - -By default, GitLab security scanners use `registry.gitlab.com/gitlab-org/security-products/analyzers` as the -base address for Docker images. You can override this globally by setting the CI/CD variable -`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once. +For an overview of GitLab application security, see [Shifting Security Left](https://www.youtube.com/watch?v=XnYstHObqlA&t). ## Security scanning tools @@ -73,29 +37,17 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | [Dependency List](dependency_list/index.md) **(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. | | [Dynamic Application Security Testing (DAST)](dast/index.md) **(ULTIMATE)** | Analyze running web applications for known vulnerabilities. | +| [DAST API](dast_api/index.md) **(ULTIMATE)** | Analyze running web APIs for known vulnerabilities. | | [API fuzzing](api_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities in web APIs with fuzzing. | | [Secret Detection](secret_detection/index.md) | Analyze Git history for leaked secrets. | | [Security Dashboard](security_dashboard/index.md) **(ULTIMATE)** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) | Analyze source code for known vulnerabilities. | | [Coverage fuzzing](coverage_fuzzing/index.md) **(ULTIMATE)** | Find unknown bugs and vulnerabilities with coverage-guided fuzzing. | -### Use security scanning tools with Pipelines for Merge Requests - -The security scanning tools can all be added to pipelines with [templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Security). -See each tool for details on how to use include each template in your CI/CD configuration. - -By default, the application security jobs are configured to run for branch pipelines only. -To use them with [pipelines for merge requests](../../ci/merge_request_pipelines/index.md), -you may need to override the default `rules:` configuration to add: - -```yaml -rules: - - if: $CI_PIPELINE_SOURCE == "merge_request_event" -``` +## Security scanning with Auto DevOps -## Security Scanning with Auto DevOps - -When [Auto DevOps](../../topics/autodevops/) is enabled, all GitLab Security scanning tools are configured using default settings. +To enable all GitLab Security scanning tools, with default settings, enable +[Auto DevOps](../../topics/autodevops/): - [Auto SAST](../../topics/autodevops/stages.md#auto-sast) - [Auto Secret Detection](../../topics/autodevops/stages.md#auto-secret-detection) @@ -106,170 +58,125 @@ When [Auto DevOps](../../topics/autodevops/) is enabled, all GitLab Security sca While you cannot directly customize Auto DevOps, you can [include the Auto DevOps template in your project's `.gitlab-ci.yml` file](../../topics/autodevops/customize.md#customizing-gitlab-ciyml). -## Maintenance and update of the vulnerabilities database - -The scanning tools and vulnerabilities database are updated regularly. +## Security scanning without Auto DevOps -| Secure scanning tool | Vulnerabilities database updates | -|:-------------------------------------------------------------|-------------------------------------------| -| [Container Scanning](container_scanning/index.md) | Uses `clair`. The latest `clair-db` version is used for each job by running the [`latest` Docker image tag](https://gitlab.com/gitlab-org/gitlab/blob/438a0a56dc0882f22bdd82e700554525f552d91b/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L37). The `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). | -| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | -| [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/master/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | -| [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | +To enable all GitLab security scanning tools, with the option of customizing settings, add the +GitLab CI/CD templates to your `.gitlab-ci.yml` file. -Currently, you do not have to update GitLab to benefit from the latest vulnerabilities definitions. -The security tools are released as Docker images. The vendored job definitions that enable them use -major release tags according to [Semantic Versioning](https://semver.org/). Each new release of the -tools overrides these tags. -The Docker images are updated to match the previous GitLab releases, so users automatically get the -latest versions of the scanning tools without having to do anything. There are some known issues -with this approach, however, and there is a -[plan to resolve them](https://gitlab.com/gitlab-org/gitlab/-/issues/9725). - -## View security scan information in merge requests **(FREE)** +To enable Static Application Security Testing, Dependency Scanning, License Scanning, and Secret +Detection, add: -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4393) in GitLab Free 13.5. -> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/273205) in 13.6. -> - Report download dropdown [added](https://gitlab.com/gitlab-org/gitlab/-/issues/273418) in 13.7. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249550) in GitLab 13.9. - -Merge requests which have run security scans let you know that the generated -reports are available to download. To download a report, click on the -**Download results** dropdown, and select the desired report. - - - -## View details of a DAST vulnerability - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36332) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +```yaml +include: + - template: Security/Dependency-Scanning.gitlab-ci.yml + - template: Security/License-Scanning.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml + - template: Security/Secret-Detection.gitlab-ci.yml +``` -Vulnerabilities detected by DAST occur in the live web application. Rectification of these types of -vulnerabilities requires specific information. DAST provides the information required to -investigate and rectify the underlying cause. +To enable Dynamic Application Security Testing (DAST) scanning, add the following to your +`.gitlab-ci.yml`. Replace `https://staging.example.com` with a staging server's web address: -To view details of DAST vulnerabilities: +```yaml +include: + - template: Security/DAST.gitlab-ci.yml -1. To see all vulnerabilities detected: - - In a project, go to the project's **{shield}** **Security & Compliance** page. - - Only in a merge request, go the merge request's **Security** tab. +variables: + DAST_WEBSITE: https://staging.example.com +``` -1. Select the vulnerability's description. The following details are provided: +For more details about each of the security scanning tools, see their respective +[documentation sections](#security-scanning-tools). -| Field | Description | -|:-----------------|:------------------------------------------------------------------ | -| Description | Description of the vulnerability. | -| Project | Namespace and project in which the vulnerability was detected. | -| Method | HTTP method used to detect the vulnerability. | -| URL | URL at which the vulnerability was detected. | -| Request Headers | Headers of the request. | -| Response Status | Response status received from the application. | -| Response Headers | Headers of the response received from the application. | -| Evidence | Evidence of the data found that verified the vulnerability. Often a snippet of the request or response, this can be used to help verify that the finding is a vulnerability. | -| Identifiers | Identifiers of the vulnerability. | -| Severity | Severity of the vulnerability. | -| Scanner Type | Type of vulnerability report. | -| Links | Links to further details of the detected vulnerability. | -| Solution | Details of a recommended solution to the vulnerability (optional). | +### Override the default registry base address -### Hide sensitive information in headers +By default, GitLab security scanners use `registry.gitlab.com/gitlab-org/security-products/analyzers` as the +base address for Docker images. You can override this globally by setting the CI/CD variable +`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once. -HTTP request and response headers may contain sensitive information, including cookies and -authorization credentials. By default, content of specific headers are masked in DAST vulnerability -reports. You can specify the list of all headers to be masked. For details, see -[Hide sensitive information](dast/index.md#hide-sensitive-information). +### Use security scanning tools with Pipelines for Merge Requests -## Addressing vulnerabilities +By default, the application security jobs are configured to run for branch pipelines only. +To use them with [pipelines for merge requests](../../ci/merge_request_pipelines/index.md), +you may need to override the default `rules:` configuration to add: -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.8. +```yaml +rules: + - if: $CI_PIPELINE_SOURCE == "merge_request_event" +``` -For each security vulnerability in a merge request or [Vulnerability Report](vulnerability_report/index.md), -you can: +## Default behavior of GitLab security scanning tools -- [Dismiss the vulnerability](#dismiss-a-vulnerability). -- Create a [confidential](../project/issues/confidential_issues.md) - [issue](vulnerabilities/index.md#create-a-gitlab-issue-for-a-vulnerability). -- Apply an [automatically remediation](#apply-an-automatic-remediation-for-a-vulnerability). +### Secure jobs in your pipeline -### Dismiss a vulnerability +If you add the security scanning jobs as described in [Security scanning with Auto DevOps](#security-scanning-with-auto-devops) or [Security scanning without Auto DevOps](#security-scanning-without-auto-devops) to your `.gitlab-ci.yml` each added [security scanning tool](#security-scanning-tools) behave as described below. -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0, a dismissal reason. +For each compatible analyzer, a job is created in the `test`, `dast` or `fuzz` stage of your pipeline and runs on the next new branch pipeline. Features such as the [Security Dashboard](security_dashboard/index.md), [Vulnerability Report](vulnerability_report/index.md), and [Dependency List](dependency_list/index.md) that rely on this scan data only show results from pipelines on the default branch. Please note that one tool may use many analyzers. -You can dismiss a vulnerability for the entire project. +Our language and package manager specific jobs attempt to assess which analyzer(s) they should run for your project so that you can do less configuration. -1. Select the vulnerability in the Security Dashboard. -1. In the top-right, from the **Status** selector menu, select **Dismissed**. -1. Optional. Add a reason for the dismissal and select **Save comment**. +If you want to override this to increase the pipeline speed you may choose which analyzers to exclude if you know they are not applicable (languages or package managers not contained in your project) by following variable customization directions for that specific tool. -To undo this action, select a different status from the same menu. +### Secure job status -#### Dismiss multiple vulnerabilities +Jobs pass if they are able to complete a scan. A _pass_ result does NOT indicate if they did, or did not, identify findings. The only exception is coverage fuzzing, which fails if it identifies findings. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35816) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. +Jobs fail if they are unable to complete a scan. You can view the pipeline logs for more information. -You can dismiss multiple vulnerabilities at once. +All jobs are permitted to fail by default. This means that if they fail it do not fail the pipeline. -1. In the list of vulnerabilities, select the checkbox for each vulnerability you want to dismiss. - To select all, select the checkbox in the table header. -1. Above the table, select a dismissal reason. -1. Select **Dismiss Selected**. +If you want to prevent vulnerabilities from being merged, you should do this by adding [Security Approvals in Merge Requests](#security-approvals-in-merge-requests) which prevents unknown, high or critical findings from being merged without an approval from a specific group of people that you choose. -### Create an issue for a vulnerability +We do not recommend changing the job [`allow_failure` setting](../../ci/yaml/README.md#allow_failure) as that fails the entire pipeline. -You can create a GitLab or Jira issue for a vulnerability. For details, see [Vulnerability Pages](vulnerabilities/index.md). +### JSON Artifact -#### Link to an existing issue +The artifact generated by the secure analyzer contains all findings it discovers on the target branch, regardless of whether they were previously found, dismissed, or completely new (it puts in everything that it finds). -If you already have an open issue, you can link to it from the vulnerability. +## View security scan information in merge requests **(FREE)** -- The vulnerability page shows related issues, but the issue page doesn't show the vulnerability it's related to. -- An issue can only be related to one vulnerability at a time. -- Issues can be linked across groups and projects. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4393) in GitLab Free 13.5. +> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/273205) in 13.6. +> - Report download dropdown [added](https://gitlab.com/gitlab-org/gitlab/-/issues/273418) in 13.7. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249550) in GitLab 13.9. -To link to an existing issue: +### All tiers -1. Open the vulnerability. -1. [Add a linked issue](../project/issues/related_issues.md). +Merge requests which have run security scans let you know that the generated +reports are available to download. To download a report, click on the +**Download results** dropdown, and select the desired report. -### Apply an automatic remediation for a vulnerability + -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5656) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.7. +### Ultimate -Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. -The following scanners are supported: +A merge request contains a security widget which displays a summary of the NEW results. New results are determined by comparing the current findings against existing findings in the target (default) branch (if there are prior findings). -- [Dependency Scanning](dependency_scanning/index.md). - Automatic Patch creation is only available for Node.js projects managed with - `yarn`. -- [Container Scanning](container_scanning/index.md). +We recommended you run a scan of the `default` branch before enabling feature branch scans for your developers. Otherwise, there is no base for comparison and all feature branches display the full scan results in the merge request security widget. -#### Manually apply the suggested patch +The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both NEW and EXISTING findings. -To manually apply the patch that GitLab generated for a vulnerability: +From the merge request security widget, select **Expand** to unfold the widget, displaying any new and no longer detected (removed) findings by scan type. Select **View Full Report** to go directly to the **Security** tab in the latest branch pipeline. -1. Select the **Resolve with merge request** dropdown, then select **Download patch to resolve**: +## View security scan information in the pipeline Security tab -  +A pipeline's security tab lists all findings in the current branch. It includes new findings introduced by this branch and existing vulnerabilities that were already present when the branch was created. These results likely do not match the findings displayed in the Merge Request security widget as those do not include the existing vulnerabilities (with the exception of showing any existing vulnerabilities that are no longer detected in the feature branch). -1. Ensure your local project has the same commit checked out that was used to generate the patch. -1. Run `git apply remediation.patch`. -1. Verify and commit the changes to your branch. +For more details, see [security tab](security_dashboard/index.md#pipeline-security). -#### Create a merge request with the suggested patch +## View security scan information in the Security Dashboard -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9224) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9. +The Security Dashboard show vulnerabilities present in a project's default branch. Data is updated every 24 hours. Vulnerability count updates resulting from any feature branches introducing new vulnerabilities that are merged to default are included after the daily data refresh. -In some cases, you can create a merge request that automatically remediates the -vulnerability. Any vulnerability that has a -[solution](#apply-an-automatic-remediation-for-a-vulnerability) can have a merge -request created to automatically solve the issue. +For more details, see [Security Dashboard](security_dashboard/index.md). -If this action is available: +## View security scan information in the Vulnerability Report -1. Select the **Resolve with merge request** dropdown, then select **Resolve with merge request**. +The vulnerability report shows the results of the last completed pipeline on the default branch. It is updated on every pipeline completion. All detected vulnerabilities are shown as well as any previous ones that are no longer detected in the latest scan. Vulnerabilities that are no longer detected may have been remediated or otherwise removed and can be marked as `Resolved` after proper verification. Vulnerabilities that are no longer detected are denoted with an icon for filtering and review. -  +By default, the vulnerability report does not show vulnerabilities of `dismissed` or `resolved` status so you can focus on open vulnerabilities. You can change the Status filter to see these. -A merge request is created. It that applies the solution to the source branch. +[Read more about the Vulnerability report](vulnerability_report/index.md). ## Security approvals in merge requests @@ -297,7 +204,7 @@ rating. ### Enabling Security Approvals within a project -To enable the `Vulnerability-Check` or `License-Check` Security Approvals, a [project approval rule](../project/merge_requests/merge_request_approvals.md#adding--editing-a-default-approval-rule) +To enable the `Vulnerability-Check` or `License-Check` Security Approvals, a [project approval rule](../project/merge_requests/approvals/rules.md#add-an-approval-rule) must be created. A [security scanner job](#security-scanning-tools) must be enabled for `Vulnerability-Check`, and a [license scanning](../compliance/license_compliance/index.md#configuration) job must be enabled for `License-Check`. When the proper jobs aren't configured, the following @@ -412,6 +319,70 @@ You can do it quickly by following the hyperlink given to run a new pipeline.  +## DAST On-Demand Scans + +If you don’t want scans running in your normal DevOps process you can use on-demand scans instead. For more details, see [on-demand scans](dast/index.md#on-demand-scans). This feature is only available for DAST. If you run an on-demand scan against the default branch, it is reported as a "successful pipeline" and these results are included in the security dashboard and vulnerability report. + +## Security report validation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321918) in GitLab 13.11. + +As of GitLab 13.11, we've introduced the **optional** validation of the security report artifacts based on the +[report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/tree/master/dist). +If you enable validation, GitLab validates the report artifacts before ingesting the vulnerabilities. +This prevents ingesting broken vulnerability data into the database. + +### Enable security report validation + +To enable report artifacts validation, set the `VALIDATE_SCHEMA` environment variable to `"true"` for the jobs in the `.gitlab-ci.yml` file. + +For example, the configuration below enables validation for only the `sast` job: + + ```yaml + include: + - template: Security/Dependency-Scanning.gitlab-ci.yml + - template: Security/License-Scanning.gitlab-ci.yml + - template: Security/SAST.gitlab-ci.yml + - template: Security/Secret-Detection.gitlab-ci.yml + + stages: + - security-scan + + dependency_scanning: + stage: security-scan + + license_scanning: + stage: security-scan + + sast: + stage: security-scan + variables: + VALIDATE_SCHEMA: "true" + + .secret-analyzer: + stage: security-scan + ``` + +## Interacting with findings and vulnerabilities + +There are a variety of locations and ways to interact with the results of the security scanning tools: + +- [Scan information in merge requests](#view-security-scan-information-in-merge-requests) +- [Project Security Dashboard](security_dashboard/#project-security-dashboard) +- [Security pipeline tab](security_dashboard/#pipeline-security) +- [Group Security Dashboard](security_dashboard/#group-security-dashboard) +- [Security Center](security_dashboard/#security-center) +- [Vulnerability Report](vulnerability_report/index.md) +- [Vulnerability Pages](vulnerabilities/index.md) +- [Dependency List](dependency_list/index.md) + +For more details about which findings or vulnerabilities you can view in each of those locations, select the respective link. Each page details the ways in which you can interact with the findings and vulnerabilities. As an example, in most cases findings start out as _detected_ status. You have the option to: + +- Change the status. +- Create an issue. +- Link it to an existing issue. +- In some cases, [apply an automatic remediation for a vulnerability](vulnerabilities/index.md#remediate-a-vulnerability-automatically). + ## Troubleshooting ### Getting error message `sast job: stage parameter should be [some stage name here]` @@ -480,7 +451,7 @@ Found errors in your .gitlab-ci.yml: ``` This error appears when the included job's `rules` configuration has been [overridden](sast/index.md#overriding-sast-jobs) -with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#onlyexcept-basic) +with [the deprecated `only` or `except` syntax.](../../ci/yaml/README.md#only--except) To fix this issue, you must either: - [Transition your `only/except` syntax to `rules`](#transitioning-your-onlyexcept-syntax-to-rules). @@ -491,7 +462,7 @@ To fix this issue, you must either: #### Transitioning your `only/except` syntax to `rules` When overriding the template to control job execution, previous instances of -[`only` or `except`](../../ci/yaml/README.md#onlyexcept-basic) are no longer compatible +[`only` or `except`](../../ci/yaml/README.md#only--except) are no longer compatible and must be transitioned to [the `rules` syntax](../../ci/yaml/README.md#rules). If your override is aimed at limiting jobs to only run on `master`, the previous syntax diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 7c013a2a9de..c9c65e94b32 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -59,14 +59,14 @@ mirroring the packages inside your own offline network. ### Interacting with the vulnerabilities Once a vulnerability is found, you can interact with it. Read more on how to -[address the vulnerabilities](../index.md#addressing-vulnerabilities). +[address the vulnerabilities](../vulnerabilities/index.md). Please note that in some cases the reported vulnerabilities provide metadata that can contain external links exposed in the UI. These links might not be accessible within an offline environment. ### Automatic remediation for vulnerabilities -The [automatic remediation for vulnerabilities](../index.md#apply-an-automatic-remediation-for-a-vulnerability) feature is available for offline Dependency Scanning and Container Scanning, but may not work +The [automatic remediation for vulnerabilities](../vulnerabilities/index.md#remediate-a-vulnerability-automatically) feature is available for offline Dependency Scanning and Container Scanning, but may not work depending on your instance's configuration. We can only suggest solutions, which are generally more current versions that have been patched, when we are able to access up-to-date registry services hosting the latest versions of that dependency or image. diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 208fbdfa5f3..92f0d6924b3 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -66,7 +66,8 @@ scan_execution_policy: enabled: true rules: - type: pipeline - branch: master + branches: + - master actions: - scan: dast scanner_profile: Scanner Profile A @@ -76,7 +77,8 @@ scan_execution_policy: enabled: true rules: - type: pipeline - branch: main + branches: + - main actions: - scan: dast scanner_profile: Scanner Profile C @@ -108,7 +110,17 @@ This rule enforces the defined actions whenever the pipeline runs for a selected | Field | Type | Possible values | Description | |-------|------|-----------------|-------------| | `type` | `string` | `pipeline` | The rule's type. | -| `branch` | `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | + +### `schedule` rule type + +This rule enforces the defined actions and schedules a scan on the provided date/time. + +| Field | Type | Possible values | Description | +|------------|------|-----------------|-------------| +| `type` | `string` | `schedule` | The rule's type. | +| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | ### `scan` action type @@ -129,6 +141,9 @@ Note the following: - Once you associate the site profile and scanner profile by name in the policy, it is not possible to modify or delete them. If you want to modify them, you must first disable the policy by setting the `active` flag to `false`. +- When configuring policies with a scheduled DAST scan, the author of the commit in the security + policy project's repository must have access to the scanner and site profiles. Otherwise, the scan + is not scheduled successfully. Here's an example: @@ -140,17 +155,20 @@ scan_execution_policy: enabled: true rules: - type: pipeline - branch: release/* + branches: + - release/* actions: - scan: dast scanner_profile: Scanner Profile A site_profile: Site Profile B -- name: Enforce DAST in every pipeline in main branch - description: This policy enforces pipeline configuration to have a job with DAST scan for main branch +- name: Enforce DAST scan every 10 minutes + description: This policy enforces a DAST scan to run every 10 minutes enabled: true rules: - - type: pipeline - branch: main + - type: schedule + branches: + - main + cadence: */10 * * * * actions: - scan: dast scanner_profile: Scanner Profile C @@ -160,11 +178,7 @@ scan_execution_policy: In this example, the DAST scan runs with the scanner profile `Scanner Profile A` and the site profile `Site Profile B` for every pipeline executed on branches that match the `release/*` wildcard (for example, branch name `release/v1.2.1`); and the DAST scan runs with -the scanner profile `Scanner Profile C` and the site profile `Site Profile D` for every pipeline executed on `main` branch. - -NOTE: -All scanner and site profiles must be configured and created for each project that is assigned to the selected Security Policy Project. -If they are not created, the job will fail with the error message. +the scanner profile `Scanner Profile C` and the site profile `Site Profile D` every 10 minutes. ## Security Policy project selection diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index c085dafac12..0e69f3b68eb 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -45,7 +45,7 @@ GitLab, but users can also integrate their own **custom images**. ## SAST analyzer features -For an analyzer to be considered Generally Available, it is expected to minimally +For an analyzer to be considered Generally Available, it is expected to minimally support the following features: - [Customizable configuration](index.md#available-variables) diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index cbd05f6267e..886726d5d67 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -134,16 +134,16 @@ All open source (OSS) analyzers have been moved to the GitLab Free tier as of Gi Different features are available in different [GitLab tiers](https://about.gitlab.com/pricing/), as shown in the following table: -| Capability | In Free | In Ultimate | -|:-------------------------------------------------------------------------------------------------------------|:--------------------|:-------------------| -| [Configure SAST Scanners](#configuration) | **{check-circle}** | **{check-circle}** | -| [Customize SAST Settings](#customizing-the-sast-settings) | **{check-circle}** | **{check-circle}** | -| View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | -| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | -| [Address vulnerabilities](../../application_security/index.md#addressing-vulnerabilities) | **{dotted-circle}** | **{check-circle}** | -| [Access to Security Dashboard](../../application_security/security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | -| [Configure SAST in the UI](#configure-sast-in-the-ui) | **{dotted-circle}** | **{check-circle}** | -| [Customize SAST Rulesets](#customize-rulesets) | **{dotted-circle}** | **{check-circle}** | +| Capability | In Free | In Ultimate | +|:---------------------------------------------------------------------------------------|:--------------------|:-------------------| +| [Configure SAST Scanners](#configuration) | **{check-circle}** | **{check-circle}** | +| [Customize SAST Settings](#customizing-the-sast-settings) | **{check-circle}** | **{check-circle}** | +| View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | +| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | +| [Address vulnerabilities](../../application_security/vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | +| [Access to Security Dashboard](../../application_security/security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | +| [Configure SAST in the UI](#configure-sast-in-the-ui) | **{dotted-circle}** | **{check-circle}** | +| [Customize SAST Rulesets](#customize-rulesets) | **{dotted-circle}** | **{check-circle}** | ## Contribute your scanner @@ -222,7 +222,7 @@ the pipeline configuration, the last mention of the variable takes precedence. ### Overriding SAST jobs WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. To override a job definition, (for example, change properties like `variables` or `dependencies`), @@ -247,8 +247,8 @@ You can customize the default scanning rules provided by our SAST analyzers. Ruleset customization supports two capabilities: -1. Disabling predefined rules -1. Modifying the default behavior of a given analyzer +1. Disabling predefined rules (available for all analyzers). +1. Modifying the default behavior of a given analyzer (only available for `nodejs-scan` and `gosec`). These capabilities can be used simultaneously. @@ -464,7 +464,7 @@ Some analyzers make it possible to filter out vulnerabilities under a given thre | CI/CD variable | Default value | Description | |------------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec` ). Parent directories also match patterns. You might need to exclude temporary directories used by your build tool as these can generate false positives. | +| `SAST_EXCLUDED_PATHS` | `spec, test, tests, tmp` | Exclude vulnerabilities from output based on the paths. This is a comma-separated list of patterns. Patterns can be globs, or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. You might need to exclude temporary directories used by your build tool as these can generate false positives. To exclude paths, copy and paste the default excluded paths, then **add** your own paths to be excluded. If you don't specify the default excluded paths, you will override the defaults and _only_ paths you specify will be excluded from the SAST scans. | | `SEARCH_MAX_DEPTH` | 4 | SAST searches the repository to detect the programming languages used, and selects the matching analyzers. Set the value of `SEARCH_MAX_DEPTH` to specify how many directory levels the search phase should span. After the analyzers have been selected, the _entire_ repository is analyzed. | | `SAST_BANDIT_EXCLUDED_PATHS` | | Comma-separated list of paths to exclude from scan. Uses Python's [`fnmatch` syntax](https://docs.python.org/2/library/fnmatch.html); For example: `'*/tests/*, */venv/*'` | | `SAST_BRAKEMAN_LEVEL` | 1 | Ignore Brakeman vulnerabilities under given confidence level. Integer, 1=Low 3=High. | @@ -517,7 +517,6 @@ removed, or promoted to regular features at any time. Experimental features available are: - Enable scanning of iOS and Android apps using the [MobSF analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/mobsf/). -- Enable the [semgrep analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/). #### Enable experimental features @@ -652,14 +651,15 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan:2 registry.gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit:2 registry.gitlab.com/gitlab-org/security-products/analyzers/pmd-apex:2 registry.gitlab.com/gitlab-org/security-products/analyzers/security-code-scan:2 +registry.gitlab.com/gitlab-org/security-products/analyzers/semgrep:2 registry.gitlab.com/gitlab-org/security-products/analyzers/sobelow:2 registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 ``` The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you're able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -681,7 +681,7 @@ Support for custom certificate authorities was introduced in the following versi | `phpcs-security-audit` | [v2.8.2](https://gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit/-/releases/v2.8.2) | | `pmd-apex` | [v2.1.0](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex/-/releases/v2.1.0) | | `security-code-scan` | [v2.7.3](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/releases/v2.7.3) | -| `semgrep` | [v0.0.1](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/releases/v0.0.1) | +| `semgrep` | [v0.0.1](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep/-/releases/v0.0.1) | | `sobelow` | [v2.2.0](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow/-/releases/v2.2.0) | | `spotbugs` | [v2.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs/-/releases/v2.7.1) | @@ -710,7 +710,7 @@ documentation for instructions. ## Running SAST in SELinux -By default SAST analyzers are supported in GitLab instances hosted on SELinux. Adding a `before_script` in an [overriden SAST job](#overriding-sast-jobs) may not work as runners hosted on SELinux have restricted permissions. +By default SAST analyzers are supported in GitLab instances hosted on SELinux. Adding a `before_script` in an [overridden SAST job](#overriding-sast-jobs) may not work as runners hosted on SELinux have restricted permissions. ## Troubleshooting diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index f137ec26114..02d117b1c0a 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -118,7 +118,7 @@ To enable Secret Detection for GitLab 13.1 and later, you must include the `Secret-Detection.gitlab-ci.yml` template that's provided as a part of your GitLab installation. For GitLab versions earlier than 11.9, you can copy and use the job as defined in that template. -Add the following to your `.gitlab-ci.yml` file: +Ensure your `.gitlab-ci.yml` file has a `stage` called `test`, and add the following to your `.gitlab-ci.yml` file: ```yaml include: @@ -133,6 +133,31 @@ The results are saved as a that you can later download and analyze. Due to implementation limitations, we always take the latest Secret Detection artifact available. +### Enable Secret Detection via an automatic merge request **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4496) in GitLab 13.11. +> - [Deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. +> - Enabled on GitLab.com. +> - Recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-configure-secret-detection-via-a-merge-request). **(ULTIMATE SELF)** + +WARNING: +This feature might not be available to you. Check the **version history** note above for details. + +There can be +[risks when disabling released features](../../../user/feature_flags.md#risks-when-disabling-released-features). +Refer to this feature's version history for more details. + +To enable Secret Detection in a project, you can create a merge request +from the Security Configuration page. + +1. In the project where you want to enable Secret Detection, go to + **Security & Compliance > Configuration**. +1. In the **Secret Detection** row, select **Configure via Merge Request**. + +This automatically creates a merge request with the changes necessary to enable Secret Detection +that you can review and merge to complete the configuration. + ### Customizing settings The Secret Detection scan settings can be changed through [CI/CD variables](#available-variables) @@ -144,7 +169,7 @@ declare a job with the same name as the SAST job to override. Place this new job inclusion and specify any additional keys under it. WARNING: -Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic) +Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#only--except) is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead. #### GIT_DEPTH @@ -316,8 +341,8 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/secrets:3 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database) -with new definitions, so consider if you're able to make periodic updates yourself. +process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), @@ -380,3 +405,22 @@ secret_detection: variables: GIT_DEPTH: 100 ``` + +### Enable or disable Configure Secret Detection via a Merge Request + +Configure Secret Detection via a Merge Request is under development but ready for production use. +It is deployed behind a feature flag that is **enabled by default**. +[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) +can opt to disable it. + +To enable it: + +```ruby +Feature.enable(:sec_secret_detection_ui_enable) +``` + +To disable it: + +```ruby +Feature.disable(:sec_secret_detection_ui_enable) +``` diff --git a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png b/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png Binary files differdeleted file mode 100644 index c89179e739d..00000000000 --- a/doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/security_center_dashboard_link_v12_4.png b/doc/user/application_security/security_dashboard/img/security_center_dashboard_link_v12_4.png Binary files differdeleted file mode 100644 index e0e80810b08..00000000000 --- a/doc/user/application_security/security_dashboard/img/security_center_dashboard_link_v12_4.png +++ /dev/null diff --git a/doc/user/application_security/security_dashboard/img/security_center_settings_v13_4.png b/doc/user/application_security/security_dashboard/img/security_center_settings_v13_4.png Binary files differindex 4223494c294..74592e2cea5 100644 --- a/doc/user/application_security/security_dashboard/img/security_center_settings_v13_4.png +++ b/doc/user/application_security/security_dashboard/img/security_center_settings_v13_4.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 326c88f9eef..f0b3d895df5 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -73,18 +73,31 @@ CSV file containing details of the resources scanned. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235558) in GitLab 13.6. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285476) in GitLab 13.10, options to zoom in on a date range, and download the vulnerabilities chart. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285477) in GitLab 13.11, date range slider to visualise data between given dates. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285477) in GitLab 13.11, date range slider to visualize data between given dates. -At the project level, the Security Dashboard displays a chart with the number of vulnerabilities over time. -Access it by navigating to **Security & Compliance > Security Dashboard**. We display historical -data up to 365 days. The chart's data is updated daily. +A project's Security Dashboard displays a chart with the total number of vulnerabilities +over time. It updates daily with up to 365 days of historical data. By default, +it shows statistics for all vulnerability severities. + +To access the dashboard, from your project's home page go to **Security & Compliance > Security Dashboard**.  -Filter the historical data by clicking on the corresponding legend name. The image above, for example, shows -only the graph for vulnerabilities with **high** severity. +### Filter the vulnerabilities chart + +To filter the chart by vulnerability severity, select the corresponding legend name. + +In the previous example, the chart shows statistics only for vulnerabilities of medium or unknown severity. + +### Customize vulnerabilities chart display -To zoom in, select the left-most icon, then select the desired rangeby dragging across the chart. Select **Remove Selection** (**{redo}**) to reset to the original date range. +To customize the view of the vulnerability chart, you can select: + +- A specific time frame by using the time range handles (**{scroll-handle}**). +- A specific area of the chart by using the left-most icon (**{marquee-selection}**) then drag + across the chart. To reset to the original range, select **Remove Selection** (**{redo}**). + +### Download a copy of the vulnerabilities chart To download an SVG image of the chart, select **Save chart to an image** (**{download}**). @@ -137,10 +150,8 @@ the following:  -You can access the Security Center from the menu -bar at the top of the page. Under **More**, select **Security**. - - +To view the Security Center, from the navigation bar at the top of the page, select +**More > Security**. ### Adding projects to the Security Center @@ -198,4 +209,4 @@ Each scenario can be a third-level heading, e.g. `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> -Read more on how to [address the vulnerabilities](../index.md#addressing-vulnerabilities). +Read more on how to [address the vulnerabilities](../vulnerabilities/index.md). diff --git a/doc/user/application_security/terminology/index.md b/doc/user/application_security/terminology/index.md index e046b18b2a4..1316f1b9644 100644 --- a/doc/user/application_security/terminology/index.md +++ b/doc/user/application_security/terminology/index.md @@ -78,6 +78,8 @@ An asset that has the potential to be vulnerable, identified in a project by an include but are not restricted to source code, binary packages, containers, dependencies, networks, applications, and infrastructure. +Findings are all potential vulnerability items scanners identify in MRs/feature branches. Only after merging to default does a finding become a [vulnerability](#vulnerability). + ### Insignificant finding A legitimate finding that a particular customer doesn't care about. @@ -153,6 +155,8 @@ A flaw that has a negative impact on the security of its environment. Vulnerabil error or weakness, and don't describe where the error is located (see [finding](#finding)). Each vulnerability maps to a unique finding. +Vulnerabilities exist in the default branch. Findings (see [finding](#finding)) are all potential vulnerability items scanners identify in MRs/feature branches. Only after merging to default does a finding become a vulnerability. + ### Vulnerability finding When a [report finding](#report-finding) is stored to the database, it becomes a vulnerability diff --git a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_11.png b/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_11.png Binary files differdeleted file mode 100644 index 05df41483c4..00000000000 --- a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_11.png +++ /dev/null diff --git a/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_12.png b/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_12.png Binary files differnew file mode 100644 index 00000000000..1f02fd30f8e --- /dev/null +++ b/doc/user/application_security/threat_monitoring/img/threat_monitoring_policy_alert_list_v13_12.png diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index ced4669e241..825bc64d52b 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -209,19 +209,26 @@ Feature.disable(:threat_monitoring_alerts) > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3438) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.9. -The policy alert list displays your policy's alert activity. You can sort the list by the -**Date and time** column, and the **Status** column. Use the selector menu in the **Status** column -to set the status for each alert: +The policy alert list displays your policy's alert activity. You can sort the list by these columns: + +- Date and time +- Events +- Status + +You can filter the list with the **Policy Name** filter and the **Status** filter at the top. Use +the selector menu in the **Status** column to set the status for each alert: - Unreviewed - In review - Resolved - Dismissed -By default, the list doesn't display resolved or dismissed alerts. To show these alerts, clear the -checkbox **Hide dismissed alerts**. +By default, the list doesn't display resolved or dismissed alerts. + + - +Clicking an alert's row opens the alert drawer, which shows more information about the alert. A user +can also create an incident from the alert and update the alert status in the alert drawer. Clicking an alert's name takes the user to the [alert details page](../../../operations/incident_management/alerts.md#alert-details-page). diff --git a/doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png b/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4.png Binary files differindex 55694fc7926..55694fc7926 100644 --- a/doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png +++ b/doc/user/application_security/vulnerabilities/img/create_mr_from_vulnerability_v13_4.png diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index b98d28f8c9f..965b856504d 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Each security vulnerability in a project's [Vulnerability Report](../vulnerability_report/index.md) has an individual page which includes: - Details of the vulnerability. -- The status of the vulnerability within the project. +- The status of the vulnerability in the project. - Available actions for the vulnerability. - Any issues related to the vulnerability. @@ -21,8 +21,10 @@ On the vulnerability's page, you can: - [Change the vulnerability's status](#change-vulnerability-status). - [Create an issue](#create-an-issue-for-a-vulnerability). - [Link issues to the vulnerability](#link-gitlab-issues-to-the-vulnerability). -- [Automatically remediate the vulnerability](#automatically-remediate-the-vulnerability), if an +- [Remediate a vulnerability automatically](#remediate-a-vulnerability-automatically), if an automatic solution is available. +- [Remediate a vulnerability manually](#remediate-a-vulnerability-manually), if a solution is + available. ## Change vulnerability status @@ -60,7 +62,7 @@ To create a GitLab issue for a vulnerability: 1. In GitLab, go to the vulnerability's page. 1. Select **Create issue**. -An issue is created in the project, prepopulated with information from the vulnerability report. +An issue is created in the project, pre-populated with information from the vulnerability report. The issue is then opened so you can take further action. ### Create a Jira issue for a vulnerability @@ -120,7 +122,76 @@ that the resolution of one issue would resolve multiple vulnerabilities. Linked issues are shown in the Vulnerability Report and the vulnerability's page. -## Automatically remediate the vulnerability +## Link to an existing issue -You can fix some vulnerabilities by applying the solution that GitLab automatically -generates for you. [Read more about the automatic remediation for vulnerabilities feature](../index.md#apply-an-automatic-remediation-for-a-vulnerability). +If you already have an open issue, you can link to it from the vulnerability. + +- The vulnerability page shows related issues, but the issue page doesn't show the vulnerability it's related to. +- An issue can only be related to one vulnerability at a time. +- Issues can be linked across groups and projects. + +To link to an existing issue: + +1. Open the vulnerability. +1. [Add a linked issue](../../project/issues/related_issues.md). + +## Remediate a vulnerability automatically + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5656) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.7. + +Some vulnerabilities can be fixed by applying the solution that GitLab automatically generates. +The following scanners are supported: + +- [Dependency Scanning](../dependency_scanning/index.md). + Automatic Patch creation is only available for Node.js projects managed with + `yarn`. +- [Container Scanning](../container_scanning/index.md). + +### Remediate a vulnerability manually + +To manually apply the patch that GitLab generated for a vulnerability: + +1. Select the **Resolve with merge request** dropdown, then select **Download patch to resolve**: + +  + +1. Ensure your local project has the same commit checked out that was used to generate the patch. +1. Run `git apply remediation.patch`. +1. Verify and commit the changes to your branch. + +### Create a merge request with the suggested patch + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9224) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9. + +In some cases, you can create a merge request that automatically remediates the +vulnerability. Any vulnerability that has a +[solution](#remediate-a-vulnerability-automatically) can have a merge +request created to automatically solve the issue. + +If this action is available: + +1. Select the **Resolve with merge request** dropdown, then select **Resolve with merge request**. + +  + +A merge request is created. It applies the solution to the source branch. + +## Vulnerability scanner maintenance + +The following vulnerability scanners and their databases are regularly updated: + +| Secure scanning tool | Vulnerabilities database updates | +|:----------------------------------------------------------------|----------------------------------| +| [Container Scanning](../container_scanning/index.md) | Uses either `trivy` or `clair`. For the `trivy` scanner, a job runs on a daily basis to build a new image with the latest vulnerability database updates from the [upstream `trivy-db`](https://github.com/aquasecurity/trivy-db). For the `clair` scanner, the latest `clair-db` version is used; `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). | +| [Dependency Scanning](../dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | +| [Dynamic Application Security Testing (DAST)](../dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/master/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | +| [Static Application Security Testing (SAST)](../sast/index.md) | Relies exclusively on [the tools GitLab wraps](../sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | + +You do not have to update GitLab to benefit from the latest vulnerabilities definitions. +The security tools are released as Docker images. The vendored job definitions that enable them use +major release tags according to [semantic versioning](https://semver.org/). Each new release of the +tools overrides these tags. +The Docker images are updated to match the previous GitLab releases. Although +you automatically get the latest versions of the scanning tools, +there are some [known issues](https://gitlab.com/gitlab-org/gitlab/-/issues/9725) +with this approach. diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index 40b8f418737..75366a49a55 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -50,6 +50,7 @@ the following tables: | [`pmd-apex`](https://gitlab.com/gitlab-org/security-products/analyzers/pmd-apex) | **{check-circle}** Yes | Integer | `1`, `2`, `3`, `4`, `5` | | [`kubesec`](https://gitlab.com/gitlab-org/security-products/analyzers/kubesec) | **{check-circle}** Yes | String | `CriticalSeverity`, `InfoSeverity` | | [`secrets`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) | **{check-circle}** Yes | N/A | Hardcodes all severity levels to `Critical` | +| [`semgrep`](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) | **{check-circle}** Yes | String | `error`, `warning`, `note`, `none` | ## Dependency Scanning @@ -61,9 +62,10 @@ the following tables: ## Container Scanning -| GitLab analyzer | Outputs severity levels? | Native severity level type | Native severity level example | +| GitLab scanner | Outputs severity levels? | Native severity level type | Native severity level example | |------------------------------------------------------------------------|--------------------------|----------------------------|--------------------------------------------------------------| -| [`klar`](https://gitlab.com/gitlab-org/security-products/analyzers/klar) | **{check-circle}** Yes | String | `Negligible`, `Low`, `Medium`, `High`, `Critical`, `Defcon1` | +| [`clair`](https://gitlab.com/gitlab-org/security-products/analyzers/klar) | **{check-circle}** Yes | String | `Negligible`, `Low`, `Medium`, `High`, `Critical`, `Defcon1` | +| [`trivy`](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning)| **{check-circle}** Yes | String | `Unknown`, `Low`, `Medium`, `High`, `Critical` | ## Fuzz Testing diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 8f7740f9bfc..012992c8a72 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -162,3 +162,26 @@ computer. NOTE: It may take several minutes for the download to start if your project contains thousands of vulnerabilities. Don't close the page until the download finishes. + +## Dismiss a vulnerability + +> The option of adding a dismissal reason was introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. + +You can dismiss a vulnerability for the entire project: + +1. Select the vulnerability in the Security Dashboard. +1. In the top-right, from the **Status** selector menu, select **Dismissed**. +1. Optional. Add a reason for the dismissal and select **Save comment**. + +To undo this action, select a different status from the same menu. + +### Dismiss multiple vulnerabilities + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35816) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.9. + +You can dismiss multiple vulnerabilities at once: + +1. In the list of vulnerabilities, select the checkbox for each vulnerability you want to dismiss. + To select all, select the checkbox in the table header. +1. Above the table, select a dismissal reason. +1. Select **Dismiss Selected**. |
