From 85dc423f7090da0a52c73eb66faf22ddb20efff9 Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Sat, 19 Sep 2020 01:45:44 +0000 Subject: Add latest changes from gitlab-org/gitlab@13-4-stable-ee --- doc/user/application_security/api_fuzzing/index.md | 739 +++++++++++++++++++++ .../application_security/configuration/index.md | 40 +- .../container_scanning/index.md | 118 ++-- .../application_security/coverage_fuzzing/index.md | 8 +- doc/user/application_security/cve_id_request.md | 69 ++ doc/user/application_security/dast/index.md | 250 ++++--- .../dependency_scanning/analyzers.md | 27 +- .../dependency_scanning/index.md | 54 +- .../img/adding_a_dismissal_reason_v13_0.png | Bin 35841 -> 0 bytes .../img/adding_a_dismissal_reason_v13_4.png | Bin 0 -> 25574 bytes .../img/create_issue_from_vulnerability_v13_3.png | Bin 0 -> 5079 bytes .../img/create_issue_with_list_hover.png | Bin 36833 -> 0 bytes .../img/create_mr_from_vulnerability_v13_4.png | Bin 0 -> 33743 bytes .../img/cve_id_request_button.png | Bin 0 -> 5220 bytes .../img/cve_request_communication.png | Bin 0 -> 45402 bytes .../img/cve_request_communication_publication.png | Bin 0 -> 66617 bytes .../img/interacting_with_vulnerability_v13_0.png | Bin 29141 -> 0 bytes .../img/interacting_with_vulnerability_v13_3.png | Bin 0 -> 54508 bytes .../img/new_cve_request_issue.png | Bin 0 -> 96795 bytes ...urity_approval_rules_and_enabled_jobs_v13_4.png | Bin 0 -> 99883 bytes ...ured_security_approval_rules_and_jobs_v13_4.png | Bin 0 -> 82526 bytes .../img/vulnerability-check_v13_0.png | Bin 30789 -> 0 bytes .../img/vulnerability-check_v13_4.png | Bin 0 -> 75105 bytes doc/user/application_security/index.md | 113 ++-- .../offline_deployments/index.md | 10 +- doc/user/application_security/sast/analyzers.md | 32 +- doc/user/application_security/sast/index.md | 98 +-- .../application_security/secret_detection/index.md | 8 +- .../img/group_vulnerability_report_v13_3.png | Bin 36339 -> 0 bytes .../img/group_vulnerability_report_v13_4.png | Bin 0 -> 42099 bytes .../instance_security_dashboard_empty_v13_4.png | Bin 0 -> 38731 bytes ...nstance_security_dashboard_export_csv_v13_0.png | Bin 5563 -> 0 bytes ...nstance_security_dashboard_export_csv_v13_4.png | Bin 0 -> 10596 bytes .../img/instance_security_dashboard_v13_4.png | Bin 0 -> 62615 bytes ...e_security_dashboard_with_projects_v13_2_sm.png | Bin 58332 -> 0 bytes .../img/pipeline_security_dashboard_v13_2.png | Bin 73101 -> 0 bytes .../img/pipeline_security_dashboard_v13_3.png | Bin 0 -> 51201 bytes .../img/pipeline_security_v13_3.gif | Bin 548942 -> 0 bytes .../img/project_security_dashboard_v13_3.png | Bin 0 -> 168847 bytes .../img/vulnerability_list_table_v13_1.png | Bin 74381 -> 0 bytes .../img/vulnerability_list_table_v13_4.png | Bin 0 -> 79904 bytes .../security_dashboard/index.md | 30 +- doc/user/application_security/terminology/index.md | 171 +++++ .../threat_monitoring/index.md | 51 +- .../img/vulnerability_page_v13_1.png | Bin 41387 -> 0 bytes .../application_security/vulnerabilities/index.md | 34 +- 46 files changed, 1426 insertions(+), 426 deletions(-) create mode 100644 doc/user/application_security/api_fuzzing/index.md create mode 100644 doc/user/application_security/cve_id_request.md delete mode 100644 doc/user/application_security/img/adding_a_dismissal_reason_v13_0.png create mode 100644 doc/user/application_security/img/adding_a_dismissal_reason_v13_4.png create mode 100644 doc/user/application_security/img/create_issue_from_vulnerability_v13_3.png delete mode 100644 doc/user/application_security/img/create_issue_with_list_hover.png create mode 100644 doc/user/application_security/img/create_mr_from_vulnerability_v13_4.png create mode 100644 doc/user/application_security/img/cve_id_request_button.png create mode 100644 doc/user/application_security/img/cve_request_communication.png create mode 100644 doc/user/application_security/img/cve_request_communication_publication.png delete mode 100644 doc/user/application_security/img/interacting_with_vulnerability_v13_0.png create mode 100644 doc/user/application_security/img/interacting_with_vulnerability_v13_3.png create mode 100644 doc/user/application_security/img/new_cve_request_issue.png create mode 100644 doc/user/application_security/img/unconfigured_security_approval_rules_and_enabled_jobs_v13_4.png create mode 100644 doc/user/application_security/img/unconfigured_security_approval_rules_and_jobs_v13_4.png delete mode 100644 doc/user/application_security/img/vulnerability-check_v13_0.png create mode 100644 doc/user/application_security/img/vulnerability-check_v13_4.png delete mode 100644 doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_3.png create mode 100644 doc/user/application_security/security_dashboard/img/group_vulnerability_report_v13_4.png create mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_empty_v13_4.png delete mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_0.png create mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_export_csv_v13_4.png create mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_v13_4.png delete mode 100644 doc/user/application_security/security_dashboard/img/instance_security_dashboard_with_projects_v13_2_sm.png delete mode 100644 doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_2.png create mode 100644 doc/user/application_security/security_dashboard/img/pipeline_security_dashboard_v13_3.png delete mode 100644 doc/user/application_security/security_dashboard/img/pipeline_security_v13_3.gif create mode 100644 doc/user/application_security/security_dashboard/img/project_security_dashboard_v13_3.png delete mode 100644 doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_1.png create mode 100644 doc/user/application_security/security_dashboard/img/vulnerability_list_table_v13_4.png create mode 100644 doc/user/application_security/terminology/index.md delete mode 100644 doc/user/application_security/vulnerabilities/img/vulnerability_page_v13_1.png (limited to 'doc/user/application_security') diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md new file mode 100644 index 00000000000..ae22655e30b --- /dev/null +++ b/doc/user/application_security/api_fuzzing/index.md @@ -0,0 +1,739 @@ +--- +stage: Secure +group: Fuzz Testing +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference, howto +--- + +# Web API Fuzz Testing **(ULTIMATE)** + +You can add web API fuzzing to your [GitLab CI/CD](../../../ci/README.md) +pipelines. This helps you discover bugs and potential security issues that other QA processes may miss. +API fuzzing performs fuzz testing of API operation parameters. +Fuzz testing sets operation parameters to unexpected values in an effort to cause unexpected behavior and errors in the API backend. + +We recommend that you use fuzz testing in addition to [GitLab Secure](../index.md)'s +other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/README.md), +you can run fuzz tests as part your CI/CD workflow. + +## Requirements + +- One of the following web API types: + - REST API + - SOAP + - GraphQL + - Form bodies, JSON, or XML +- An OpenAPI definition, or HTTP Archive (HAR) of requests to test + +## When fuzzing scans run + +When using the `API-Fuzzing.gitlab-ci.yml` template, the `fuzz` job runs last, as shown here. To +ensure API fuzzing scans the latest code, your CI pipeline should deploy changes to a test +environment in one of the jobs preceding the `fuzz` job: + +```yaml +stages: + - build + - test + - deploy + - fuzz +``` + +Note that if your pipeline is configured to deploy to the same web server on each run, running a +pipeline while another is still running could cause a race condition in which one pipeline +overwrites the code from another. The API to scan should be excluded from changes for the duration +of a fuzzing scan. The only changes to the API should be from the fuzzing scanner. Be aware that +any changes made to the API (for example, by users, scheduled tasks, database changes, code +changes, other pipelines, or other scanners) during a scan could cause inaccurate results. + +## Configuration + +There are two ways to perform scans. See the configuration section for the one you wish to use: + +- [OpenAPI v2 specification](#openapi-specification) +- [HTTP Archive (HAR)](#http-archive-har) + +Examples of both configurations can be found here: + +- [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/openapi) +- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har) + +### OpenAPI Specification + +The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an +API description format for REST APIs. This section shows you how to configure API fuzzing by using +an OpenAPI specification to provide information about the target API to test. OpenAPI specifications +are provided as a filesystem resource or URL. + +Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: + +1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + ``` + +1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. + +1. The [configuration file](#configuration-files) has several testing profiles defined with varying + amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this + profile completes quickly, allowing for easier configuration validation. + + Provide the profile by adding the `FUZZAPI_PROFILE` variable to your `.gitlab-ci.yml` file, + substituting `Quick-10` for the profile you choose: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + ``` + +1. Provide the location of the OpenAPI v2 specification. You can provide the specification as a file + or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + ``` + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `FUZZAPI_TARGET_URL`: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +DANGER: **Danger:** +**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run fuzzing against a test server. + +### HTTP Archive (HAR) + +The [HTTP Archive format (HAR)](http://www.softwareishard.com/blog/har-12-spec/) +is an archive file format for logging HTTP transactions. When used with GitLab's API fuzzer, HAR +must contain records of calling the web API to test. The API fuzzer extracts all the requests and +uses them to perform testing. + +You can use various tools to generate HAR files: + +- [Fiddler](https://www.telerik.com/fiddler): Web debugging proxy +- [Insomnia Core](https://insomnia.rest/): API client +- [Chrome](https://www.google.com/chrome): Browser +- [Firefox](https://www.mozilla.org/en-US/firefox/): Browser + +DANGER: **Warning:** +HAR files may contain sensitive information such as authentication tokens, API keys, and session +cookies. We recommend that you review the HAR file contents before adding them to a repository. + +Follow these steps to configure API fuzzing to use a HAR file that provides information about the +target API to test: + +1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) + the [`API-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) + that's provided as part of your GitLab installation. To do so, add the following to your + `.gitlab-ci.yml` file: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + ``` + +1. Add the configuration file [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml) to your repository's root as `.gitlab-api-fuzzing.yml`. + +1. The [configuration file](#configuration-files) has several testing profiles defined with varying + amounts of fuzzing. We recommend that you start with the `Quick-10` profile. Testing with this + profile completes quickly, allowing for easier configuration validation. + + Provide the profile by adding the `FUZZAPI_PROFILE` variable to your `.gitlab-ci.yml` file, + substituting `Quick-10` for the profile you choose: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + ``` + +1. Add the `FUZZAPI_HAR` variable and set it to the HAR file's location: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + ``` + +1. The target API instance's base URL is also required. Provide it by using the `FUZZAPI_TARGET_URL` + variable or an `environment_url.txt` file. + + Adding the URL in an `environment_url.txt` file at your project's root is great for testing in + dynamic environments. To run API fuzzing against an app dynamically created during a GitLab CI/CD + pipeline, have the app persist its domain in an `environment_url.txt` file. API fuzzing + automatically parses that file to find its scan target. You can see an + [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + + Here's an example of using `FUZZAPI_TARGET_URL`: + + ```yaml + include: + - template: API-Fuzzing.gitlab-ci.yml + + variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + FUZZAPI_TARGET_URL: http://test-deployment/ + ``` + +This is a minimal configuration for API Fuzzing. From here you can: + +- [Run your first scan](#running-your-first-scan). +- [Add authentication](#authentication). +- Learn how to [handle false positives](#handling-false-positives). + +DANGER: **Danger:** +**NEVER** run fuzz testing against a production server. Not only can it perform *any* function that +the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting +data. Only run fuzzing against a test server. + +### Authentication + +Authentication is handled by providing the authentication token as a header or cookie. You can +provide a script that performs an authentication flow or calculates the token. + +#### HTTP Basic Authentication + +[HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) +is an authentication method built into the HTTP protocol and used in-conjunction with +[transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). +To use HTTP basic authentication, two variables are added to your `.gitlab-ci.yml` file: + +- `FUZZAPI_HTTP_USERNAME`: The username for authentication. +- `FUZZAPI_HTTP_PASSWORD`: The password for authentication. + +For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui) +(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the +GitLab projects page at **Settings > CI/CD**, in the **Variables** section. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_HAR: test-api-recording.har + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_HTTP_USERNAME: testuser + FUZZAPI_HTTP_PASSWORD: $TEST_API_PASSWORD + +``` + +#### Bearer Tokens + +Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web +Tokens (JWT). Bearer tokens are transmitted using the `Authorization` HTTP header. To use bearer +tokens with API fuzzing, you need one of the following: + +- A token that doesn't expire +- A way to generate a token that lasts the length of testing +- A Python script that API fuzzing can call to generate the token + +##### Token doesn't expire + +If the bearer token doesn't expire, you can provide it using the `FUZZAPI_OVERRIDES_ENV` variable. +The `FUZZAPI_OVERRIDES_ENV` content is a JSON snippet that provides headers and cookies that should +be added to outgoing HTTP requests made by API fuzzing. + +Create a CI/CD variable, for example `TEST_API_BEARERAUTH`, with the value +`{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can +create CI/CD variables from the GitLab projects page at **Settings > CI/CD** in the **Variables** +section. + +Set `FUZZAPI_OVERRIDES_ENV` in your `.gitlab-ci.yml` file: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: $TEST_API_BEARERAUTH +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +##### Token generated at test-runtime + +If the bearer token must be generated, and the resulting token doesn't expire during testing, you +can provide to API fuzzing a file containing the token. This file can be generated by a prior stage +and job, or as part of the API fuzzing job. + +API fuzzing expects to receive a JSON file with the following structure: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +This file can be generated by a prior stage and provided to API fuzzing through the +`FUZZAPI_OVERRIDES_FILE` variable. + +Set `FUZZAPI_OVERRIDES_FILE` in your `.gitlab-ci.yml` file: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +##### Token has short expiration + +If the bearer token must be generated and expires prior to the scan's completion, you can provide a +program or script for the API fuzzer to execute on a provided interval. The provided script runs in +an Alpine Linux container that has Python 3 and Bash installed. If the Python script requires +additional packages, it must detect this and install the packages at runtime. + +The script must create a JSON file containing the bearer token in a specific format: + +```json +{ + "headers" : { + "Authorization" : "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +You must provide three variables, each set for correct operation: + +- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. +- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. +- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_CMD: renew_token.py + FUZZAPI_OVERRIDES_INTERVAL: 300 +``` + +To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and +the test API's application logs. + +### Configuration files + +To get started quickly, GitLab provides you with the configuration file +[`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml). +This file has several testing profiles that perform various amounts of testing. The run time of each +increases as the numbers go up. To use a configuration file, add it to your repository's root as +`.gitlab-api-fuzzing.yml`. + +| Profile | Scan Type | +|:---------|:-----------| +|Quick-10 |Fuzzing 10 times per parameter | +|Medium-20 |Fuzzing 20 times per parameter | +|Medium-50 |Fuzzing 50 times per parameter | +|Long-100 |Fuzzing 100 times per parameter | + +### Available variables + +| Environment variable | Description | +|-----------------------------|--------------------| +| `FUZZAPI_VERSION` |Specify API Fuzzing container version. Defaults to `latest`. | +| `FUZZAPI_TARGET_URL` |Base URL of API testing target. | +|[`FUZZAPI_CONFIG`](#configuration-files)|API Fuzzing configuration file. Defaults to `.gitlab-apifuzzer.yml`. | +|[`FUZZAPI_PROFILE`](#configuration-files)|Configuration profile to use during testing. Defaults to `Quick`. | +| `FUZZAPI_REPORT` |Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. | +|[`FUZZAPI_OPENAPI`](#openapi-specification)|OpenAPI specification file or URL. | +|[`FUZZAPI_HAR`](#http-archive-har)|HTTP Archive (HAR) file. | +|[`FUZZAPI_OVERRIDES_FILE`](#overrides) |Path to a JSON file containing overrides. | +|[`FUZZAPI_OVERRIDES_ENV`](#overrides) |JSON string containing headers to override. | +|[`FUZZAPI_OVERRIDES_CMD`](#overrides) |Overrides command. | +|[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) |How often to run overrides command in seconds. Defaults to `0` (once). | +|[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) |Username for HTTP authentication. | +|[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) |Password for HTTP authentication. | + + + +### Overrides + +API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests +made. You can use this to inject semver headers, authentication, and so on. The +[authentication section](#authentication) includes examples of using overrides for that purpose. + +Overrides uses a JSON document to define the headers and cookies: + +```json +{ + "headers": { + "header1": "value", + "header2": "value" + }, + "cookies": { + "cookie1": "value", + "cookie2": "value" + } +} +``` + +Example usage for setting a single header: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + } +} +``` + +Example usage for setting both a header and cookie: + +```json +{ + "headers": { + "Authorization": "Bearer dXNlcm5hbWU6cGFzc3dvcmQ=" + }, + "cookies": { + "flags": "677" + } +} +``` + +You can provide this JSON document as a file or environment variable. You may also provide a command +to generate the JSON document. The command can run at intervals to support values that expire. + +#### Using a file + +To provide the overrides JSON as a file, the `FUZZAPI_OVERRIDES_FILE` environment variable is set. The path is relative to the job current working directory. + +Example `.gitlab-ci.yml`: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json +``` + +#### Using an environment variable + +To provide the overrides JSON as an environment variable, use the `FUZZAPI_OVERRIDES_ENV` variable. +This allows you to place the JSON as CI/CD variables that can be masked and protected. + +In this example `.gitlab-ci.yml`, the JSON is provided directly: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: '{"headers":{"X-API-Version":"2"}}' +``` + +In this example `.gitlab-ci.yml`, the CI/CD variable `SECRET_OVERRIDES` provides the JSON. This is a +[group or instance level environment variable defined in the UI](../../../ci/variables/README.md#instance-level-cicd-environment-variables): + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_ENV: $SECRET_OVERRIDES +``` + +#### Using a command + +If the value must be generated or regenerated on expiration, you can provide a program or script for +the API fuzzer to execute on a specified interval. The provided script runs in an Alpine Linux +container that has Python 3 and Bash installed. If the Python script requires additional packages, +it must detect this and install the packages at runtime. The script creates the overrides JSON file +as defined above. + +You must provide three variables, each set for correct operation: + +- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. +- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. +- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_CMD: renew_token.py + FUZZAPI_OVERRIDES_INTERVAL: 300 +``` + +## Running your first scan + +When configured correctly, a CI/CD pipeline contains a `Fuzz` stage and a `apifuzzer_fuzz` job. The +job only fails when an invalid configuration is provided. During normal operation, the job always +succeeds even if faults are identified during fuzz testing. + +Faults are displayed on the **Tests** pipeline tab with the suite name **API-Fuzzing**. The **Name** +field on the **Tests** page includes the fuzz-tested operation and parameter. The **Trace** field +contains a writeup of the identified fault. This writeup contains information on what the fuzzer +tested and how it detected something wrong. + +To prevent an excessive number of reported faults, the API fuzzing scanner limits the number of +faults it reports to one per parameter. + +### Fault Writeup + +The faults that API fuzzing finds aren't associated with a specific vulnerability type. They require +investigation to determine what type of issue they are and if they should be fixed. See +[handling false positives](#handling-false-positives) for information about configuration changes +you can make to limit the number of false positives reported. + +This table contains a description of fields in an API fuzzing fault writeup. + +| Writeup Item | Description | +|:-------------|:------------| +| Operation | The operation tested. | +| Parameter | The field modified. This can be a path segment, header, query string, or body element. | +| Endpoint | The endpoint being tested. | +| Check | Check module producing the test. Checks can be turned on and off. | +| Assert | Assert module that detected a failure. Assertions can be configured and turned on and off. | +| CWE | Fuzzing faults always have the same CWE. | +| OWASP | Fuzzing faults always have the same OWASP ID. | +| Exploitability | Fuzzing faults always have an `unknown` exploitability. | +| Impact | Fuzzing faults always have an `unknown` risk impact. | +| Description | Verbose description of what the check did. Includes the original parameter value and the modified (mutated) value. | +| Detection | Why a failure was detected and reported. This is related to the Assert that was used. | +| Original Request | The original, unmodified HTTP request. Useful when reviewing the actual request to see what changes were made. | +| Actual Request | The request that produced the failure. This request has been modified in some way by the Check logic. | +| Actual Response | The response to the actual request. | +| Recorded Request | An unmodified request. | +| Recorded Response | The response to the unmodified request. You can compare this with the actual request when triaging this fault. | + +## Handling False Positives + +False positives can be handled in two ways: + +- Turn off the Check producing the false positive. This prevents the check from generating any + faults. Example checks are the JSON Fuzzing Check, and Form Body Fuzzing Check. +- Fuzzing checks have several methods of detecting when a fault is identified, called _Asserts_. + Asserts can also be turned off and configured. For example, the API fuzzer by default uses HTTP + status codes to help identify when something is a real issue. If an API returns a 500 error during + testing, this creates a fault. This isn't always desired, as some frameworks return 500 errors + often. + +### Turn off a Check + +Checks perform testing of a specific type and can be turned on and off for specific configuration +profiles. The provided [configuration files](#configuration-files) define several profiles that you +can use. The profile definition in the configuration file lists all the checks that are active +during a scan. To turn off a specific check, simply remove it from the profile definition in the +configuration file. The profiles are defined in the `Profiles` section of the configuration file. + +Example profile definition: + +```yaml +Profiles: + - Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +To turn off the General Fuzzing Check you can remove these lines: + +```yaml +- Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +This results in the following YAML: + +```yaml +- Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + +### Turn off an Assertion for a Check + +Assertions detect faults in tests produced by checks. Many checks support multiple Assertions such +as Log Analysis, Response Analysis, and Status Code. When a fault is found, the Assertion used is +provided. To identify which Assertions are on by default, see the Checks default configuration in +the configuration file. The section is called `Checks`. + +This example shows the FormBody Fuzzing Check: + +```yaml +Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 30 + UnicodeFuzzing: true + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: StatusCodeAssertion +``` + +Here you can see three Assertions are on by default. A common source of false positives is +`StatusCodeAssertion`. To turn it off, modify its configuration in the `Profiles` section. This +example provides only the other two Assertions (`LogAnalysisAssertion`, +`ResponseAnalysisAssertion`). This prevents `FormBodyFuzzingCheck` from using `StatusCodeAssertion`: + +```yaml +Profiles: + - Name: Quick-10 + DefaultProfile: Quick + Routes: + - Route: *Route0 + Checks: + - Name: FormBodyFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + Assertions: + - Name: LogAnalysisAssertion + - Name: ResponseAnalysisAssertion + - Name: GeneralFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: JsonFuzzingCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true + - Name: XmlInjectionCheck + Configuration: + FuzzingCount: 10 + UnicodeFuzzing: true +``` + + + +## Glossary + +- Assert: Assertions are detection modules used by checks to trigger a fault. Many assertions have + configurations. A check can use multiple Assertions. For example, Log Analysis, Response Analysis, + and Status Code are common Assertions used together by checks. Checks with multiple Assertions + allow them to be turned on and off. +- Check: Performs a specific type of test, or performed a check for a type of vulnerability. For + example, the JSON Fuzzing Check performs fuzz testing of JSON payloads. The API fuzzer is + comprised of several checks. Checks can be turned on and off in a profile. +- Fault: During fuzzing, a failure identified by an Assert is called a fault. Faults are + investigated to determine if they are a security vulnerability, a non-security issue, or a false + positive. Faults don't have a known vulnerability type until they are investigated. Example + vulnerability types are SQL Injection and Denial of Service. +- Profile: A configuration file has one or more testing profiles, or sub-configurations. You may + have a profile for feature branches and another with extra testing for a main branch. diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 1195d07d7b7..a6ad701360e 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -7,23 +7,41 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Security Configuration **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20711) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.6. +> - SAST configuration was [enabled](https://gitlab.com/groups/gitlab-org/-/epics/3659) in 13.3 and [improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in 13.4. +> - DAST Profiles feature was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40474) in 13.4. -The Security Configuration page displays the configuration state of each security feature in the -current project. The page uses the project's latest default branch [CI pipeline](../../../ci/pipelines/index.md) -to determine each feature's configuration state. If a job with the expected security report artifact -exists in the pipeline, the feature is considered enabled. +The Security Configuration page displays the configuration state of each security control in the +current project. -You can only enable SAST from the Security Configuration page. Documentation links are included for -the other features. For details about configuring SAST, see [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). +To view a project's security configuration, go to the project's home page, +then in the left sidebar go to **Security & Compliance > Configuration**. + +For each security control the page displays: + +- **Status** - Status of the security control: enabled, not enabled, or available. +- **Manage** - A management option or a link to the documentation. + +## Status + +The status of each security control is determined by the project's latest default branch +[CI pipeline](../../../ci/pipelines/index.md). +If a job with the expected security report artifact exists in the pipeline, the feature's status is +_enabled_. + +For SAST, click **View history** to see the `.gitlab-ci.yml` fileโ€™s history. NOTE: **Note:** If the latest pipeline used [Auto DevOps](../../../topics/autodevops/index.md), all security features are configured by default. -## View Security Configuration +## Manage -To view a project's security configuration: +You can configure the following security controls: -1. Go to the project's home page. -1. In the left sidebar, go to **Security & Configuration** > **Configuration**. +- Auto DevOps + - Click **Enable Auto DevOps** to enable it for the current project. For more details, see [Auto DevOps](../../../topics/autodevops/index.md). +- SAST + - Click either **Enable** or **Configure** to use SAST for the current project. For more details, see [Configure SAST in the UI](../sast/index.md#configure-sast-in-the-ui). +- DAST Profiles + - Click **Manage** to manage the available DAST profiles used for on-demand scans. For more details, see [DAST on-demand scans](../dast/index.md#on-demand-scans). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 6b7086ddc71..880e5a3875a 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -26,7 +26,7 @@ To integrate security scanners other than Clair and Klar into GitLab, see You can enable container scanning by doing one of the following: - [Include the CI job](#configuration) in your existing `.gitlab-ci.yml` file. -- Implicitly use [Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning-ultimate) +- Implicitly use [Auto Container Scanning](../../../topics/autodevops/stages.md#auto-container-scanning) provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab compares the found vulnerabilities between the source and target branches, and shows the @@ -40,12 +40,12 @@ information directly in the merge request. ## Requirements -To enable Container Scanning in your pipeline, you need the following: +To enable container scanning in your pipeline, you need the following: -- [GitLab Runner](https://docs.gitlab.com/runner/) with the [Docker](https://docs.gitlab.com/runner/executors/docker.html) - or [Kubernetes](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -- Docker `18.09.03` or higher installed on the same computer as the Runner. If you're using the - shared Runners on GitLab.com, then this is already the case. +- [GitLab Runner](https://docs.gitlab.com/runner/) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) + or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. +- Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the + shared runners on GitLab.com, then this is already the case. - [Build and push](../../packages/container_registry/index.md#container-registry-examples-with-gitlab-cicd) your Docker image to your project's container registry. The name of the Docker image should use the following [predefined environment variables](../../../ci/variables/predefined_variables.md): @@ -72,7 +72,7 @@ To enable Container Scanning in your pipeline, you need the following: ## Configuration -How you enable Container Scanning depends on your GitLab version: +How you enable container scanning depends on your GitLab version: - GitLab 11.9 and later: [Include](../../../ci/yaml/README.md#includetemplate) the [`Container-Scanning.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml) @@ -91,16 +91,16 @@ include: The included template: - Creates a `container_scanning` job in your CI/CD pipeline. -- Pulls the built Docker image from your project's [Container Registry](../../packages/container_registry/index.md) +- Pulls the built Docker image from your project's [container registry](../../packages/container_registry/index.md) (see [requirements](#requirements)) and scans it for possible vulnerabilities. GitLab saves the results as a -[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning-ultimate) +[Container Scanning report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscontainer_scanning) that you can download and analyze later. When downloading, you always receive the most-recent artifact. -The following is a sample `.gitlab-ci.yml` that builds your Docker image, pushes it to the Container -Registry, and scans the containers: +The following is a sample `.gitlab-ci.yml` that builds your Docker image, pushes it to the container +registry, and scans the containers: ```yaml variables: @@ -127,7 +127,7 @@ include: - template: Container-Scanning.gitlab-ci.yml ``` -### Customizing the Container Scanning settings +### Customizing the container scanning settings There may be cases where you want to customize how GitLab scans your containers. For example, you may want to enable more verbose output from Clair or Klar, access a Docker registry that requires @@ -136,7 +136,7 @@ parameter in your `.gitlab-ci.yml` to set [environment variables](#available-var The environment variables you set in your `.gitlab-ci.yml` overwrite those in `Container-Scanning.gitlab-ci.yml`. -This example [includes](../../../ci/yaml/README.md#include) the Container Scanning template and +This example [includes](../../../ci/yaml/README.md#include) the container scanning template and enables verbose output from Clair by setting the `CLAIR_OUTPUT` environment variable to `High`: ```yaml @@ -153,30 +153,30 @@ variables: #### Available variables -Container Scanning can be [configured](#customizing-the-container-scanning-settings) -using environment variables. - -| Environment Variable | Default | Description | -| -------------------- | ----------- | ------- | -| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | -| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from klar. | -| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the clair server process. | -| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | -| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | -| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | -| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | -| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | -| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. | -| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone Container Scanning Tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | -| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | -| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | -| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | -| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | -| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | -| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. | -| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | - -### Overriding the Container Scanning template +You can [configure](#customizing-the-container-scanning-settings) container +scanning by using the following environment variables: + +| Environment Variable | Default | Description | +| ------------------------------ | ------------- | ----------- | +| `ADDITIONAL_CA_CERT_BUNDLE` | `""` | Bundle of CA certs that you want to trust. | +| `CLAIR_DB_CONNECTION_STRING` | `postgresql://postgres:password@clair-vulnerabilities-db:5432/postgres?sslmode=disable&statement_timeout=60000` | This variable represents the [connection string](https://www.postgresql.org/docs/9.3/libpq-connect.html#AEN39692) to the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) database and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. The host value for the connection string must match the [alias](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) value of the `Container-Scanning.gitlab-ci.yml` template file, which defaults to `clair-vulnerabilities-db`. | +| `CLAIR_DB_IMAGE` | `arminc/clair-db:latest` | The Docker image name and tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes, or to refer to a locally hosted vulnerabilities database for an on-premise offline installation. | +| `CLAIR_DB_IMAGE_TAG` | `latest` | (**DEPRECATED - use `CLAIR_DB_IMAGE` instead**) The Docker image tag for the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db). It can be useful to override this value with a specific version, for example, to provide a consistent set of vulnerabilities for integration testing purposes. | +| `CLAIR_OUTPUT` | `Unknown` | Severity level threshold. Vulnerabilities with severity level higher than or equal to this threshold are outputted. Supported levels are `Unknown`, `Negligible`, `Low`, `Medium`, `High`, `Critical` and `Defcon1`. | +| `CLAIR_TRACE` | `"false"` | Set to true to enable more verbose output from the clair server process. | +| `CLAIR_VULNERABILITIES_DB_URL` | `clair-vulnerabilities-db` | (**DEPRECATED - use `CLAIR_DB_CONNECTION_STRING` instead**) This variable is explicitly set in the [services section](https://gitlab.com/gitlab-org/gitlab/-/blob/898c5da43504eba87b749625da50098d345b60d6/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L23) of the `Container-Scanning.gitlab-ci.yml` file and defaults to `clair-vulnerabilities-db`. This value represents the address that the [PostgreSQL server hosting the vulnerabilities definitions](https://hub.docker.com/r/arminc/clair-db) is running on and **shouldn't be changed** unless you're running the image locally as described in the [Running the standalone container scanning tool](#running-the-standalone-container-scanning-tool) section. | +| `CI_APPLICATION_REPOSITORY` | `$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG` | Docker repository URL for the image to be scanned. | +| `CI_APPLICATION_TAG` | `$CI_COMMIT_SHA` | Docker repository tag for the image to be scanned. | +| `DOCKER_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access secure Docker registries using HTTPS with bad (or self-signed) SSL certificates. | +| `DOCKER_PASSWORD` | `$CI_REGISTRY_PASSWORD` | Password for accessing a Docker registry requiring authentication. | +| `DOCKER_USER` | `$CI_REGISTRY_USER` | Username for accessing a Docker registry requiring authentication. | +| `DOCKERFILE_PATH` | `Dockerfile` | The path to the `Dockerfile` to be used for generating remediations. By default, the scanner looks for a file named `Dockerfile` in the root directory of the project, so this variable should only be configured if your `Dockerfile` is in a non-standard location, such as a subdirectory. See [Solutions for vulnerabilities](#solutions-for-vulnerabilities-auto-remediation) for more details. | +| `KLAR_TRACE` | `"false"` | Set to true to enable more verbose output from klar. | +| `REGISTRY_INSECURE` | `"false"` | Allow [Klar](https://github.com/optiopay/klar) to access insecure registries (HTTP only). Should only be set to `true` when testing the image locally. | +| `SECURE_ANALYZERS_PREFIX` | `"registry.gitlab.com/gitlab-org/security-products/analyzers"` | Set the Docker registry base address from which to download the analyzer. | +| `SECURE_LOG_LEVEL` | `info` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. | + +### Overriding the container scanning template If you want to override the job definition (for example, to change properties like `variables`), you must declare a `container_scanning` job after the template inclusion, and then @@ -201,35 +201,35 @@ instead. To allowlist specific vulnerabilities, follow these steps: 1. Set `GIT_STRATEGY: fetch` in your `.gitlab-ci.yml` file by following the instructions in - [overriding the Container Scanning template](#overriding-the-container-scanning-template). + [overriding the container scanning template](#overriding-the-container-scanning-template). 1. Define the allowlisted vulnerabilities in a YAML file named `vulnerability-allowlist.yml`. This must use the format described in the [allowlist example file](https://gitlab.com/gitlab-org/security-products/analyzers/klar/-/raw/master/testdata/vulnerability-allowlist.yml). 1. Add the `vulnerability-allowlist.yml` file to your project's Git repository. -### Running Container Scanning in an offline environment +### Running container scanning in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access -to external resources through the internet, some adjustments are required for the Container Scanning job to +to external resources through the internet, some adjustments are required for the container scanning job to successfully run. For more information, see [Offline environments](../offline_deployments/index.md). -#### Requirements for offline Container Scanning +#### Requirements for offline container Scanning -To use Container Scanning in an offline environment, you need: +To use container scanning in an offline environment, you need: - GitLab Runner with the [`docker` or `kubernetes` executor](#requirements). -- To configure a local Docker Container Registry with copies of the Container Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [Container Scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). +- To configure a local Docker container registry with copies of the container scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [container scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry). NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. -#### Make GitLab Container Scanning analyzer images available inside your Docker registry +#### Make GitLab container scanning analyzer images available inside your Docker registry -For Container Scanning, import the following default images from `registry.gitlab.com` into your +For container scanning, import the following default images from `registry.gitlab.com` into your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext @@ -249,7 +249,7 @@ For details on saving and transporting Docker images as a file, see Docker's doc [`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/), [`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/). -#### Set Container Scanning CI job variables to use local Container Scanner analyzers +#### Set container scanning CI job variables to use local container scanner analyzers 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry: @@ -267,10 +267,10 @@ For details on saving and transporting Docker images as a file, see Docker's doc self-signed certificate, then you must set `DOCKER_INSECURE: "true"` in the above `container_scanning` section of your `.gitlab-ci.yml`. -#### Automating Container Scanning vulnerability database updates with a pipeline +#### Automating container scanning vulnerability database updates with a pipeline It can be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to -automatically build a new version of the vulnerabilities database on a preset schedule. Automating +build a new version of the vulnerabilities database on a preset schedule. Automating this with a pipeline means you won't have to do it manually each time. You can use the following `.gitlab-yml.ci` as a template: @@ -293,9 +293,9 @@ build_latest_vulnerabilities: The above template works for a GitLab Docker registry running on a local installation, however, if you're using a non-GitLab Docker registry, you'll need to change the `$CI_REGISTRY` value and the `docker login` credentials to match the details of your local registry. -## Running the standalone Container Scanning Tool +## Running the standalone container scanning tool -It's possible to run the [GitLab Container Scanning Tool](https://gitlab.com/gitlab-org/security-products/analyzers/klar) +It's possible to run the [GitLab container scanning tool](https://gitlab.com/gitlab-org/security-products/analyzers/klar) against a Docker container without needing to run it within the context of a CI job. To scan an image directly, follow these steps: @@ -329,10 +329,10 @@ The results are stored in `gl-container-scanning-report.json`. ## Reports JSON format -The Container Scanning tool emits a JSON report file. For more information, see the +The container scanning tool emits a JSON report file. For more information, see the [schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/container-scanning-report-format.json). -Here's an example Container Scanning report: +Here's an example container scanning report: ```json-doc { @@ -401,7 +401,7 @@ For more information about the vulnerabilities database update, check the ## Interacting with the vulnerabilities -Once a vulnerability is found, you can [interact with it](../index.md#interacting-with-the-vulnerabilities). +After a vulnerability is found, you can [interact with it](../index.md#interacting-with-the-vulnerabilities). ## Solutions for vulnerabilities (auto-remediation) @@ -413,7 +413,7 @@ the [`DOCKERFILE_PATH`](#available-variables) environment variable. To ensure th has access to this file, it's necessary to set [`GIT_STRATEGY: fetch`](../../../ci/yaml/README.md#git-strategy) in your `.gitlab-ci.yml` file by following the instructions described in this document's -[overriding the Container Scanning template](#overriding-the-container-scanning-template) section. +[overriding the container scanning template](#overriding-the-container-scanning-template) section. Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vulnerabilities-auto-remediation). @@ -421,8 +421,8 @@ Read more about the [solutions for vulnerabilities](../index.md#solutions-for-vu ### `docker: Error response from daemon: failed to copy xattrs` -When the GitLab Runner uses the Docker executor and NFS is used -(for example, `/var/lib/docker` is on an NFS mount), Container Scanning might fail with +When the runner uses the `docker` executor and NFS is used +(for example, `/var/lib/docker` is on an NFS mount), container scanning might fail with an error like the following: ```plaintext @@ -430,6 +430,6 @@ docker: Error response from daemon: failed to copy xattrs: failed to set xattr " ``` This is a result of a bug in Docker which is now [fixed](https://github.com/containerd/continuity/pull/138 "fs: add WithAllowXAttrErrors CopyOpt"). -To prevent the error, ensure the Docker version that the Runner is using is +To prevent the error, ensure the Docker version that the runner is using is `18.09.03` or higher. For more information, see [issue #10241](https://gitlab.com/gitlab-org/gitlab/-/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 1672e9fbb25..dff71cb9445 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -25,9 +25,11 @@ Docker image with the fuzz engine to run your app. | Language | Fuzzing Engine | Example | |----------|----------------|---------| -| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | [c-cpp-example](https://gitlab.com/gitlab-org/security-products/demos/c-cpp-fuzzing-example) | -| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example) | -| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | | +| C/C++ | [libFuzzer](https://llvm.org/docs/LibFuzzer.html) | [c-cpp-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/c-cpp-fuzzing-example) | +| GoLang | [go-fuzz (libFuzzer support)](https://github.com/dvyukov/go-fuzz) | [go-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example) | +| Swift | [libfuzzer](https://github.com/apple/swift/blob/master/docs/libFuzzerIntegration.md) | [swift-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/swift-fuzzing-example) | +| Rust | [cargo-fuzz (libFuzzer support)](https://github.com/rust-fuzz/cargo-fuzz) | [rust-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/rust-fuzzing-example) | +| Java | [JQF](https://github.com/rohanpadhye/JQF) | [java-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/java-fuzzing-example) | ## Configuration diff --git a/doc/user/application_security/cve_id_request.md b/doc/user/application_security/cve_id_request.md new file mode 100644 index 00000000000..94cacf2882f --- /dev/null +++ b/doc/user/application_security/cve_id_request.md @@ -0,0 +1,69 @@ +--- +type: tutorial +stage: Secure +group: Vulnerability Research +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +--- + +# CVE ID Requests + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41203) in GitLab 13.4, only for public projects on GitLab.com. + +As part of [GitLab's role as a CVE Numbering Authority](https://about.gitlab.com/security/cve) +([CNA](https://cve.mitre.org/cve/cna.html)), you may request +[CVE](https://cve.mitre.org/index.html) identifiers from GitLab to track +vulnerabilities found within your project. + +## Overview + +CVE identifiers track specific vulnerabilities within projects. Having a CVE assigned to a +vulnerability in your project helps your users stay secure and informed. For example, +[dependency scanning tools](../application_security/dependency_scanning/index.md) +can detect when vulnerable versions of your project are used as a dependency. + +## Conditions + +If the following conditions are met, a **Request CVE ID** button appears in your issue sidebar: + +- The project is hosted in GitLab.com. +- The project is public. +- You are a maintainer of the project. +- The issue is confidential. + +## Submitting a CVE ID Request + +Clicking the **Request CVE ID** button in the issue sidebar takes you to the new issue page for +[GitLab's CVE project](https://gitlab.com/gitlab-org/cves). + +![CVE ID request button](img/cve_id_request_button.png) + +Creating the confidential issue starts the CVE request process. + +![New CVE ID request issue](img/new_cve_request_issue.png) + +You are required to fill in the issue description, which includes: + +- A description of the vulnerability +- The project's vendor and name +- Impacted versions +- Fixed versions +- The vulnerability type (a [CWE](https://cwe.mitre.org/data/index.html) identifier) +- A [CVSS v3 vector](https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator) + +## CVE Assignment + +GitLab triages your submitted CVE ID request and communicates with you throughout the CVE validation +and assignment process. + +![CVE ID request communication](img/cve_request_communication.png) + +Once a CVE identifier is assigned, you may use and reference it as you see fit. + +Details of the vulnerability submitted in the CVE ID request are published according to your +schedule. It's common to request a CVE for an unpatched vulnerability, reference the assigned CVE +identifier in release notes, and later publish the vulnerability's details after the fix is +released. + +Separate communications notify you when different stages of the publication process are complete. + +![CVE ID request publication communication](img/cve_request_communication_publication.png) diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index b2020d48d38..73a8e727389 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -26,7 +26,7 @@ If you're using [GitLab CI/CD](../../../ci/README.md), you can analyze your runn for known vulnerabilities using Dynamic Application Security Testing (DAST). You can take advantage of DAST by either [including the CI job](#configuration) in your existing `.gitlab-ci.yml` file or by implicitly using -[Auto DAST](../../../topics/autodevops/stages.md#auto-dast-ultimate), +[Auto DAST](../../../topics/autodevops/stages.md#auto-dast), provided by [Auto DevOps](../../../topics/autodevops/index.md). GitLab checks the DAST report, compares the found vulnerabilities between the source and target @@ -106,7 +106,7 @@ The included template creates a `dast` job in your CI/CD pipeline and scans your project's source code for possible vulnerabilities. The results are saved as a -[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast) that you can later download and analyze. Due to implementation limitations we always take the latest DAST artifact available. Behind the scenes, the [GitLab DAST Docker image](https://gitlab.com/gitlab-org/security-products/dast) @@ -177,13 +177,13 @@ include: variables: DAST_WEBSITE: https://example.com DAST_AUTH_URL: https://example.com/sign-in - DAST_USERNAME_FIELD: session[user] # the name of username field at the sign-in HTML form - DAST_PASSWORD_FIELD: session[password] # the name of password field at the sign-in HTML form - DAST_AUTH_EXCLUDE_URLS: http://example.com/sign-out,http://example.com/sign-out-2 # optional, URLs to skip during the authenticated scan; comma-separated, no spaces in between + DAST_USERNAME_FIELD: session[user] # the name of username field at the sign-in HTML form + DAST_PASSWORD_FIELD: session[password] # the name of password field at the sign-in HTML form + DAST_AUTH_EXCLUDE_URLS: http://example.com/sign-out,http://example.com/sign-out-2 # optional, URLs to skip during the authenticated scan; comma-separated, no spaces in between ``` The results are saved as a -[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast-ultimate) +[DAST report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportsdast) that you can later download and analyze. Due to implementation limitations, we always take the latest DAST artifact available. @@ -206,6 +206,9 @@ variables: DAST_FULL_SCAN_ENABLED: "true" ``` +NOTE: **Note:** +If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). + #### Domain validation The DAST job can be run anywhere, which means you can accidentally hit live web servers @@ -364,7 +367,7 @@ dast: DAST_API_SPECIFICATION: api-specification.yml ``` -#### Full scan +#### Full API scan API scans support full scanning, which can be enabled by using the `DAST_FULL_SCAN_ENABLED` environment variable. Domain validation is not supported for full API scans. @@ -395,7 +398,11 @@ variables: DAST_API_HOST_OVERRIDE: api-test.host.com ``` -Note that `DAST_API_HOST_OVERRIDE` is only applied to specifications imported by URL. +NOTE: **Note:** +Using a host override is ONLY supported when importing the API +specification from a URL. It does not work and will be ignored when importing +the specification from a file. This is due to a limitation in the ZAP OpenAPI +extension. #### Authentication using headers @@ -412,6 +419,31 @@ variables: DAST_REQUEST_HEADERS: "Authorization: Bearer my.token" ``` +### URL scan + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.4. + +A URL scan allows you to specify which parts of a website are scanned by DAST. + +#### Define the URLs to scan + +To specify the paths to be scanned, add a comma-separated list of the paths to the `DAST_PATHS` environment variable. Note that you can only scan paths of a single host. + +```yaml +include: + - template: DAST.gitlab-ci.yml + +variables: + DAST_PATHS=/page1.html,/category1/page1.html,/page3.html +``` + +NOTE: **Note:** +`DAST_AUTH_EXCLUDE_URLS` are ignored when `DAST_PATHS` is set. + +#### Full Scan + +To perform a [full scan](#full-scan) on the listed paths, use the `DAST_FULL_SCAN_ENABLED` environment variable. + ### Customizing the DAST settings CAUTION: **Deprecation:** @@ -451,12 +483,12 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_USERNAME_FIELD` | string | The name of username field at the sign-in HTML form. | | `DAST_PASSWORD_FIELD` | string | The name of password field at the sign-in HTML form. | | `DAST_MASK_HTTP_HEADERS` | string | Comma-separated list of request and response headers to be masked (GitLab 13.1). Must contain **all** headers to be masked. Refer to [list of headers that are masked by default](#hide-sensitive-information). | -| `DAST_AUTH_EXCLUDE_URLS` | URLs | The URLs to skip during the authenticated scan; comma-separated, no spaces in between. Not supported for API scans. | +| `DAST_AUTH_EXCLUDE_URLS` | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. | | `DAST_FULL_SCAN_ENABLED` | boolean | Set to `true` to run a [ZAP Full Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Full-Scan) instead of a [ZAP Baseline Scan](https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan). Default: `false` | | `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` | boolean | Set to `true` to require [domain validation](#domain-validation) when running DAST full scans. Not supported for API scans. Default: `false` | | `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false` | -| `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Example: `example.com:8080` | -| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were supressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | +| `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` | +| `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | | `DAST_REQUEST_HEADERS` | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | | `DAST_DEBUG` | boolean | Enable debug message output. Default: `false` | | `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. | @@ -465,6 +497,7 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia | `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. | | `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false` | | `DAST_USE_AJAX_SPIDER` | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false` | +| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html` | | `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. | | `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` | @@ -484,8 +517,8 @@ dast: ``` You must then overwrite the `script` command to pass in the appropriate -argument. For example, passive scanning can be delayed using option `-D`. The following -configuration delays passive scanning by five minutes: +argument. For example, vulnerability definitions in alpha can be included with +`-a`. The following configuration includes those definitions: ```yaml include: @@ -494,7 +527,7 @@ include: dast: script: - export DAST_WEBSITE=${DAST_WEBSITE:-$(cat environment_url.txt)} - - /analyze -D 300 -t $DAST_WEBSITE + - /analyze -a -t $DAST_WEBSITE ``` ### Custom ZAProxy configuration @@ -559,8 +592,8 @@ To use DAST in an offline environment, you need: NOTE: **Note:** GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy), -meaning the Runner tries to pull Docker images from the GitLab container registry even if a local -copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) +meaning the runner tries to pull Docker images from the GitLab container registry even if a local +copy is available. The GitLab Runner [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy) in an offline environment if you prefer using only locally available Docker images. However, we recommend keeping the pull policy setting to `always` if not in an offline environment, as this enables the use of updated scanners in your CI/CD pipelines. @@ -600,110 +633,171 @@ security reports without requiring internet access. Alternatively, you can use the variable `SECURE_ANALYZERS_PREFIX` to override the base registry address of the `dast` image. -## On-Demand Scans +## Site profile -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. -> - It's deployed behind a feature flag, enabled by default. -> - It's enabled on GitLab.com. -> - It's able to be enabled or disabled per-project. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans). +A site profile describes the attributes of a web site to scan on demand with DAST. A site profile is +required for an on-demand DAST scan. -You can run a passive DAST scan against a target website, outside the DevOps lifecycle. These scans -are always associated with the default branch of your project and the results are available in the -project dashboard. +A site profile contains the following: -### Site profile +- **Profile name**: A name you assign to the site to be scanned. +- **Target URL**: The URL that DAST runs against. -An on-demand scan requires a site profile, which includes a profile name and target URL. The profile -name allows you to describe the site to be scanned. The target URL specifies the URL against which -the DAST scan is run. +### Create a site profile -### Run an on-demand scan +To create a site profile: -NOTE: **Note:** -You must have permission to run an on-demand DAST scan against a protected branch. -The default branch is automatically protected. For more details, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **New Profile > Site Profile**. +1. Type in a unique **Profile name** and **Target URL** then click **Save profile**. -Running an on-demand scan requires an existing site profile. If a site profile for the target URL -doesn't exist, first [create a site profile](#create-a-site-profile). An on-demand DAST scan has -a fixed timeout of 60 seconds. +### Edit a site profile -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Select a site profile from the profiles dropdown. -- Click **Run scan**. +To edit an existing site profile: -#### Create a site profile +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **Edit** in the row of the profile to edit. +1. Edit the **Profile name** and **Target URL**, then click **Save profile**. -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Click **New Site Profile**. -- Type in a unique **Profile name** and **Target URL** then click **Save profile**. +### Delete a site profile -#### Delete a site profile +To delete an existing site profile: -- Navigate to your project's home page, then click **On-demand Scans** in the left sidebar. -- Click **Create new DAST scan**. -- Click **Delete** in the matching site profile's row. +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **{remove}** in the row of the profile to delete. -### Enable or disable On-demand Scans and site profiles +## Scanner profile -On-demand Scans with site profiles is enabled by default. You can disable On-demand Scans -instance-wide, or disable it for specific projects if you prefer. DAST site profiles are not -available if the On-demand Scans feature is disabled. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4. +> - [Deployed behind a feature flag](../../feature_flags.md), enabled by default. +> - Enabled on GitLab.com. +> - Can be enabled or disabled per-project. +> - Recommended for production use. +> - For GitLab self-managed instances, GitLab administrators can [disable this feature](#enable-or-disable-dast-scanner-profiles). -Use of On-demand Scans with site profiles requires **both** the following feature flags enabled: +A scanner profile defines the scanner settings used to run an on-demand scan: -- security_on_demand_scans_feature_flag -- security_on_demand_scans_site_profiles_feature_flag +- **Profile name:** A name you give the scanner profile. For example, "Spider_15". +- **Spider timeout:** The maximum number of minutes allowed for the spider to traverse the site. +- **Target timeout:** The maximum number of seconds DAST waits for the site to be available before + starting the scan. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable or enable the feature flags. +### Create a scanner profile + +To create a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **New Profile > Scanner Profile**. +1. Enter a unique **Profile name**, the desired **Spider timeout**, and the **Target timeout**. +1. Click **Save profile**. + +### Edit a scanner profile + +To edit a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **Edit** in the scanner profile's row. -#### Enable or disable On-demand Scans +### Delete a scanner profile -To disable On-demand Scans: +To delete a scanner profile: + +1. From your project's home page, go to **Security & Compliance > Configuration**. +1. Click **Manage** in the **DAST Profiles** row. +1. Click **{remove}** in the scanner profile's row. + +### Enable or disable DAST scanner profiles + +The scanner profile feature is ready for production use. It's deployed behind a feature flag that +is **enabled by default**. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) can opt to disable it. + +To disable it: ```ruby # Instance-wide -Feature.disable(:security_on_demand_scans_feature_flag) +Feature.disable(:security_on_demand_scans_scanner_profiles) # or by project -Feature.disable(:security_on_demand_scans_feature_flag, Project.find()) +Feature.disable(:security_on_demand_scans_scanner_profiles, Project.find()) ``` -To enable On-demand Scans: +To enable it: ```ruby # Instance-wide -Feature.enable(:security_on_demand_scans_feature_flag) +Feature.enable(:security_on_demand_scans_scanner_profiles) # or by project -Feature.enable(:security_on_demand_scans_feature_flag, Project.find()) +Feature.enable(:security_on_demand_scans_scanner_profiles, Project.find()) ``` -#### Enable or disable site profiles +## On-demand scans + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. +> - It's deployed behind a feature flag, enabled by default. +> - It's enabled on GitLab.com. +> - It's able to be enabled or disabled per-project. +> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans). + +An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger +the scan. You must start it manually. + +An on-demand DAST scan: + +- Uses settings in the site profile and scanner profile you select when you run the scan, + instead of those in the `.gitlab-ci.yml` file. +- Is associated with your project's default branch. + +### Run an on-demand DAST scan + +NOTE: **Note:** +You must have permission to run an on-demand DAST scan against a protected branch. +The default branch is automatically protected. For more details, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). + +To run an on-demand DAST scan, you need: + +- A [scanner profile](#create-a-scanner-profile). +- A [site profile](#create-a-site-profile). + +1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left sidebar. +1. Click **Create new DAST scan**. +1. In **Scanner settings**, select a scanner profile from the dropdown. +1. In **Site profiles**, select a site profile from the dropdown. +1. Click **Run scan**. + +The on-demand DAST scan runs and the project's dashboard shows the results. + +### Enable or disable On-demand Scans + +The On-demand DAST Scans feature is enabled by default. You can disable on-demand scans +instance-wide, or disable it for specific projects if you prefer. + +To run on-demand DAST scans, an administrator must enable the +`security_on_demand_scans_feature_flag` feature flag. -The Site Profiles feature is enabled instance-wide by default. You can disable it instance-wide, or disable it -for specific projects if you prefer. [GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can disable or enable the feature flag. +can disable or enable the feature flags. -To disable Site Profiles: +To disable On-demand DAST Scans: ```ruby # Instance-wide -Feature.disable(:security_on_demand_scans_site_profiles_feature_flag) +Feature.disable(:security_on_demand_scans_feature_flag) # or by project -Feature.disable(:security_on_demand_scans_site_profiles_feature_flag, Project.find()) +Feature.disable(:security_on_demand_scans_feature_flag, Project.find()) ``` -To enable Site Profiles: +To enable On-demand DAST Scans: ```ruby # Instance-wide -Feature.enable(:security_on_demand_scans_site_profiles_feature_flag) +Feature.enable(:security_on_demand_scans_feature_flag) # or by project -Feature.enable(:security_on_demand_scans_site_profiles_feature_flag, Project.find()) +Feature.enable(:security_on_demand_scans_feature_flag, Project.find()) ``` ## Reports @@ -825,6 +919,10 @@ variables: Here, DAST is being allocated 3072 MB. Change the number after `-Xmx` to the required memory amount. +### DAST job exceeding the job timeout + +If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). +