diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-02-18 09:45:46 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-02-18 09:45:46 +0000 |
| commit | a7b3560714b4d9cc4ab32dffcd1f74a284b93580 (patch) | |
| tree | 7452bd5c3545c2fa67a28aa013835fb4fa071baf /doc/user/application_security | |
| parent | ee9173579ae56a3dbfe5afe9f9410c65bb327ca7 (diff) | |
| download | gitlab-ce-a7b3560714b4d9cc4ab32dffcd1f74a284b93580.tar.gz | |
Add latest changes from gitlab-org/gitlab@14-8-stable-eev14.8.0-rc42
Diffstat (limited to 'doc/user/application_security')
32 files changed, 1622 insertions, 506 deletions
diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index be6b06a0797..4eb721f8832 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -114,6 +114,7 @@ To generate an API Fuzzing configuration snippet: > - Support for OpenAPI Specification v3.0 was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228652) in GitLab 13.9. > - Support for OpenAPI Specification using YAML format was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0. > - Support for OpenAPI Specification v3.1 was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327268) in GitLab 14.2. +> - Support to generate media type `application/xml` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327268) in GitLab 14.8. The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs. This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test. @@ -125,6 +126,7 @@ the body generation is limited to these body types: - `application/x-www-form-urlencoded` - `multipart/form-data` - `application/json` +- `application/xml` #### Configure Web API fuzzing with an OpenAPI Specification @@ -459,7 +461,7 @@ Follow these steps to provide the bearer token with `FUZZAPI_OVERRIDES_ENV`: ``` 1. To validate that authentication is working, run an API fuzzing test and review the fuzzing logs - and the test API's application logs. + and the test API's application logs. See the [overrides section](#overrides) for more information about override commands. ##### Token generated at test runtime @@ -493,7 +495,7 @@ variables: FUZZAPI_PROFILE: Quick FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_FILE: api-fuzzing-overrides.json ``` To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and @@ -535,7 +537,7 @@ variables: FUZZAPI_PROFILE: Quick-10 FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_FILE: api-fuzzing-overrides.json FUZZAPI_OVERRIDES_CMD: renew_token.py FUZZAPI_OVERRIDES_INTERVAL: 300 ``` @@ -575,6 +577,9 @@ profile increases as the number of tests increases. |[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | |[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | +|[`FUZZAPI_OVERRIDES_CMD_VERBOSE`](#overrides) | When set to any value. It shows overrides command output as part of the job output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.8. | +|`FUZZAPI_PRE_SCRIPT` | Run user command or script before scan session starts. | +|`FUZZAPI_POST_SCRIPT` | Run user command or script after scan session has finished. | |[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | |[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | |[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | @@ -754,7 +759,7 @@ variables: FUZZAPI_PROFILE: Quick FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_FILE: api-fuzzing-overrides.json ``` #### Using a CI/CD variable @@ -799,16 +804,181 @@ variables: If the value must be generated or regenerated on expiration, you can provide a program or script for the API fuzzer to execute on a specified interval. The provided script runs in an Alpine Linux -container that has Python 3 and Bash installed. If the Python script requires additional packages, -it must detect this and install the packages at runtime. The script creates the overrides JSON file -as defined above. +container that has Python 3 and Bash installed. + +You have to set the environment variable `FUZZAPI_OVERRIDES_CMD` to the program or script you would like +to execute. The provided command creates the overrides JSON file as defined previously. + +You might want to install other scripting runtimes like NodeJS or Ruby, or maybe you need to install a dependency for +your overrides command. In this case, we recommend setting the `FUZZAPI_PRE_SCRIPT` to the file path of a script which +provides those prerequisites. The script provided by `FUZZAPI_PRE_SCRIPT` is executed once, before the analyzer starts. + +See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) +page for information about installing Alpine Linux packages. You must provide three CI/CD variables, each set for correct operation: - `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. -- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. +- `FUZZAPI_OVERRIDES_CMD`: Overrides command in charge of generating the overrides JSON file periodically. - `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. +Optionally: + +- `FUZZAPI_PRE_SCRIPT`: Script to install runtimes or dependencies before the analyzer starts. + +```yaml +stages: + - fuzz + +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick + FUZZAPI_OPENAPI: test-api-specification.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_OVERRIDES_FILE: api-fuzzing-overrides.json + FUZZAPI_OVERRIDES_CMD: renew_token.py + FUZZAPI_OVERRIDES_INTERVAL: 300 +``` + +#### Debugging overrides + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.8. + +By default the output of the overrides command is hidden. If the overrides command returns a non zero exit code, the command is displayed as part of your job output. Optionally, you can set the variable `FUZZAPI_OVERRIDES_CMD_VERBOSE` to any value in order to display overrides command output as it is generated. This is useful when testing your overrides script, but should be disabled afterwards as it slows down testing. + +It is also possible to write messages from your script to a log file that is collected when the job completes or fails. The log file must be created in a specific location and follow a naming convention. + +Adding some basic logging to your overrides script is useful in case the script fails unexpectedly during normal running of the job. The log file is automatically included as an artifact of the job, allowing you to download it after the job has finished. + +Following our example, we provided `renew_token.py` in the environmental variable `FUZZAPI_OVERRIDES_CMD`. Please notice two things in the script: + +- Log file is saved in the location indicated by the environment variable `CI_PROJECT_DIR`. +- Log file name should match `gl-*.log`. + +```python +#!/usr/bin/env python + +# Example of an overrides command + +# Override commands can update the overrides json file +# with new values to be used. This is a great way to +# update an authentication token that will expire +# during testing. + +import logging +import json +import os +import requests +import backoff + +# [1] Store log file in directory indicated by env var CI_PROJECT_DIR +working_directory = os.environ['CI_PROJECT_DIR'] + +# [2] File name should match the pattern: gl-*.log +log_file_path = os.path.join(working_directory, 'gl-user-overrides.log') + +# Set up logger +logging.basicConfig(filename=log_file_path, level=logging.DEBUG) + +# Use `backoff` decorator to retry in case of transient errors. +@backoff.on_exception(backoff.expo, + (requests.exceptions.Timeout, + requests.exceptions.ConnectionError), + max_time=30) +def get_auth_response(): + return requests.get('https://authorization.service/api/get_api_token', auth=(os.environ['AUTH_USER'], os.environ['AUTH_PWD'])) + + +# In our example, access token is retrieved from a given endpoint +try: + + # Performs a http request, response sample: + # { "Token" : "b5638ae7-6e77-4585-b035-7d9de2e3f6b3" } + response = get_auth_response() + + # Check that the request is successful. may raise `requests.exceptions.HTTPError` + response.raise_for_status() + + # Gets JSON data + response_body = response.json() + +# If needed specific exceptions can be caught +# requests.ConnectionError : A network connection error problem occurred +# requests.HTTPError : HTTP request returned an unsuccessful status code. [Response.raise_for_status()] +# requests.ConnectTimeout : The request timed out while trying to connect to the remote server +# requests.ReadTimeout : The server did not send any data in the allotted amount of time. +# requests.TooManyRedirects : The request exceeds the configured number of maximum redirections +# requests.exceptions.RequestException : All exceptions that related to Requests +except requests.exceptions.RequestException as requests_error: + # logs exceptions related to `Requests` + logging.error(f'Error, failed while performing HTTP request. Error message: {requests_error}') + raise +except requests.exceptions.JSONDecodeError as json_decode_error: + # logs errors related decoding JSON response + logging.error(f'Error, failed while decoding JSON response. Error message: {json_decode_error}') + raise +except Exception as e: + # logs any other error + logging.error(f'Error, unknown error while retrieving access token. Error message: {e}') + raise + +# computes object that holds overrides file content. +# It uses data fetched from request +overrides_data = { + "headers": { + "Authorization": f"Token {response_body['Token']}" + } +} + +# log entry informing about the file override computation +overrides_file_path = os.path.join( + working_directory, "api-fuzzing-overrides.json") +logging.info("Creating overrides file: %s" % overrides_file_path) + +# attempts to overwrite the file +try: + if os.path.exists(overrides_file_path): + os.unlink(overrides_file_path) + + # overwrites the file with our updated dictionary + with open(overrides_file_path, "wb+") as fd: + fd.write(json.dumps(overrides_data).encode('utf-8')) +except Exception as e: + # logs any other error + logging.error(f'Error, unkown error when overwritng file {overrides_file_path}. Error message: {e}') + raise + +# logs informing override has finished successfully +logging.info("Override file has been updated") + +# end +``` + +In the overrides command example, the Python script depends on the `backoff` library. To make sure the library is installed before executing the Python script, the `FUZZAPI_PRE_SCRIPT` is set to a script that will install the dependencies of your overrides command. +As for example, the following script `user-pre-scan-set-up.sh`: + +```shell +#!/bin/bash + +# user-pre-scan-set-up.sh +# Ensures python dependencies are installed + +echo "**** install python dependencies ****" + +python3 -m ensurepip +pip3 install --no-cache --upgrade \ + pip \ + backoff + +echo "**** python dependencies installed ****" + +# end +``` + +You have to update your configuration to set the `FUZZAPI_PRE_SCRIPT` to our new `user-pre-scan-set-up.sh` script. For example: + ```yaml stages: - fuzz @@ -820,11 +990,14 @@ variables: FUZZAPI_PROFILE: Quick FUZZAPI_OPENAPI: test-api-specification.json FUZZAPI_TARGET_URL: http://test-deployment/ - FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json + FUZZAPI_PRE_SCRIPT: user-pre-scan-set-up.sh + FUZZAPI_OVERRIDES_FILE: api-fuzzing-overrides.json FUZZAPI_OVERRIDES_CMD: renew_token.py FUZZAPI_OVERRIDES_INTERVAL: 300 ``` +In the previous sample, you could use the script `user-pre-scan-set-up.sh` to also install new runtimes or applications that later on you could use in your overrides command. + ### Exclude Paths > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0. diff --git a/doc/user/application_security/cluster_image_scanning/index.md b/doc/user/application_security/cluster_image_scanning/index.md index 5f2dd626526..0db9af7a0d3 100644 --- a/doc/user/application_security/cluster_image_scanning/index.md +++ b/doc/user/application_security/cluster_image_scanning/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/) in GitLab 14.1. WARNING: -This analyzer is in [Alpha](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha) +This analyzer is in [Alpha](../../../policy/alpha-beta-support.md#alpha-features) and is unstable. The JSON report and CI/CD configuration may be subject to change or breakage across GitLab releases. @@ -301,7 +301,9 @@ the security vulnerabilities in your groups, projects, and pipelines. ## Interacting with the vulnerabilities -After a vulnerability is found, you can [address it](../vulnerabilities/index.md). +After you find a vulnerability, you can address it in the [vulnerability report](../vulnerabilities/index.md) +or the [GitLab Agent's](../../clusters/agent/install/index.md#view-vulnerabilities-in-cluster-images) +details section. ## Troubleshooting diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index cdcd334dba6..430f8e1a2a2 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -58,11 +58,11 @@ You can configure the following security controls: - [API Fuzzing](../api_fuzzing/index.md) - Select **Enable API Fuzzing** to use API Fuzzing for the current project. For more details, read [API Fuzzing](../../../user/application_security/api_fuzzing/index.md#enable-web-api-fuzzing). - [Coverage Fuzzing](../coverage_fuzzing/index.md) - - Can be configured with `.gitlab-ci.yml`. For more details, read [Coverage Fuzzing](../../../user/application_security/coverage_fuzzing/index.md#configuration). + - Can be configured with `.gitlab-ci.yml`. For more details, read [Coverage Fuzzing](../../../user/application_security/coverage_fuzzing/index.md#enable-coverage-guided-fuzz-testing). ## Compliance **(ULTIMATE)** You can configure the following security controls: - [License Compliance](../../../user/compliance/license_compliance/index.md) - - Can be configured with `.gitlab-ci.yml`. For more details, read [License Compliance](../../../user/compliance/license_compliance/index.md#configuration). + - Can be configured with `.gitlab-ci.yml`. For more details, read [License Compliance](../../../user/compliance/license_compliance/index.md#enable-license-compliance). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 5a2dd5eb54f..08a8c46cc72 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -222,6 +222,7 @@ You can [configure](#customizing-the-container-scanning-settings) analyzers by u | `CS_DISABLE_DEPENDENCY_LIST` | `"false"` | Disable Dependency Scanning for packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` | `"true"` | Disable scanning for language-specific packages installed in the scanned image. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345434) in GitLab 14.6. | All | | `CS_DOCKER_INSECURE` | `"false"` | Allow access to secure Docker registries using HTTPS without validating the certificates. | All | +| `CS_IGNORE_UNFIXED` | `"false"` | Ignore vulnerabilities that are not fixed. | All | | `CS_REGISTRY_INSECURE` | `"false"` | Allow access to insecure registries (HTTP only). Should only be set to `true` when testing the image locally. Works with all scanners, but the registry must listen on port `80/tcp` for Trivy to work. | All | | `CS_SEVERITY_THRESHOLD` | `UNKNOWN` | Severity level threshold. The scanner outputs vulnerabilities with severity level higher than or equal to this threshold. Supported levels are Unknown, Low, Medium, High, and Critical. | Trivy | | `DOCKER_IMAGE` | `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG` | The Docker image to be scanned. If set, this variable overrides the `$CI_APPLICATION_REPOSITORY` and `$CI_APPLICATION_TAG` variables. | All | @@ -513,7 +514,7 @@ registry.gitlab.com/security-products/container-scanning/trivy:4 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which you can import or temporarily access external resources. These scanners -are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance), +are [periodically updated](../index.md#vulnerability-scanner-maintenance), and you may be able to make occasional updates on your own. For more information, see [the specific steps on how to update an image with a pipeline](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline). @@ -728,8 +729,16 @@ the security vulnerabilities in your groups, projects and pipelines. ## Vulnerabilities database update -If you use container scanning and want more information about the vulnerabilities database update, -see the [maintenance table](../vulnerabilities/index.md#vulnerability-scanner-maintenance). +All analyzer images are [updated daily](https://gitlab.com/gitlab-org/security-products/analyzers/container-scanning/-/blob/master/README.md#image-updates). + +The images include the latest advisory database available for their respective scanner. Each +scanner includes data from multiple sources: + +- [Grype](https://github.com/anchore/grype#grypes-database). +- [Trivy](https://aquasecurity.github.io/trivy/latest/vulnerability/detection/data-source/). + +Database update information for other analyzers is available in the +[maintenance table](../index.md#vulnerability-scanner-maintenance). ## Interacting with the vulnerabilities diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 89b4cdcc34d..290d4a06dcc 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -2,7 +2,6 @@ stage: Secure group: Dynamic Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, howto --- # Coverage-guided fuzz testing **(ULTIMATE)** @@ -23,8 +22,14 @@ The fuzz testing process: 1. Compiles the target application. 1. Runs the instrumented application, using the `gitlab-cov-fuzz` tool. 1. Parses and analyzes the exception information output by the fuzzer. -1. Downloads the [corpus](../terminology/index.md#corpus) and crash events from previous pipelines. +1. Downloads the [corpus](../terminology/index.md#corpus) from either: + - The previous pipelines. + - If `COVFUZZ_USE_REGISTRY` is set to `true`, the [corpus registry](#corpus-registry). +1. Downloads crash events from previous pipeline. 1. Outputs the parsed crash events and data to the `gl-coverage-fuzzing-report.json` file. +1. Updates the corpus, either: + - In the job's pipeline. + - If `COVFUZZ_USE_REGISTRY` is set to `true`, in the corpus registry. The results of the coverage-guided fuzz testing are available in the CI/CD pipeline. @@ -44,9 +49,20 @@ You can use the following fuzzing engines to test the specified languages. | Python | [`pythonfuzz`](https://gitlab.com/gitlab-org/security-products/analyzers/fuzzers/pythonfuzz) | [pythonfuzz-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/pythonfuzz-fuzzing-example) | | AFL (any language that works on top of AFL) | [AFL](https://lcamtuf.coredump.cx/afl/) | [afl-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/afl-fuzzing-example) | -## Configuration +## Confirm status of coverage-guided fuzz testing -To enable coverage-guided fuzz testing, edit the `.gitlab-ci.yml` file: +To confirm the status of coverage-guided fuzz testing: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. In the **Coverage Fuzzing** section the status is: + - **Not configured** + - **Enabled** + - A prompt to upgrade to GitLab Ultimate. + +## Enable coverage-guided fuzz testing + +To enable coverage-guided fuzz testing, edit `.gitlab-ci.yml`: 1. Add the `fuzz` stage to the list of stages. @@ -99,10 +115,13 @@ Use the following variables to configure coverage-guided fuzz testing in your CI | CI/CD variable | Description | |---------------------------|---------------------------------------------------------------------------------| -| `COVFUZZ_ADDITIONAL_ARGS` | Arguments passed to `gitlab-cov-fuzz`. Used to customize the behavior of the underlying fuzzing engine. Read the fuzzing engine's documentation for a complete list of arguments. | +| `COVFUZZ_ADDITIONAL_ARGS` | Arguments passed to `gitlab-cov-fuzz`. Used to customize the behavior of the underlying fuzzing engine. Read the fuzzing engine's documentation for a complete list of arguments. | | `COVFUZZ_BRANCH` | The branch on which long-running fuzzing jobs are to be run. On all other branches, only fuzzing regression tests are run. Default: Repository's default branch. | | `COVFUZZ_SEED_CORPUS` | Path to a seed corpus directory. Default: empty. | | `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this value when using an offline environment. Default: `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. | +| `COVFUZZ_USE_REGISTRY` | Set to `true` to have the corpus stored in the GitLab corpus registry. The variables `COVFUZZ_CORPUS_NAME` and `COVFUZZ_GITLAB_TOKEN` are required if this variable is set to `true`. Default: `false`. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | +| `COVFUZZ_CORPUS_NAME` | Name of the corpus to be used in the job. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | +| `COVFUZZ_GITLAB_TOKEN` | Environment variable configured with [Personal Access Token](../../../user/profile/personal_access_tokens.md#create-a-personal-access-token) with API read/write access. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | #### Seed corpus @@ -123,9 +142,93 @@ Each fuzzing step outputs these artifacts: You can download the JSON report file from the CI/CD pipelines page. For more information, see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). -### Coverage-guided fuzz testing report +## Corpus registry + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an +administrator to [disable the feature flags](../../../administration/feature_flags.md) named +`corpus_management` and `corpus_management_ui`. On GitLab.com, this feature is available. + +The corpus registry is a library of corpuses. Corpuses in a project's registry are available to +all jobs in that project. A project-wide registry is a more efficient way to manage corpuses than +the default option of one corpus per job. + +The corpus registry uses the package registry to store the project's corpuses. Corpuses stored in +the registry are hidden to ensure data integrity. + +In the GitLab UI, with corpus management you can: + +- View details of the corpus registry. +- Download a corpus. +- Delete a corpus. +- Create a new corpus. + +When you download a corpus, the file is named `artifacts.zip`, regardless of the filename used when +the corpus was initially uploaded. This file contains only the corpus, which is different to the +artifacts files you can download from the CI/CD pipeline. + +### View details of the corpus registry + +To view details of the corpus registry: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. In the **Coverage Fuzzing** section, select **Manage corpus**. + +### Create a corpus in the corpus registry + +To create a corpus in the corpus registry, either: + +- Create a corpus in a pipeline +- Upload an existing corpus file + +#### Create a corpus in a pipeline + +To create a corpus in a pipeline: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha). +1. In the `.gitlab-ci.yml` file, edit the `my_fuzz_target` job. +1. Set the following variables: + - Set `COVFUZZ_USE_REGISTRY` to `true`. + - Set `COVFUZZ_CORPUS_NAME` to name the corpus. + - Set `COVFUZZ_GITLAB_TOKEN` to the value of the personal access token. + +After the `my_fuzz_target` job runs, the corpus is stored in the corpus registry, with the name +provided by the `COVFUZZ_CORPUS_NAME` variable. The corpus is updated on every pipeline run. + +#### Upload a corpus file + +To upload an existing corpus file: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. In the **Coverage Fuzzing** section, select **Manage corpus**. +1. Select **New corpus**. +1. Complete the fields. +1. Select **Upload file**. +1. Select **Add**. + +You can now reference the corpus in the `.gitlab-ci.yml` file. Ensure the value used in the +`COVFUZZ_CORPUS_NAME` variable matches exactly the name given to the uploaded corpus file. + +### Use a corpus stored in the corpus registry + +To use a corpus stored in the corpus registry, you must reference it by its name. To confirm the +name of the relevant corpus, view details of the corpus registry. + +Prerequisites: + +- [Enable coverage-guide fuzz testing](#enable-coverage-guided-fuzz-testing) in the project. + +1. Set the following variables in the `.gitlab-ci.yml` file: + - Set `COVFUZZ_USE_REGISTRY` to `true`. + - Set `COVFUZZ_CORPUS_NAME` to the name of the corpus. + - Set `COVFUZZ_GITLAB_TOKEN` to the value of the personal access token. + +## Coverage-guided fuzz testing report + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](../../../policy/alpha-beta-support.md#alpha-features). For detailed information about the `gl-coverage-fuzzing-report.json` file's format, read the [schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json). @@ -161,9 +264,9 @@ Example coverage-guided fuzzing report: ## Duration of coverage-guided fuzz testing -The available durations for coverage-guided fuzz testing are: 10 minutes (default) and 60 minutes. +The available durations for coverage-guided fuzz testing are: -- 10-minute duration: Recommended for the default branch. +- 10-minute duration (default): Recommended for the default branch. - 60-minute duration: Recommended for the development branch and merge requests. The longer duration provides greater coverage. In the `COVFUZZ_ADDITIONAL_ARGS` variable set the value `--regression=true`. @@ -256,3 +359,16 @@ vulnerability: engines listed in [Supported fuzzing engines and languages](#supported-fuzzing-engines-and-languages). <!-- vale gitlab.Acronyms = YES --> + +## Troubleshooting + +### Error "Unable to extract corpus folder from artifacts zip file" + +If you see this error message, and `COVFUZZ_USE_REGISTRY` is set to `true`, ensure that the uploaded +corpus file extracts into a folder named `corpus`. + +### Error "400 Bad request - Duplicate package is not allowed" + +If you see this error message when running the fuzzing job with `COVFUZZ_USE_REGISTRY` set to `true`, +ensure that duplicates are allowed. For more details, see +[duplicate Generic packages](../../packages/generic_packages/#do-not-allow-duplicate-generic-packages). diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index 5d1e57553f4..4cde1847419 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -10,7 +10,7 @@ type: reference, howto > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323423) in GitLab 13.12. WARNING: -This product is in an early-access stage and is considered a [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) feature. +This product is in an early-access stage and is considered a [beta](../../../policy/alpha-beta-support.md#beta-features) feature. GitLab DAST's new browser-based crawler is a crawl engine built by GitLab to test Single Page Applications (SPAs) and traditional web applications. Due to the reliance of modern web applications on JavaScript, handling SPAs or applications that are dependent on JavaScript is paramount to ensuring proper coverage of an application for Dynamic Application Security Testing (DAST). diff --git a/doc/user/application_security/dast/checks/1004.1.md b/doc/user/application_security/dast/checks/1004.1.md index dfbc600b05b..9626973eb36 100644 --- a/doc/user/application_security/dast/checks/1004.1.md +++ b/doc/user/application_security/dast/checks/1004.1.md @@ -4,7 +4,7 @@ group: Dynamic Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Sensitive cookie without `HttpOnly` attribute +# Sensitive cookie without HttpOnly attribute ## Description diff --git a/doc/user/application_security/dast/checks/16.2.md b/doc/user/application_security/dast/checks/16.2.md index 95461e8677d..484460b6642 100644 --- a/doc/user/application_security/dast/checks/16.2.md +++ b/doc/user/application_security/dast/checks/16.2.md @@ -38,7 +38,7 @@ the `Server` header. ## Links -- [cwe](https://cwe.mitre.org/data/definitions/16.html) +- [CWE](https://cwe.mitre.org/data/definitions/16.html) - [Apache ServerTokens](https://blog.mozilla.org/security/2016/08/26/mitigating-mime-confusion-attacks-in-firefox/) - [NGINX server_tokens](https://nginx.org/en/docs/http/ngx_http_core_module.html#server_tokens) - [IIS 10 Remove Server Header](https://docs.microsoft.com/en-us/iis/configuration/system.webserver/security/requestfiltering/#attributes) diff --git a/doc/user/application_security/dast/checks/16.3.md b/doc/user/application_security/dast/checks/16.3.md index 6f80a2a32c6..e4fc2468dae 100644 --- a/doc/user/application_security/dast/checks/16.3.md +++ b/doc/user/application_security/dast/checks/16.3.md @@ -32,4 +32,4 @@ information from the `X-Powered-By` header. ## Links - [CWE](https://cwe.mitre.org/data/definitions/16.html) -- [PHP `expose_php`](https://www.php.net/manual/en/ini.core.php#ini.expose-php) +- [PHP expose_php](https://www.php.net/manual/en/ini.core.php#ini.expose-php) diff --git a/doc/user/application_security/dast/checks/200.1.md b/doc/user/application_security/dast/checks/200.1.md new file mode 100644 index 00000000000..98a482b4a0f --- /dev/null +++ b/doc/user/application_security/dast/checks/200.1.md @@ -0,0 +1,29 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of sensitive information to an unauthorized actor (private IP address) + +## Description + +A private RFC 1918 was identified in the target application. Public facing websites should not be issuing +requests to private IP Addresses. Attackers attempting to execute subsequent attacks, such as Server-Side +Request Forgery (SSRF), may be able to use this information to identify additional internal targets. + +## Remediation + +Identify the resource that is incorrectly specifying an internal IP address and replace it with it's public +facing version, or remove the reference from the target application. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 200.1 | true | 200 | Passive | Low | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/200.html) +- [RFC](https://datatracker.ietf.org/doc/html/rfc1918) diff --git a/doc/user/application_security/dast/checks/548.1.md b/doc/user/application_security/dast/checks/548.1.md new file mode 100644 index 00000000000..94f747739c5 --- /dev/null +++ b/doc/user/application_security/dast/checks/548.1.md @@ -0,0 +1,45 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exposure of information through directory listing + +## Description + +The target web server is configured to list the contents of directories that do not contain an index file +such as `index.html`. This could lead to accidental exposure of sensitive information, or give an attacker +details on how filenames and directories are structured and stored. + +## Remediation + +Directory indexing should be disabled. + +Apache: +For Apache based web sites, ensure all `<Directory>` definitions have `Options -Indexes` configured in the +`apache2.conf` or `httpd.conf` configuration file. + +NGINX: +For NGINX based websites, ensure all `location` definitions have the `autoindex off` directive set in the +`nginx.conf` file. + +IIS: +For IIS based websites version 7.0 and above you can use the `<directoryBrowse enabled="false" />` element +in the `applicationHost.config` or `Web.config` files. + +For all other server types, please consult your product's documentation on how to disable directory +indexing. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 548.1 | false | 548 | Passive | Low | + +## Links + +- [CWE](https://cwe.mitre.org/data/definitions/598.html) +- [Apache Options](https://httpd.apache.org/docs/2.4/mod/core.html#options) +- [NGINX autoindex](https://nginx.org/en/docs/http/ngx_http_autoindex_module.html) +- [IIS directoryBrowse element](https://docs.microsoft.com/en-us/iis/configuration/system.webserver/directorybrowse) diff --git a/doc/user/application_security/dast/checks/614.1.md b/doc/user/application_security/dast/checks/614.1.md index 46f7f61b0c7..ec68ce33529 100644 --- a/doc/user/application_security/dast/checks/614.1.md +++ b/doc/user/application_security/dast/checks/614.1.md @@ -4,7 +4,7 @@ group: Dynamic Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Sensitive cookie without `Secure` attribute +# Sensitive cookie without Secure attribute ## Description @@ -35,6 +35,6 @@ Set-Cookie: {cookie_name}=<random secure value>; Secure ## Links -- [owasp](https://owasp.org/www-community/controls/SecureCookieAttribute) +- [OWASP](https://owasp.org/www-community/controls/SecureCookieAttribute) - [CWE](https://cwe.mitre.org/data/definitions/614.html) - [Mozilla MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#restrict_access_to_cookies) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index a3b89e09751..97224554723 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -10,12 +10,14 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne | ID | Check | Severity | Type | |:---|:------|:---------|:-----| -| [1004.1](1004.1.md) | Sensitive cookie without `HttpOnly` attribute | Low | Passive | +| [1004.1](1004.1.md) | Sensitive cookie without HttpOnly attribute | Low | Passive | | [16.1](16.1.md) | Missing Content-Type header | Low | Passive | | [16.2](16.2.md) | Server header exposes version information | Low | Passive | | [16.3](16.3.md) | X-Powered-By header exposes version information | Low | Passive | | [16.4](16.4.md) | X-Backend-Server header exposes server information | Info | Passive | | [16.5](16.5.md) | AspNet header exposes version information | Low | Passive | | [16.6](16.6.md) | AspNetMvc header exposes version information | Low | Passive | -| [614.1](614.1.md) | Sensitive cookie without `Secure` attribute | Low | Passive | +| [200.1](200.1.md) | Exposure of sensitive information to an unauthorized actor (private IP address) | Low | Passive | +| [548.1](548.1.md) | Exposure of information through directory listing | Low | Passive | +| [614.1](614.1.md) | Sensitive cookie without Secure attribute | Low | Passive | | [693.1](693.1.md) | Missing X-Content-Type-Options: nosniff | Low | Passive | diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index aeaa93f4a85..0865cc10691 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -53,7 +53,7 @@ results. On failure, the analyzer outputs an - [GitLab Runner](../../../ci/runners/index.md) available, with the [`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). - Target application deployed. For more details, read [Deployment options](#deployment-options). -- DAST runs in the `test` stage, which is available by default. If you redefine the stages in the `.gitlab-ci.yml` file, the `test` stage is required. +- DAST runs in the `dast` stage, which must be added manually to your `.gitlab-ci.yml`. ### Deployment options @@ -588,6 +588,28 @@ Using the [`DAST_MASK_HTTP_HEADERS` CI/CD variable](#available-cicd-variables), headers whose values you want masked. For details on how to mask headers, see [Customizing the DAST settings](#customize-dast-settings). +#### Use Mutual TLS + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299596) in GitLab 14.8. + +Mutual TLS allows a target application server to verify that requests are from a known source. Browser-based scans do not support Mutual TLS. + +**Requirements** + +- Base64-encoded PKCS12 certificate +- Password of the base64-encoded PKCS12 certificate + +To enable Mutual TLS: + +1. If the PKCS12 certificate is not already base64-encoded, convert it to base64 encoding. For security reasons, we recommend encoding the certificate locally, **not** using a web-hosted conversion service. For example, to encode the certificate on either macOS or Linux: + + ```shell + base64 <path-to-pkcs12-certificate-file> + ``` + +1. Create a [masked variable](../../../ci/variables/index.md) named `DAST_PKCS12_CERTIFICATE_BASE64` and store the base64-encoded PKCS12 certificate's value in that variable. +1. Create a masked variable `DAST_PKCS12_PASSWORD` and store the PKCS12 certificate's password in that variable. + #### Available CI/CD variables These CI/CD variables are specific to DAST. They can be used to customize the behavior of DAST to your requirements. @@ -623,6 +645,8 @@ These CI/CD variables are specific to DAST. They can be used to customize the be | `DAST_PASSWORD_FIELD` <sup>1,2</sup> | string | The selector of password field at the sign-in HTML form. Example: `id:password` | | `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. | | `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. | +| `DAST_PKCS12_CERTIFICATE_BASE64` | string | The PKCS12 certificate used for sites that require Mutual TLS. Must be encoded as base64 text. | +| `DAST_PKCS12_PASSWORD` | string | The password of the certificate used in `DAST_PKCS12_CERTIFICATE_BASE64`. | | `DAST_REQUEST_HEADERS` <sup>1</sup> | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` | | `DAST_SKIP_TARGET_CHECK` | boolean | Set to `true` to prevent DAST from checking that the target is available before scanning. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229067) in GitLab 13.8. | | `DAST_SPIDER_MINS` <sup>1</sup> | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | @@ -1172,7 +1196,7 @@ To edit an existing site profile: 1. Edit the fields then select **Save profile**. If a site profile is linked to a security policy, a user cannot edit the profile from this page. See -[Scan Execution Policies](../policies/index.md#scan-execution-policy-editor) +[Scan execution policies](../policies/scan-execution-policies.md) for more information. #### Delete a site profile @@ -1186,7 +1210,7 @@ To delete an existing site profile: 1. Select **Delete** to confirm the deletion. If a site profile is linked to a security policy, a user cannot delete the profile from this page. -See [Scan Execution Policies](../policies/index.md#scan-execution-policy-editor) +See [Scan execution policies](../policies/scan-execution-policies.md) for more information. #### Validate a site profile @@ -1329,7 +1353,7 @@ To edit a scanner profile: 1. Select **Save profile**. If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. -See [Scan Execution Policies](../policies/index.md#scan-execution-policy-editor) +See [Scan execution policies](../policies/scan-execution-policies.md) for more information. #### Delete a scanner profile @@ -1343,7 +1367,7 @@ To delete a scanner profile: 1. Select **Delete**. If a scanner profile is linked to a security policy, a user cannot delete the profile from this -page. See [Scan Execution Policies](../policies/index.md#scan-execution-policy-editor) +page. See [Scan execution policies](../policies/scan-execution-policies.md) for more information. ## Auditing @@ -1365,7 +1389,7 @@ The JSON report artifacts are not a public API of DAST and their format is expec The DAST tool always emits a JSON report file called `gl-dast-report.json` and sample reports can be found in the -[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/master/test/end-to-end/expect). +[DAST repository](https://gitlab.com/gitlab-org/security-products/dast/-/tree/main/test/end-to-end/expect). ## Optimizing DAST diff --git a/doc/user/application_security/dast/run_dast_offline.md b/doc/user/application_security/dast/run_dast_offline.md index 86621d73524..63163167a6c 100644 --- a/doc/user/application_security/dast/run_dast_offline.md +++ b/doc/user/application_security/dast/run_dast_offline.md @@ -36,7 +36,7 @@ For DAST, import the following default DAST analyzer image from `registry.gitlab The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. -These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +These scanners are [periodically updated](../index.md#vulnerability-scanner-maintenance) with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index f1eba505589..cc20b49764f 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -69,8 +69,8 @@ starting in GitLab 14.0, GitLab will not check your repository's root for config ### OpenAPI Specification -> Support for OpenAPI Specification using YAML format was -> [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0. +> - Support for OpenAPI Specification using YAML format was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330583) in GitLab 14.0. +> - Support to generate media type `application/xml` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327268) in GitLab 14.8. The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an API description format for REST APIs. This section shows you how to configure API fuzzing using an OpenAPI Specification to provide information about the target API to test. @@ -82,6 +82,7 @@ the body generation is limited to these body types: - `application/x-www-form-urlencoded` - `multipart/form-data` - `application/json` +- `application/xml` Follow these steps to configure DAST API in GitLab with an OpenAPI specification: @@ -478,6 +479,9 @@ Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: `{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the **Variables** section. + Due to the format of `TEST_API_BEARERAUTH` it's not possible to mask the variable. + To mask the token's value, you can create a second variable with the token value's, and define + `TEST_API_BEARERAUTH` with the value `{"headers":{"Authorization":"Bearer $MASKED_VARIABLE"}}`. 1. In your `.gitlab-ci.yml` file, set `DAST_API_OVERRIDES_ENV` to the variable you just created: @@ -529,7 +533,7 @@ variables: DAST_API_PROFILE: Quick DAST_API_OPENAPI: test-api-specification.json DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_FILE: dast-api-overrides.json ``` To validate that authentication is working, run an DAST API test and review the job logs and @@ -571,13 +575,12 @@ variables: DAST_API_PROFILE: Quick DAST_API_OPENAPI: test-api-specification.json DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_FILE: dast-api-overrides.json DAST_API_OVERRIDES_CMD: renew_token.py DAST_API_OVERRIDES_INTERVAL: 300 ``` -To validate that authentication is working, run an DAST API test and review the job logs and -the test API's application logs. +To validate that authentication is working, run an DAST API test and review the job logs and the test API's application logs. See the [overrides section](#overrides) for more information about override commands. ### Configuration files @@ -644,6 +647,9 @@ can be added, removed, and modified by creating a custom configuration. |[`DAST_API_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`DAST_API_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | |[`DAST_API_OVERRIDES_CMD`](#overrides) | Overrides command. | +|[`DAST_API_OVERRIDES_CMD_VERBOSE`](#overrides) | When set to any value. It shows overrides command output as part of the job output. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.6. | +|`DAST_API_PRE_SCRIPT` | Run user command or script before scan session starts. | +|`DAST_API_POST_SCRIPT` | Run user command or script after scan session has finished. | |[`DAST_API_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | |[`DAST_API_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | |[`DAST_API_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | @@ -825,7 +831,7 @@ variables: DAST_API_PROFILE: Quick DAST_API_OPENAPI: test-api-specification.json DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_FILE: dast-api-overrides.json ``` #### Using a CI/CD variable @@ -869,17 +875,29 @@ variables: #### Using a command If the value must be generated or regenerated on expiration, you can provide a program or script for -the DAST API scanner to execute on a specified interval. The provided script runs in an Alpine Linux -container that has Python 3 and Bash installed. If the Python script requires additional packages, -it must detect this and install the packages at runtime. The script creates the overrides JSON file -as defined above. +the DAST API scanner to execute on a specified interval. The provided command runs in an Alpine Linux +container that has Python 3 and Bash installed. + +You have to set the environment variable `DAST_API_OVERRIDES_CMD` to the program or script you would like +to execute. The provided command creates the overrides JSON file as defined previously. + +You might want to install other scripting runtimes like NodeJS or Ruby, or maybe you need to install a dependency for +your overrides command. In this case, we recommend setting the `DAST_API_PRE_SCRIPT` to the file path of a script which +provides those prerequisites. The script provided by `DAST_API_PRE_SCRIPT` is executed once, before the analyzer starts. + +See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) +page for information about installing Alpine Linux packages. You must provide three CI/CD variables, each set for correct operation: - `DAST_API_OVERRIDES_FILE`: File generated by the provided command. -- `DAST_API_OVERRIDES_CMD`: Command to generate JSON file. +- `DAST_API_OVERRIDES_CMD`: Overrides command in charge of generating the overrides JSON file periodically. - `DAST_API_OVERRIDES_INTERVAL`: Interval in seconds to run command. +Optionally: + +- `DAST_API_PRE_SCRIPT`: Script to install runtimes or dependencies before the scan starts. + ```yaml stages: - dast @@ -891,11 +909,167 @@ variables: DAST_API_PROFILE: Quick DAST_API_OPENAPI: test-api-specification.json DAST_API_TARGET_URL: http://test-deployment/ - DAST_API_OVERRIDES_FILE: output/dast-api-overrides.json + DAST_API_OVERRIDES_FILE: dast-api-overrides.json DAST_API_OVERRIDES_CMD: renew_token.py DAST_API_OVERRIDES_INTERVAL: 300 ``` +#### Debugging overrides + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.8. + +By default the output of the overrides command is hidden. If the overrides command returns a non zero exit code, the command is displayed as part of your job output. Optionally, you can set the variable `DAST_API_OVERRIDES_CMD_VERBOSE` to any value in order to display overrides command output as it is generated. This is useful when testing your overrides script, but should be disabled afterwards as it slows down testing. + +It is also possible to write messages from your script to a log file that is collected when the job completes or fails. The log file must be created in a specific location and following a naming convention. + +Adding some basic logging to your overrides script is useful in case the script fails unexpectedly during normal running of the job. The log file is automatically included as an artifact of the job, allowing you to download it after the job has finished. + +Following our example, we provided `renew_token.py` in the environment variable `DAST_API_OVERRIDES_CMD`. Please notice two things in the script: + +- Log file is saved in the location indicated by the environmental variable `CI_PROJECT_DIR`. +- Log file name should match `gl-*.log`. + +```python +#!/usr/bin/env python + +# Example of an overrides command + +# Override commands can update the overrides json file +# with new values to be used. This is a great way to +# update an authentication token that will expire +# during testing. + +import logging +import json +import os +import requests +import backoff + +# [1] Store log file in directory indicated by env var CI_PROJECT_DIR +working_directory = os.environ['CI_PROJECT_DIR'] + +# [2] File name should match the pattern: gl-*.log +log_file_path = os.path.join(working_directory, 'gl-user-overrides.log') + +# Set up logger +logging.basicConfig(filename=log_file_path, level=logging.DEBUG) + +# Use `backoff` decorator to retry in case of transient errors. +@backoff.on_exception(backoff.expo, + (requests.exceptions.Timeout, + requests.exceptions.ConnectionError), + max_time=30) +def get_auth_response(): + return requests.get('https://authorization.service/api/get_api_token', auth=(os.environ['AUTH_USER'], os.environ['AUTH_PWD'])) + +# In our example, access token is retrieved from a given endpoint +try: + + # Performs a http request, response sample: + # { "Token" : "b5638ae7-6e77-4585-b035-7d9de2e3f6b3" } + response = get_auth_response() + + # Check that the request is successful. may raise `requests.exceptions.HTTPError` + response.raise_for_status() + + # Gets JSON data + response_body = response.json() + +# If needed specific exceptions can be caught +# requests.ConnectionError : A network connection error problem occurred +# requests.HTTPError : HTTP request returned an unsuccessful status code. [Response.raise_for_status()] +# requests.ConnectTimeout : The request timed out while trying to connect to the remote server +# requests.ReadTimeout : The server did not send any data in the allotted amount of time. +# requests.TooManyRedirects : The request exceeds the configured number of maximum redirections +# requests.exceptions.RequestException : All exceptions that related to Requests +except requests.exceptions.RequestException as requests_error: + # logs exceptions related to `Requests` + logging.error(f'Error, failed while performing HTTP request. Error message: {requests_error}') + raise +except requests.exceptions.JSONDecodeError as json_decode_error: + # logs errors related decoding JSON response + logging.error(f'Error, failed while decoding JSON response. Error message: {json_decode_error}') + raise +except Exception as e: + # logs any other error + logging.error(f'Error, unknown error while retrieving access token. Error message: {e}') + raise + +# computes object that holds overrides file content. +# It uses data fetched from request +overrides_data = { + "headers": { + "Authorization": f"Token {response_body['Token']}" + } +} + +# log entry informing about the file override computation +# the location of the overrides json file is also CI_PROJECT_DIR +overrides_file_path = os.path.join( + working_directory, "dast-api-overrides.json") +logging.info("Creating overrides file: %s" % overrides_file_path) + +# attempts to overwrite the file +try: + if os.path.exists(overrides_file_path): + os.unlink(overrides_file_path) + + # overwrites the file with our updated dictionary + with open(overrides_file_path, "wb+") as fd: + fd.write(json.dumps(overrides_data).encode('utf-8')) +except Exception as e: + # logs any other error + logging.error(f'Error, unkown error when overwritng file {overrides_file_path}. Error message: {e}') + raise + +# logs informing override has finished successfully +logging.info("Override file has been updated") + +# end +``` + +In the overrides command example, the Python script depends on the `backoff` library. To make sure the library is installed before executing the Python script, the `DAST_API_PRE_SCRIPT` is set to a script that will install the dependencies of your overrides command. +As for example, the following script `user-pre-scan-set-up.sh` + +```shell +#!/bin/bash + +# user-pre-scan-set-up.sh +# Ensures python dependencies are installed + +echo "**** install python dependencies ****" + +python3 -m ensurepip +pip3 install --no-cache --upgrade \ + pip \ + backoff + +echo "**** python dependencies installed ****" + +# end +``` + +You have to update your configuration to set the `DAST_API_PRE_SCRIPT` to our new `user-pre-scan-set-up.sh` script. For example: + +```yaml +stages: + - dast + +include: + - template: DAST-API.gitlab-ci.yml + +variables: + DAST_API_PROFILE: Quick + DAST_API_OPENAPI: test-api-specification.json + DAST_API_TARGET_URL: http://test-deployment/ + DAST_API_PRE_SCRIPT: user-pre-scan-set-up.sh + DAST_API_OVERRIDES_FILE: dast-api-overrides.json + DAST_API_OVERRIDES_CMD: renew_token.py + DAST_API_OVERRIDES_INTERVAL: 300 +``` + +In the previous sample, you could use the script `user-pre-scan-set-up.sh` to also install new runtimes or applications that later on you could use in our overrides command. + ### Exclude Paths > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0. diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 1b502b306bb..551488c0dc0 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -29,11 +29,16 @@ Dependency Scanning supports the following official analyzers: The analyzers are published as Docker images, which Dependency Scanning uses to launch dedicated containers for each analysis. +The Dependency Scanning analyzers' current major version number is 2. + Dependency Scanning is pre-configured with a set of **default images** that are maintained by GitLab, but users can also integrate their own **custom images**. WARNING: -The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#deprecation-of-bundler-audit-dependency-scanning-tool). +The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). + +WARNING: +The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). ## Official default analyzers diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index a8cc33d5545..a169b78a193 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -338,12 +338,12 @@ To support the following package managers, the GitLab analyzers proceed in two s | Package Manager | Preinstalled Versions | Tested Versions | | ------ | ------ | ------ | | Bundler | [2.1.4](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/blob/v2.11.3/Dockerfile#L15)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [1.17.3](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/master/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | -| sbt | [1.3.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/master/.gitlab-ci.yml#L263), [1.1.4](https://gitlab.com/gitlab-org/security-products/tests/scala-sbt-multiproject/-/blob/main/project/build.properties#L1), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/master/.gitlab-ci.yml#L272), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/master/.gitlab-ci.yml#L281), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/master/.gitlab-ci.yml#L290), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/master/.gitlab-ci.yml#L299) | +| sbt | [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L330), [1.1.4](https://gitlab.com/gitlab-org/security-products/tests/scala-sbt-multiproject/-/blob/main/project/build.properties#L1), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L339), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L348), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L357), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L366), [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L384) | | Maven | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L3) | [3.6.3](https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/blob/master/pom.xml#L3) | -| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5) | [5.6.4](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/master/gradle/wrapper/gradle-wrapper.properties#L3), [6.5](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14/gradle/wrapper/gradle-wrapper.properties#L3), [6.7-rc-1](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-15/gradle/wrapper/gradle-wrapper.properties#L3), [6.9](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14-gradle-6-9/gradle/wrapper/gradle-wrapper.properties#L3), [7.0-rc-2](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-16/gradle/wrapper/gradle-wrapper.properties#L3) | -| setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/tests/python-setuptools/-/blob/main/setup.py) | +| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5) | [5.6.4](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/master/gradle/wrapper/gradle-wrapper.properties#L3), [6.5](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14/gradle/wrapper/gradle-wrapper.properties#L3), [6.7-rc-1](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-15/gradle/wrapper/gradle-wrapper.properties#L3), [6.9](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14-gradle-6-9/gradle/wrapper/gradle-wrapper.properties#L3), [7.0-rc-2](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-16/gradle/wrapper/gradle-wrapper.properties#L3) | +| setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/tests/python-setuptools/-/blob/main/setup.py) | | pip | [20.2.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L26) | [20.x](https://gitlab.com/gitlab-org/security-products/tests/python-pip/-/blob/master/requirements.txt) | -| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/pipfile-lock-FREEZE/Pipfile.lock#L6)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/master/Pipfile) | +| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/pipfile-lock-FREEZE/Pipfile.lock#L6)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/master/Pipfile) | <!-- markdownlint-disable MD044 --> <ol> @@ -389,7 +389,8 @@ The following analyzers are executed, each of which have different behavior when Does not support multiple lockfiles. When multiple lockfiles exist, `bundler-audit` analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. -We execute both analyzers because they use different sources of vulnerability data. The result is more comprehensive analysis than if only one was executed. +WARNING: +The `bundler-audit` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#bundler-audit-dependency-scanning-tool). #### Python @@ -418,6 +419,11 @@ The following analyzers are executed, each of which have different behavior when Does not support multiple lockfiles. When multiple lockfiles exist, `Retire.js` analyzes the first lockfile discovered while traversing the directory tree in alphabetical order. +From GitLab 14.8 the `Gemnasium` analyzer scans supported JavaScript projects for vendored libraries +(that is, those checked into the project but not managed by the package manager). + +WARNING: +The `retire.js` analyzer is deprecated and will be removed in GitLab 15.0 since it duplicates the functionality of the `gemnasium` analyzer. For more information, read the [deprecation announcement](../../../update/deprecations.md#retire-js-dependency-scanning-tool). We execute both analyzers because they use different sources of vulnerability data. The result is more comprehensive analysis than if only one was executed. #### PHP, Go, C, C++, .NET, C# @@ -547,7 +553,7 @@ The following variables allow configuration of global dependency scanning settin The following variables are used for configuring specific analyzers (used for a specific language/framework). | CI/CD variable | Analyzer | Default | Description | -| ------------------------------------ | ------------------ | ---------------------------- |------------ | +|--------------------------------------| ------------------ | ---------------------------- |------------ | | `BUNDLER_AUDIT_UPDATE_DISABLED` | `bundler-audit` | `"false"` | Disable automatic updates for the `bundler-audit` analyzer. Use if you're running dependency scanning in an offline, air-gapped environment.| | `BUNDLER_AUDIT_ADVISORY_DB_URL` | `bundler-audit` | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. | | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME` | `bundler-audit` | `master` | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. | @@ -556,6 +562,7 @@ The following variables are used for configuring specific analyzers (used for a | `GEMNASIUM_DB_REMOTE_URL` | `gemnasium` | `https://gitlab.com/gitlab-org/security-products/gemnasium-db.git` | Repository URL for fetching the Gemnasium database. | | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | | `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | +| `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. | | `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | @@ -565,11 +572,36 @@ The following variables are used for configuring specific analyzers (used for a | `PIP_REQUIREMENTS_FILE` | `gemnasium-python` | | Pip requirements file to be scanned. | | `DS_PIP_VERSION` | `gemnasium-python` | | Force the install of a specific pip version (example: `"19.3"`), otherwise the pip installed in the Docker image is used. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12811) in GitLab 12.7) | | `DS_PIP_DEPENDENCY_PATH` | `gemnasium-python` | | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12412) in GitLab 12.2) | -| `DS_PYTHON_VERSION` | `retire.js` | | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12296) in GitLab 12.1, [removed](https://www.python.org/doc/sunset-python-2/) in GitLab 13.7). | | `RETIREJS_JS_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json` | Path or URL to `retire.js` JS vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | | `RETIREJS_NODE_ADVISORY_DB` | `retire.js` | `https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/npmrepository.json` | Path or URL to `retire.js` node vulnerability data file. Note that if the URL hosting the data file uses a custom SSL certificate, for example in an offline installation, you can pass the certificate in the `ADDITIONAL_CA_CERT_BUNDLE` variable. | | `RETIREJS_ADVISORY_DB_INSECURE` | `retire.js` | `false` | Enable fetching remote JS and Node vulnerability data files (defined by the `RETIREJS_JS_ADVISORY_DB` and `RETIREJS_NODE_ADVISORY_DB` variables) from hosts using an insecure or self-signed SSL (TLS) certificate. | +#### Other variables + +The previous tables are not an exhaustive list of all variables that can be used. They contain all specific GitLab and analyzer variables we support and test. There are many variables, such as environment variables, that you can pass in and they will work. This is a large list, many of which we may be unaware of, and as such is not documented. + +For example, to pass the non-GitLab environment variable `HTTPS_PROXY` to all Dependency Scanning jobs, +set it as a [custom CI/CD variable in your `.gitlab-ci.yml`](../../../ci/variables/#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file) +file like this: + +```yaml +variables: + HTTPS_PROXY: "https://squid-proxy:3128" +``` + +Alternatively we may use it in specific jobs, like Dependency Scanning: + +```yaml +dependency_scanning: + variables: + HTTPS_PROXY: $HTTPS_PROXY +``` + +As we have not tested all variables you may find some will work and others will not. +If one does not work and you need it we suggest +[submitting a feature request](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20proposal%20-%20detailed&issue[title]=Docs%20feedback%20-%20feature%20proposal:%20Write%20your%20title) +or [contributing to the code](../../../development/index.md) to enable it to be used. + ### Using a custom SSL CA certificate authority You can use the `ADDITIONAL_CA_CERT_BUNDLE` CI/CD variable to configure a custom SSL CA certificate authority. The `ADDITIONAL_CA_CERT_BUNDLE` value should contain the [text representation of the X.509 PEM public-key certificate](https://tools.ietf.org/html/rfc7468#section-5.1). For example, to configure this value in the `.gitlab-ci.yml` file, use the following: @@ -613,7 +645,7 @@ vulnerabilities in your groups, projects and pipelines. Read more about the ## Vulnerabilities database update For more information about the vulnerabilities database update, see the -[maintenance table](../vulnerabilities/index.md#vulnerability-scanner-maintenance). +[maintenance table](../index.md#vulnerability-scanner-maintenance). ## Dependency List @@ -735,6 +767,87 @@ Here's an example dependency scanning report: } ``` +### CycloneDX reports + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta-features). + +In addition to the [JSON report file](#reports-json-format), the [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) +Dependency Scanning tool outputs a [CycloneDX](https://cyclonedx.org/) report for +each supported lock or build file it detects. These CycloneDX reports are named +`cyclonedx-<package-type>-<package-manager>.json`, and are saved in the same directory +as the detected lock or build files. + +For example, if your project has the following structure: + +```plaintext +. +├── ruby-project/ +│ └── Gemfile.lock +├── ruby-project-2/ +│ └── Gemfile.lock +├── php-project/ +│ └── composer.lock +└── go-project/ + └── go.sum +``` + +Then the Gemnasium scanner generates the following CycloneDX reports: + +```plaintext +. +├── ruby-project/ +│ ├── Gemfile.lock +│ └── cyclonedx-gem-bundler.json +├── ruby-project-2/ +│ ├── Gemfile.lock +│ └── cyclonedx-gem-bundler.json +├── php-project/ +│ ├── composer.lock +│ └── cyclonedx-packagist-composer.json +└── go-project/ + ├── go.sum + └── cyclonedx-go-go.json +``` + +The CycloneDX reports can be downloaded [the same way as other job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). + +### Merging multiple CycloneDX Reports + +You can use a CI/CD job to merge multiple CycloneDX Reports into a single report. +For example: + +```yaml +stages: + - test + - merge-cyclonedx-reports + +include: + - template: Security/Dependency-Scanning.gitlab-ci.yml + +merge cyclonedx reports: + stage: merge-cyclonedx-reports + image: alpine:latest + script: + - wget https://github.com/CycloneDX/cyclonedx-cli/releases/download/v0.22.0/cyclonedx-linux-musl-x64 -O /usr/local/bin/cyclonedx-cli + - chmod 755 /usr/local/bin/cyclonedx-cli + - apk --update add --no-cache icu-dev libstdc++ + - find * -name "cyclonedx-*.json" -exec cyclonedx-cli merge --input-files {} --output-file cyclonedx-all.json + + artifacts: + paths: + - cyclonedx-all.json +``` + +GitLab uses [CycloneDX Properties](https://cyclonedx.org/use-cases/#properties--name-value-store) +to store implementation-specific details in the metadata of each CycloneDX report, +such as the location of build and lock files. If multiple CycloneDX reports are merged together, +this information is removed from the resulting merged file. + +NOTE: +CycloneDX reports are a [Beta](../../../policy/alpha-beta-support.md#beta-features) feature, +and the reports are subject to change during the beta period. Do not build integrations +that rely on the format of these reports staying consistent, as the format might change +before the feature is made generally available. + ## Versioning and release process Please check the [Release Process documentation](https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md). @@ -789,7 +902,7 @@ registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved process by which external resources can be imported or temporarily accessed. -These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +These scanners are [periodically updated](../index.md#vulnerability-scanner-maintenance) with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 4d5c1da3c47..89d3531bccd 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -36,9 +36,15 @@ GitLab IaC scanning supports a variety of IaC configuration files. Our IaC secur |------------------------------------------|----------------------------------|-------------------------------| | Ansible | [KICS](https://kics.io/) | 14.5 | | AWS CloudFormation | [KICS](https://kics.io/) | 14.5 | +| Azure Resource Manager <sup>1</sup> | [KICS](https://kics.io/) | 14.5 | +| Dockerfile | [KICS](https://kics.io/) | 14.5 | +| Google Deployment Manager | [KICS](https://kics.io/) | 14.5 | | Kubernetes | [KICS](https://kics.io/) | 14.5 | +| OpenAPI | [KICS](https://kics.io/) | 14.5 | | Terraform | [KICS](https://kics.io/) | 14.5 | +1. IaC scanning can analyze Azure Resource Manager templates in JSON format. If you write templates in the [Bicep](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview) language, you must use [the bicep CLI](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-cli) to convert your Bicep files into JSON before GitLab IaC scanning can analyze them. + ### Making IaC analyzers available to all GitLab tiers All open source (OSS) analyzers are available with the GitLab Free tier. Future proprietary analyzers may be restricted to higher tiers. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index f5c727a1418..6a0b81335fd 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -2,7 +2,6 @@ stage: Secure group: Static Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, howto --- # Secure your application **(ULTIMATE)** @@ -46,6 +45,25 @@ GitLab uses the following tools to scan and report known vulnerabilities found i | [Coverage fuzzing](coverage_fuzzing/index.md) | Find unknown bugs and vulnerabilities with coverage-guided fuzzing. | | [Cluster Image Scanning](cluster_image_scanning/index.md) | Scan Kubernetes clusters for known vulnerabilities. | +## Vulnerability scanner maintenance + +The following vulnerability scanners and their databases are regularly updated: + +| Secure scanning tool | Vulnerabilities database updates | +|:----------------------------------------------------------------|:---------------------------------| +| [Container Scanning](container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. For more details, see [Vulnerabilities database update](container_scanning/index.md#vulnerabilities-database-update). | +| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Security Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | +| [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | +| [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | + +In versions of GitLab that use the same major version of the analyzer, you do not have to update +GitLab to benefit from the latest vulnerabilities definitions. The security tools are released as +Docker images. The vendored job definitions that enable them use major release tags according to +[semantic versioning](https://semver.org/). Each new release of the tools overrides these tags. +Although in a major analyzer version you automatically get the latest versions of the scanning +tools, there are some [known issues](https://gitlab.com/gitlab-org/gitlab/-/issues/9725) with this +approach. + ## Security scanning with Auto DevOps To enable all GitLab Security scanning tools, with default settings, enable @@ -98,10 +116,10 @@ base address for Docker images. You can override this globally by setting the CI the container-scanning analyzer which uses `registry.gitlab.com/security-products/container-scanning` as its registry. -### Use security scanning tools with Pipelines for Merge Requests +### Use security scanning tools with merge request pipelines By default, the application security jobs are configured to run for branch pipelines only. -To use them with [pipelines for merge requests](../../ci/pipelines/merge_request_pipelines.md), +To use them with [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md), you may need to override the default `rules:` configuration to add: ```yaml @@ -149,8 +167,8 @@ The artifact generated by the secure analyzer contains all findings it discovers ### All tiers Merge requests which have run security scans let you know that the generated -reports are available to download. To download a report, click on the -**Download results** dropdown, and select the desired report. +reports are available to download. To download a report, select +**Download results**, and select the desired report.  @@ -200,12 +218,17 @@ security issues: ### Vulnerability-Check rule +WARNING: +This feature is in its end-of-life process. It is [deprecated](../../update/deprecations.md#vulnerability-check) +for use in GitLab 14.8, and is planned for removal in GitLab 15.0. Users should migrate to the new +[Security Approval Policies](policies/#scan-result-policy-editor). + To prevent a merge request introducing a security vulnerability in a project, enable the Vulnerability-Check rule. While this rule is enabled, additional merge request approval by [eligible approvers](../project/merge_requests/approvals/rules.md#eligible-approvers) is required when the latest security report in a merge request: -- Contains vulnerabilities with states (for example, `previously detected`, `dismissed`) matching the rule's vulnerability states. Only `newly detected` will be considered if the target branch differs from the project default branch. +- Contains vulnerabilities with states (for example, `previously detected`, `dismissed`) matching the rule's vulnerability states. Only `newly detected` are considered if the target branch differs from the project default branch. - Contains vulnerabilities with severity levels (for example, `high`, `critical`, or `unknown`) matching the rule's severity levels. - Contains a vulnerability count higher than the rule allows. @@ -218,7 +241,7 @@ An approval is optional when the security report: the rule's severity levels. - Contains a vulnerability count equal to or less than what the rule allows. -Project members assigned [at least the Maintainer role](../permissions.md#project-members-permissions) can enable or edit +Project members with at least the Maintainer role can enable or edit the Vulnerability-Check rule. #### Enable the Vulnerability-Check rule @@ -451,28 +474,24 @@ GitLab provides two methods of accomplishing this, each with advantages and disa - [Compliance framework pipelines](../project/settings/#compliance-pipeline-configuration) are recommended when: - - Scan execution enforcement is required for SAST IaC, Container Scanning, Dependency Scanning, + - Scan execution enforcement is required for SAST IaC, Dependency Scanning, License Compliance, API Fuzzing, or Coverage-guided Fuzzing. - - Scan execution enforcement of SAST or Secret Detection when customization of the default scan - variables is required. - Scan execution enforcement is required for scanners external to GitLab. - Enforced execution is required for custom jobs other than security scans. -- [Scan execution policies](policies/#scan-execution-policies) +- [Scan execution policies](policies/scan-execution-policies.md) are recommended when: - - Scan execution enforcement is required for DAST. - - Scan execution enforcement is required for SAST or Secret Detection with the default scan - variables. + - Scan execution enforcement is required for DAST, SAST, Secret Detection, or Container Scanning. - Scans are required to run on a regular, scheduled cadence. Additional details about the differences between the two solutions are outlined below: | | Compliance Framework Pipelines | Scan Execution Policies | | ------ | ------ | ------ | -| **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, and Secret Detection scans are supported. | +| **Flexibility** | Supports anything that can be done in a CI file. | Limited to only the items for which GitLab has explicitly added support. DAST, SAST, Secret Detection, and Container Scanning scans are supported. | | **Usability** | Requires knowledge of CI YAML. | Follows a `rules` and `actions`-based YAML structure. | -| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `gitlab-ci.yml` file. To include the project's `gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. | +| **Inclusion in CI pipeline** | The compliance pipeline is executed instead of the project's `gitlab-ci.yml` file. To include the project's `gitlab-ci.yml` file, use an `include` statement. Defined variables aren't allowed to be overwritten by the included project's YAML file. | Forced inclusion of a new job into the CI pipeline. DAST jobs that must be customized on a per-project basis can have project-level Site Profiles and Scan Profiles defined. To ensure separation of duties, these profiles are immutable when referenced in a scan execution policy. All jobs can be customized as part of the security policy itself with the same variables that are normally available to the CI job. | | **Schedulable** | Can be scheduled through a scheduled pipeline on the group. | Can be scheduled natively through the policy configuration itself. | | **Separation of Duties** | Only group owners can create compliance framework labels. Only project owners can apply compliance framework labels to projects. The ability to make or approve changes to the compliance pipeline definition is limited to individuals who are explicitly given access to the project that contains the compliance pipeline. | Only project owners can define a linked security policy project. The ability to make or approve changes to security policies is limited to individuals who are explicitly given access to the security policy project. | | **Ability to apply one standard to multiple projects** | The same compliance framework label can be applied to multiple projects inside a group. | The same security policy project can be used for multiple projects across GitLab with no requirement of being located in the same group. | diff --git a/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png b/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png Binary files differnew file mode 100644 index 00000000000..57649c58d8b --- /dev/null +++ b/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 11f2b91177a..803f3983b96 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -17,8 +17,9 @@ can access these by navigating to your project's **Security & Compliance > Polic GitLab supports the following security policies: -- [Container Network Policy](#container-network-policy) -- [Scan Execution Policy](#scan-execution-policy-schema) +- [Scan Execution Policy](scan-execution-policies.md) +- [Scan Result Policy](scan-result-policies.md) +- [Container Network Policy](#container-network-policy) (DEPRECATED) ## Policy management @@ -77,9 +78,62 @@ by the Rule mode, Rule mode is automatically disabled. If the YAML is incorrect, you must use YAML mode to fix your policy before Rule mode is available again. +## Security Policies project + +NOTE: +We recommend using the [Security Policies project](#security-policies-project) +exclusively for managing policies for the project. Do not add your application's source code to such +projects. + +The Security Policies feature is a repository to store policies. All security policies are stored in +the `.gitlab/security-policies/policy.yml` YAML file. The format for this YAML is specific to the type of policy that is being stored there. Examples and schema information are available for the following policy types: + +- [Scan execution policy](scan-execution-policies.md#example-security-policies-project) +- [Scan result policy](scan-result-policies.md#example-security-scan-result-policies-project) + +Policies created in this project are applied through a background job that runs once every 10 +minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. + +## Security Policy project selection + +NOTE: +Only project Owners have the [permissions](../../permissions.md#project-members-permissions) +to select Security Policy Project. + +When the Security Policy project is created and policies are created within that repository, you +must create an association between that project and the project you want to apply policies to: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Policies**. +1. Select **Edit Policy Project**, and search for and select the + project you would like to link from the dropdown menu. +1. Select **Save**. + +  + +### Unlink Security Policy projects + +Project owners can unlink Security Policy projects from development projects. To do this, follow +the steps described in [Security Policy project selection](#security-policy-project-selection), +but select the trash can icon in the modal. + +## Scan execution policies + +See [Scan execution policies](scan-execution-policies.md). + +## Scan result policy editor + +See [Scan result policies](scan-result-policies.md). + ## Container Network Policy -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32365) in GitLab 12.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32365) in GitLab 12.9. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. + +WARNING: +Container Network Policy is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) +for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) +in GitLab 15.0. The **Container Network Policy** section provides packet flow metrics for your application's Kubernetes namespace. This section has the following @@ -159,7 +213,7 @@ at the bottom of the editor. You can use policy alerts to track your policy's impact. Alerts are only available if you've [installed](../../clusters/agent/repository.md) -and [configured](../../clusters/agent/install/index.md#register-an-agent-with-gitlab) +and [configured](../../clusters/agent/install/index.md#register-the-agent-with-gitlab) an agent for this project. There are two ways to create policy alerts: @@ -176,276 +230,7 @@ There are two ways to create policy alerts: Once added, the UI updates and displays a warning about the dangers of too many alerts. -## Security Policies project - -NOTE: -We recommend using the [Security Policies project](#security-policies-project) -exclusively for managing policies for the project. Do not add your application's source code to such -projects. - -The Security Policies feature is a repository to store policies. All security policies are stored as -the `.gitlab/security-policies/policy.yml` YAML file with this format: - -```yaml ---- -scan_execution_policy: -- name: Enforce DAST in every pipeline - description: This policy enforces pipeline configuration to have a job with DAST scan - enabled: true - rules: - - type: pipeline - branches: - - master - actions: - - scan: dast - scanner_profile: Scanner Profile A - site_profile: Site Profile B -- name: Enforce DAST in every pipeline in main branch - description: This policy enforces pipeline configuration to have a job with DAST scan for main branch - enabled: true - rules: - - type: pipeline - branches: - - main - actions: - - scan: dast - scanner_profile: Scanner Profile C - site_profile: Site Profile D -``` - -## Security Policy project selection - -NOTE: -Only project Owners have the [permissions](../../permissions.md#project-members-permissions) -to select Security Policy Project. - -When the Security Policy project is created and policies are created within that repository, you -must create an association between that project and the project you want to apply policies to: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Security & Compliance > Policies**. -1. Select **Edit Policy Project**, and search for and select the - project you would like to link from the dropdown menu. -1. Select **Save**. - -  - -### Unlink Security Policy projects - -Project owners can unlink Security Policy projects from development projects. To do this, follow -the steps described in [Security Policy project selection](#security-policy-project-selection), -but select the trash can icon in the modal. - -## Scan execution policies - -Project owners can use scan execution policies to require that security scans run on a specified -schedule or with the project pipeline. Required scans are injected into the CI pipeline as new jobs -with a long, random job name. In the unlikely event of a job name collision, the security policy job -overwrites any pre-existing job in the pipeline. - -This feature has some overlap with [compliance framework pipelines](../../project/settings/#compliance-pipeline-configuration), -as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). -For details on the similarities and differences between these features, see -[Enforce scan execution](../#enforce-scan-execution). - -### Scan Execution Policy editor - -NOTE: -Only project Owners have the [permissions](../../permissions.md#project-members-permissions) -to select Security Policy Project. - -Once your policy is complete, save it by selecting **Create via merge request** -at the bottom of the editor. You are redirected to the merge request on the project's -configured security policy project. If one does not link to your project, a security -policy project is automatically created. Existing policies can also be -removed from the editor interface by selecting **Delete policy** -at the bottom of the editor. - - - -The policy editor currently only supports the YAML mode. The Rule mode is tracked in the [Allow Users to Edit Rule-mode Scan Execution Policies in the Policy UI](https://gitlab.com/groups/gitlab-org/-/epics/5363) epic. - -### Scan Execution Policies Schema - -The YAML file with Scan Execution Policies consists of an array of objects matching Scan Execution Policy Schema nested under the `scan_execution_policy` key. You can configure a maximum of 5 policies under the `scan_execution_policy` key. - -When you save a new policy, GitLab validates its contents against [this JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/validators/json_schemas/security_orchestration_policy.json). -If you're not familiar with how to read [JSON schemas](https://json-schema.org/), -the following sections and tables provide an alternative. - -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `scan_execution_policy` | `array` of Scan Execution Policy | | List of scan execution policies (maximum 5) | - -### Scan Execution Policy Schema - -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `name` | `string` | | Name of the policy. | -| `description` (optional) | `string` | | Description of the policy. | -| `enabled` | `boolean` | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | -| `rules` | `array` of rules | | List of rules that the policy applies. | -| `actions` | `array` of actions | | List of actions that the policy enforces. | - -### `pipeline` rule type - -This rule enforces the defined actions whenever the pipeline runs for a selected branch. - -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `type` | `string` | `pipeline` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | - -### `schedule` rule type - -This rule enforces the defined actions and schedules a scan on the provided date/time. - -| Field | Type | Possible values | Description | -|------------|------|-----------------|-------------| -| `type` | `string` | `schedule` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | -| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | -| `clusters` | `object` | | The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | - -#### `cluster` schema - -Use this schema to define `clusters` objects in the [`schedule` rule type](#schedule-rule-type). - -| Field | Type | Possible values | Description | -|--------------|---------------------|--------------------------|-------------| -| `containers` | `array` of `string` | | The container name that is scanned (only the first value is currently supported). | -| `resources` | `array` of `string` | | The resource name that is scanned (only the first value is currently supported). | -| `namespaces` | `array` of `string` | | The namespace that is scanned (only the first value is currently supported). | -| `kinds` | `array` of `string` | `deployment`/`daemonset` | The resource kind that should be scanned (only the first value is currently supported). | - -### `scan` action type - -This action executes the selected `scan` with additional parameters when conditions for at least one -rule in the defined policy are met. - -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `scan` | `string` | `dast`, `secret_detection`, `sast`, `container_scanning`, `cluster_image_scanning` | The action's type. | -| `site_profile` | `string` | Name of the selected [DAST site profile](../dast/index.md#site-profile). | The DAST site profile to execute the DAST scan. This field should only be set if `scan` type is `dast`. | -| `scanner_profile` | `string` or `null` | Name of the selected [DAST scanner profile](../dast/index.md#scanner-profile). | The DAST scanner profile to execute the DAST scan. This field should only be set if `scan` type is `dast`.| -| `variables` | `object` | | Set of variables applied and enforced for the selected scan. The object's key is the variable name with a value provided as a string. | - -Note the following: - -- You must create the [site profile](../dast/index.md#site-profile) and [scanner profile](../dast/index.md#scanner-profile) - with selected names for each project that is assigned to the selected Security Policy Project. - Otherwise, the policy is not applied and a job with an error message is created instead. -- Once you associate the site profile and scanner profile by name in the policy, it is not possible - to modify or delete them. If you want to modify them, you must first disable the policy by setting - the `active` flag to `false`. -- When configuring policies with a scheduled DAST scan, the author of the commit in the security - policy project's repository must have access to the scanner and site profiles. Otherwise, the scan - is not scheduled successfully. -- For a secret detection scan, only rules with the default ruleset are supported. [Custom rulesets](../secret_detection/index.md#custom-rulesets) - are not supported. -- A secret detection scan runs in `normal` mode when executed as part of a pipeline, and in - [`historic`](../secret_detection/index.md#full-history-secret-detection) - mode when executed as part of a scheduled scan. -- A container scanning and cluster image scanning scans configured for the `pipeline` rule type ignores the cluster defined in the `clusters` object. - They use predefined CI/CD variables defined for your project. Cluster selection with the `clusters` object is supported for the `schedule` rule type. - Cluster with name provided in `clusters` object must be created and configured for the project. To be able to successfully perform the `container_scanning`/`cluster_image_scanning` scans for the cluster you must follow instructions for the [Cluster Image Scanning feature](../cluster_image_scanning/index.md#prerequisites). -- The SAST scan uses the default template and runs in a [child pipeline](../../../ci/pipelines/parent_child_pipelines.md). - -### Example security policies project - -You can use this example in a `.gitlab/security-policies/policy.yml`, as described in -[Security policies project](#security-policies-project). - -```yaml ---- -scan_execution_policy: -- name: Enforce DAST in every release pipeline - description: This policy enforces pipeline configuration to have a job with DAST scan for release branches - enabled: true - rules: - - type: pipeline - branches: - - release/* - actions: - - scan: dast - scanner_profile: Scanner Profile A - site_profile: Site Profile B -- name: Enforce DAST and secret detection scans every 10 minutes - description: This policy enforces DAST and secret detection scans to run every 10 minutes - enabled: true - rules: - - type: schedule - branches: - - main - cadence: "*/10 * * * *" - actions: - - scan: dast - scanner_profile: Scanner Profile C - site_profile: Site Profile D - - scan: secret_detection -- name: Enforce Secret Detection and Container Scanning in every default branch pipeline - description: This policy enforces pipeline configuration to have a job with Secret Detection and Container Scanning scans for the default branch - enabled: true - rules: - - type: pipeline - branches: - - main - actions: - - scan: secret_detection - - scan: sast - variables: - SAST_EXCLUDED_ANALYZERS: brakeman - - scan: container_scanning -- name: Enforce Cluster Image Scanning on production-cluster every 24h - description: This policy enforces Cluster Image Scanning scan to run every 24 hours - enabled: true - rules: - - type: schedule - cadence: "15 3 * * *" - clusters: - production-cluster: - containers: - - database - resources: - - production-application - namespaces: - - production-namespace - kinds: - - deployment - actions: - - scan: cluster_image_scanning -``` - -In this example: - -- For every pipeline executed on branches that match the `release/*` wildcard (for example, branch - `release/v1.2.1`), DAST scans run with `Scanner Profile A` and `Site Profile B`. -- DAST and secret detection scans run every 10 minutes. The DAST scan runs with `Scanner Profile C` - and `Site Profile D`. -- Secret detection, container scanning, and SAST scans run for every pipeline executed on the `main` - branch. The SAST scan runs with the `SAST_EXCLUDED_ANALYZER` variable set to `"brakeman"`. -- Cluster Image Scanning scan runs every 24h. The scan runs on the `production-cluster` cluster and fetches vulnerabilities - from the container with the name `database` configured for deployment with the name `production-application` in the `production-namespace` namespace. - -### Example for scan execution policy editor - -You can use this example in the YAML mode of the [Scan Execution Policy editor](#scan-execution-policy-editor). -It corresponds to a single object from the previous example. - -```yaml -name: Enforce Secret Detection and Container Scanning in every default branch pipeline -description: This policy enforces pipeline configuration to have a job with Secret Detection and Container Scanning scans for the default branch -enabled: true -rules: - - type: pipeline - branches: - - main -actions: - - scan: secret_detection - - scan: container_scanning -``` - ## Roadmap -See the [Category Direction page](https://about.gitlab.com/direction/protect/container_network_security/) -for more information on the product direction of Container Network Security. +See the [Category Direction page](https://about.gitlab.com/direction/protect/security_orchestration/) +for more information on the product direction of security policies within GitLab. diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md new file mode 100644 index 00000000000..4e44162d5c5 --- /dev/null +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -0,0 +1,219 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Scan execution policies **(ULTIMATE)** + +Project owners can use scan execution policies to require that security scans run on a specified +schedule or with the project pipeline. Required scans are injected into the CI pipeline as new jobs +with a long, random job name. In the unlikely event of a job name collision, the security policy job +overwrites any pre-existing job in the pipeline. + +This feature has some overlap with [compliance framework pipelines](../../project/settings/#compliance-pipeline-configuration), +as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). +For details on the similarities and differences between these features, see +[Enforce scan execution](../#enforce-scan-execution). + +## Scan execution policy editor + +NOTE: +Only project Owners have the [permissions](../../permissions.md#project-members-permissions) +to select Security Policy Project. + +Once your policy is complete, save it by selecting **Create via merge request** +at the bottom of the editor. You are redirected to the merge request on the project's +configured security policy project. If one does not link to your project, a security +policy project is automatically created. Existing policies can also be +removed from the editor interface by selecting **Delete policy** +at the bottom of the editor. + +All scan execution policy changes are applied through a background job that runs once every 10 +minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. + + + +The policy editor currently only supports the YAML mode. The Rule mode is tracked in the [Allow Users to Edit Rule-mode Scan Execution Policies in the Policy UI](https://gitlab.com/groups/gitlab-org/-/epics/5363) epic. + +## Scan execution policies schema + +The YAML file with scan execution policies consists of an array of objects matching scan execution +policy schema nested under the `scan_execution_policy` key. You can configure a maximum of 5 +policies under the `scan_execution_policy` key. + +When you save a new policy, GitLab validates its contents against [this JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/validators/json_schemas/security_orchestration_policy.json). +If you're not familiar with how to read [JSON schemas](https://json-schema.org/), +the following sections and tables provide an alternative. + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `scan_execution_policy` | `array` of scan execution policy | | List of scan execution policies (maximum 5) | + +## Scan execution policy schema + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `name` | `string` | | Name of the policy. | +| `description` (optional) | `string` | | Description of the policy. | +| `enabled` | `boolean` | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | +| `rules` | `array` of rules | | List of rules that the policy applies. | +| `actions` | `array` of actions | | List of actions that the policy enforces. | + +## `pipeline` rule type + +This rule enforces the defined actions whenever the pipeline runs for a selected branch. + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `type` | `string` | `pipeline` | The rule's type. | +| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | + +## `schedule` rule type + +This rule enforces the defined actions and schedules a scan on the provided date/time. + +| Field | Type | Possible values | Description | +|------------|------|-----------------|-------------| +| `type` | `string` | `schedule` | The rule's type. | +| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. | +| `clusters` | `object` | | The cluster where the given policy enforces running selected scans (only for `container_scanning`/`cluster_image_scanning` scans). The key of the object is the name of the Kubernetes cluster configured for your project in GitLab. In the optionally provided value of the object, you can precisely select Kubernetes resources that are scanned. | + +### `cluster` schema + +Use this schema to define `clusters` objects in the [`schedule` rule type](#schedule-rule-type). + +| Field | Type | Possible values | Description | +|--------------|---------------------|--------------------------|-------------| +| `containers` | `array` of `string` | | The container name that is scanned (only the first value is currently supported). | +| `resources` | `array` of `string` | | The resource name that is scanned (only the first value is currently supported). | +| `namespaces` | `array` of `string` | | The namespace that is scanned (only the first value is currently supported). | +| `kinds` | `array` of `string` | `deployment`/`daemonset` | The resource kind that should be scanned (only the first value is currently supported). | + +## `scan` action type + +This action executes the selected `scan` with additional parameters when conditions for at least one +rule in the defined policy are met. + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `scan` | `string` | `dast`, `secret_detection`, `sast`, `container_scanning`, `cluster_image_scanning` | The action's type. | +| `site_profile` | `string` | Name of the selected [DAST site profile](../dast/index.md#site-profile). | The DAST site profile to execute the DAST scan. This field should only be set if `scan` type is `dast`. | +| `scanner_profile` | `string` or `null` | Name of the selected [DAST scanner profile](../dast/index.md#scanner-profile). | The DAST scanner profile to execute the DAST scan. This field should only be set if `scan` type is `dast`.| +| `variables` | `object` | | A set of CI variables, supplied as an array of `key: value` pairs, to apply and enforce for the selected scan. The `key` is the variable name, with its `value` provided as a string. This parameter supports any variable that the GitLab CI job supports for the specified scan. | + +Note the following: + +- You must create the [site profile](../dast/index.md#site-profile) and [scanner profile](../dast/index.md#scanner-profile) + with selected names for each project that is assigned to the selected Security Policy Project. + Otherwise, the policy is not applied and a job with an error message is created instead. +- Once you associate the site profile and scanner profile by name in the policy, it is not possible + to modify or delete them. If you want to modify them, you must first disable the policy by setting + the `active` flag to `false`. +- When configuring policies with a scheduled DAST scan, the author of the commit in the security + policy project's repository must have access to the scanner and site profiles. Otherwise, the scan + is not scheduled successfully. +- For a secret detection scan, only rules with the default ruleset are supported. [Custom rulesets](../secret_detection/index.md#custom-rulesets) + are not supported. +- A secret detection scan runs in `normal` mode when executed as part of a pipeline, and in + [`historic`](../secret_detection/index.md#full-history-secret-detection) + mode when executed as part of a scheduled scan. +- A container scanning and cluster image scanning scans configured for the `pipeline` rule type ignores the cluster defined in the `clusters` object. + They use predefined CI/CD variables defined for your project. Cluster selection with the `clusters` object is supported for the `schedule` rule type. + Cluster with name provided in `clusters` object must be created and configured for the project. To be able to successfully perform the `container_scanning`/`cluster_image_scanning` scans for the cluster you must follow instructions for the [Cluster Image Scanning feature](../cluster_image_scanning/index.md#prerequisites). +- The SAST scan uses the default template and runs in a [child pipeline](../../../ci/pipelines/parent_child_pipelines.md). + +## Example security policies project + +You can use this example in a `.gitlab/security-policies/policy.yml`, as described in +[Security policies project](index.md#security-policies-project). + +```yaml +--- +scan_execution_policy: +- name: Enforce DAST in every release pipeline + description: This policy enforces pipeline configuration to have a job with DAST scan for release branches + enabled: true + rules: + - type: pipeline + branches: + - release/* + actions: + - scan: dast + scanner_profile: Scanner Profile A + site_profile: Site Profile B +- name: Enforce DAST and secret detection scans every 10 minutes + description: This policy enforces DAST and secret detection scans to run every 10 minutes + enabled: true + rules: + - type: schedule + branches: + - main + cadence: "*/10 * * * *" + actions: + - scan: dast + scanner_profile: Scanner Profile C + site_profile: Site Profile D + - scan: secret_detection +- name: Enforce Secret Detection and Container Scanning in every default branch pipeline + description: This policy enforces pipeline configuration to have a job with Secret Detection and Container Scanning scans for the default branch + enabled: true + rules: + - type: pipeline + branches: + - main + actions: + - scan: secret_detection + - scan: sast + variables: + SAST_EXCLUDED_ANALYZERS: brakeman + - scan: container_scanning +- name: Enforce Cluster Image Scanning on production-cluster every 24h + description: This policy enforces Cluster Image Scanning scan to run every 24 hours + enabled: true + rules: + - type: schedule + cadence: "15 3 * * *" + clusters: + production-cluster: + containers: + - database + resources: + - production-application + namespaces: + - production-namespace + kinds: + - deployment + actions: + - scan: cluster_image_scanning +``` + +In this example: + +- For every pipeline executed on branches that match the `release/*` wildcard (for example, branch + `release/v1.2.1`), DAST scans run with `Scanner Profile A` and `Site Profile B`. +- DAST and secret detection scans run every 10 minutes. The DAST scan runs with `Scanner Profile C` + and `Site Profile D`. +- Secret detection, container scanning, and SAST scans run for every pipeline executed on the `main` + branch. The SAST scan runs with the `SAST_EXCLUDED_ANALYZER` variable set to `"brakeman"`. +- Cluster Image Scanning scan runs every 24h. The scan runs on the `production-cluster` cluster and fetches vulnerabilities + from the container with the name `database` configured for deployment with the name `production-application` in the `production-namespace` namespace. + +## Example for scan execution policy editor + +You can use this example in the YAML mode of the [scan execution policy editor](#scan-execution-policy-editor). +It corresponds to a single object from the previous example. + +```yaml +name: Enforce Secret Detection and Container Scanning in every default branch pipeline +description: This policy enforces pipeline configuration to have a job with Secret Detection and Container Scanning scans for the default branch +enabled: true +rules: + - type: pipeline + branches: + - main +actions: + - scan: secret_detection + - scan: container_scanning +``` diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md new file mode 100644 index 00000000000..7857de8c780 --- /dev/null +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -0,0 +1,175 @@ +--- +stage: Protect +group: Container Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Scan result policies **(ULTIMATE)** + +You can use scan result policies to take action based on scan results. For example, one type of scan +result policy is a security approval policy that allows approval to be required based on the +findings of one or more security scan jobs. Scan result policies are evaluated after a CI scanning +job is fully executed. + +## Scan result policy editor + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77814) in GitLab 14.8 with a flag named `scan_result_policy`. Disabled by default. + +NOTE: +Only project Owners have the [permissions](../../permissions.md#project-members-permissions) +to select Security Policy Project. + +Once your policy is complete, save it by selecting **Create merge request** at the bottom of the +editor. This redirects you to the merge request on the project's configured security policy project. +If a security policy project doesn't link to your project, GitLab creates such a project for you. +Existing policies can also be removed from the editor interface by selecting **Delete policy** at +the bottom of the editor. + +All scan result policy changes are applied through a background job that runs once every 10 minutes. +Allow up to 10 minutes for any policy changes committed to this project to take effect. + +The policy editor only supports YAML mode. To follow work on Rule mode, see the epic +[Allow Users to Edit Rule-mode scan result policies in the Policy UI](https://gitlab.com/groups/gitlab-org/-/epics/5363). + + + +## Scan result policies schema + +The YAML file with scan result policies consists of an array of objects matching the scan result +policy schema nested under the `scan_result_policy` key. You can configure a maximum of five +policies under the `scan_result_policy` key. + +When you save a new policy, GitLab validates its contents against [this JSON schema](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/validators/json_schemas/security_orchestration_policy.json). +If you're not familiar with how to read [JSON schemas](https://json-schema.org/), +the following sections and tables provide an alternative. + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `scan_result_policy` | `array` of Scan Result Policy | | List of scan result policies (maximum 5). | + +## Scan result policy schema + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `name` | `string` | | Name of the policy. | +| `description` (optional) | `string` | | Description of the policy. | +| `enabled` | `boolean` | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | +| `rules` | `array` of rules | | List of rules that the policy applies. | +| `actions` | `array` of actions | | List of actions that the policy enforces. | + +## `scan_finding` rule type + +This rule enforces the defined actions based on the information provided. + +| Field | Type | Possible values | Description | +|------------|------|-----------------|-------------| +| `type` | `string` | `scan_finding` | The rule's type. | +| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. | +| `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | +| `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical`| The severity levels for this rule to consider. | +| `vulnerability_states` | `array` of `string` | `newly_detected`, `detected`, `confirmed`, `resolved`, `dismissed` | The vulnerability states for this rule to consider when the target branch is set to the default branch. | + +## `require_approval` action type + +This action sets an approval rule to be required when conditions are met for at least one rule in +the defined policy. + +| Field | Type | Possible values | Description | +|-------|------|-----------------|-------------| +| `type` | `string` | `require_approval` | The action's type. | +| `approvals_required` | `integer` | Greater than or equal to zero | The number of MR approvals required. | +| `user_approvers` | `array` of `string` | Username of one of more users | The users to consider as approvers. | +| `user_approvers_ids` | `array` of `integer` | ID of one of more users | The IDs of users to consider as approvers. | +| `group_approvers` | `array` of `string` | Path of one of more groups | The groups to consider as approvers. | +| `group_approvers_ids` | `array` of `integer` | ID of one of more groups | The IDs of groups to consider as approvers. | + +Requirements and limitations: + +- You must add the respective [security scanning tools](../index.md#security-scanning-tools). + Otherwise, scan result policies won't have any effect. +- The maximum number of policies is five. +- Each policy can have a maximum of five rules. + +## Example security scan result policies project + +You can use this example in a `.gitlab/security-policies/policy.yml`, as described in +[Security policies project](index.md#security-policies-project): + +```yaml +--- +scan_result_policy: +- name: critical vulnerability CS approvals + description: critical severity level only for container scanning + enabled: true + rules: + - type: scan_finding + branches: + - main + scanners: + - container_scanning + vulnerabilities_allowed: 0 + severity_levels: + - critical + vulnerability_states: + - newly_detected + actions: + - type: require_approval + approvals_required: 1 + user_approvers: + - adalberto.dare +- name: secondary CS approvals + description: secondary only for container scanning + enabled: true + rules: + - type: scan_finding + branches: + - main + scanners: + - container_scanning + vulnerabilities_allowed: 1 + severity_levels: + - low + - unknown + vulnerability_states: + - newly_detected + actions: + - type: require_approval + approvals_required: 1 + user_approvers: + - sam.white +``` + +In this example: + +- Every MR that contains new `critical` vulnerabilities identified by container scanning requires + one approval from `alberto.dare`. +- Every MR that contains more than one new `low` or `unknown` vulnerability identified by container + scanning requires one approval from `sam.white`. + +## Example for Scan Result Policy editor + +You can use this example in the YAML mode of the [Scan Result Policy editor](#scan-result-policy-editor). +It corresponds to a single object from the previous example: + +```yaml +- name: critical vulnerability CS approvals + description: critical severity level only for container scanning + enabled: true + rules: + - type: scan_finding + branches: + - main + scanners: + - container_scanning + vulnerabilities_allowed: 1 + severity_levels: + - critical + vulnerability_states: + - newly_detected + actions: + - type: require_approval + approvals_required: 1 + user_approvers: + - adalberto.dare +``` diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index de2f9af82c7..3c0a2caf114 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -288,12 +288,14 @@ brakeman-sast: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235382) in GitLab 13.5. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/339614) support for > passthrough chains. Expanded to include additional passthrough types of `file`, `git`, and `url` in GitLab 14.6. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) support for overriding rules in GitLab 14.8. You can customize the default scanning rules provided by our SAST analyzers. -Ruleset customization supports two capabilities that can be used +Ruleset customization supports the following that can be used simultaneously: - [Disabling predefined rules](index.md#disable-predefined-analyzer-rules). Available for all analyzers. +- [Overriding predefined rules](index.md#override-predefined-analyzer-rules). Available for all analyzers. - Modifying the default behavior of a given analyzer by [synthesizing and passing a custom configuration](index.md#synthesize-a-custom-configuration). Available for only `nodejs-scan`, `gosec`, and `semgrep`. To customize the default scanning rules, create a file containing custom rules. These rules @@ -343,6 +345,50 @@ and `sobelow` by matching the `type` and `value` of identifiers: value = "sql_injection" ``` +#### Override predefined analyzer rules + +To override analyzer rules: + +1. In one or more `ruleset.identifier` subsections, list the rules that you want to override. Every `ruleset.identifier` section has: + + - a `type` field, to name the predefined rule identifier that the targeted analyzer uses. + - a `value` field, to name the rule to be overridden. + +1. In the `ruleset.override` context of a `ruleset` section, + provide the keys to override. Any combination of keys can be + overridden. Valid keys are: + + - description + - message + - name + - severity (valid options are: Critical, High, Medium, Low, Unknown, Info) + +##### Example: Override predefined rules of SAST analyzers + +In the following example, rules from `eslint` +and `gosec` are matched by the `type` and `value` of identifiers and +then overridden: + +```toml +[eslint] + [[eslint.ruleset]] + [eslint.ruleset.identifier] + type = "eslint_rule_id" + value = "security/detect-object-injection" + [eslint.ruleset.override] + description = "OVERRIDDEN description" + message = "OVERRIDDEN message" + name = "OVERRIDDEN name" + severity = "Critical" +[gosec] + [[gosec.ruleset]] + [gosec.ruleset.identifier] + type = "CWE" + value = "CWE-79" + [gosec.ruleset.override] + severity = "Critical" +``` + #### Synthesize a custom configuration To create a custom configuration, you can use passthrough chains. @@ -639,15 +685,33 @@ variables: ### Pre-compilation -If your project requires custom build configurations, it can be preferable to avoid -compilation during your SAST execution and instead pass all job artifacts from an -earlier stage in the pipeline. This is the current strategy when requiring -a `before_script` execution to prepare your scan job. +Most GitLab SAST analyzers directly scan your source code without compiling it first. +However, for technical reasons, some analyzers can only scan compiled code. + +By default, these analyzers automatically attempt to fetch dependencies and compile your code so it can be scanned. +Automatic compilation can fail if: + +- your project requires custom build configurations. +- you use language versions that aren't built into the analyzer. + +To resolve these issues, you can skip the analyzer's compilation step and directly provide artifacts from an earlier stage in your pipeline instead. +This strategy is called _pre-compilation_. + +Pre-compilation is available for the analyzers that support the `COMPILE` CI/CD variable. +See [Analyzer settings](#analyzer-settings) for the current list. + +To use pre-compilation: + +1. Output your project's dependencies to a directory in the project's working directory, then save that directory as an artifact by [setting the `artifacts: paths` configuration](../../../ci/yaml/index.md#artifactspaths). +1. Provide the `COMPILE: "false"` CI/CD variable to the analyzer to disable automatic compilation. +1. Add your compilation stage as a dependency for the analyzer job. + +To allow the analyzer to recognize the compiled artifacts, you must explicitly specify the path to +the vendored directory. +This configuration can vary per analyzer. For Maven projects, you can use `MAVEN_REPO_PATH`. +See [Analyzer settings](#analyzer-settings) for the complete list of available options. -To pass your project's dependencies as artifacts, the dependencies must be included -in the project's working directory and specified using the `artifacts:path` configuration. -If all dependencies are present, the `COMPILE=false` CI/CD variable can be provided to the -analyzer and compilation is skipped: +The following example pre-compiles a Maven project and provides it to the SpotBugs SAST analyzer: ```yaml stages: @@ -678,11 +742,6 @@ spotbugs-sast: sast: gl-sast-report.json ``` -To allow the analyzer to recognize the compiled artifacts, you must explicitly specify the path to -the vendored directory. This configuration can vary per analyzer but in the case of Java above, you -can use `MAVEN_REPO_PATH`. See -[Analyzer settings](#analyzer-settings) for the complete list of available options. - ### Available CI/CD variables SAST can be configured using the [`variables`](../../../ci/yaml/index.md#variables) parameter in @@ -779,7 +838,7 @@ Some analyzers can be customized with CI/CD variables. | `SAST_GOSEC_CONFIG` | Gosec | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328301)** in GitLab 14.0 - use custom rulesets instead. Path to configuration for Gosec (optional). | | `PHPCS_SECURITY_AUDIT_PHP_EXTENSIONS` | phpcs-security-audit | Comma separated list of additional PHP Extensions. | | `SAST_DISABLE_BABEL` | NodeJsScan | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64025)** in GitLab 13.5 | -| `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://r2c.dev/). Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330565) in GitLab 14.0. | +| `SAST_SEMGREP_METRICS` | Semgrep | Set to `"false"` to disable sending anonymized scan metrics to [r2c](https://r2c.dev/). Default: `true`. Introduced in GitLab 14.0 from the [confidential issue](../../project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab/-/issues/330565`. | #### Custom CI/CD variables @@ -819,86 +878,18 @@ variables: ## Reports JSON format -The SAST tool emits a JSON report file. For more information, see the -[schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). - -The JSON report file can be downloaded from the CI pipelines page, or the -pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. For more information see [Downloading artifacts](../../../ci/pipelines/job_artifacts.md). - -Here's an example SAST report: - -```json-doc -{ - "version": "2.0", - "vulnerabilities": [ - { - "id": "9e96e0ab-23da-4d7d-a09e-0acbaa5e83ca", - "category": "sast", - "name": "Predictable pseudorandom number generator", - "message": "Predictable pseudorandom number generator", - "description": "The use of java.util.Random is predictable", - "severity": "Medium", - "confidence": "Medium", - "scanner": { - "id": "find_sec_bugs", - "name": "Find Security Bugs" - }, - "location": { - "file": "groovy/src/main/groovy/com/gitlab/security_products/tests/App.groovy", - "start_line": 47, - "end_line": 47, - "class": "com.gitlab.security_products.tests.App", - "method": "generateSecretToken2", - "dependency": { - "package": {} - } - }, - "identifiers": [ - { - "type": "find_sec_bugs_type", - "name": "Find Security Bugs-PREDICTABLE_RANDOM", - "value": "PREDICTABLE_RANDOM", - "url": "https://find-sec-bugs.github.io/bugs.htm#PREDICTABLE_RANDOM" - }, - { - "type": "cwe", - "name": "CWE-330", - "value": "330", - "url": "https://cwe.mitre.org/data/definitions/330.html" - } - ] - }, - { - "id": "e6dbf91f-4c07-46f7-a365-0169489c27d1", - "category": "sast", - "message": "Probable insecure usage of temp file/directory.", - "severity": "Medium", - "confidence": "Medium", - "scanner": { - "id": "bandit", - "name": "Bandit" - }, - "location": { - "file": "python/hardcoded/hardcoded-tmp.py", - "start_line": 10, - "end_line": 10, - "dependency": { - "package": {} - } - }, - "identifiers": [ - { - "type": "bandit_test_id", - "name": "Bandit Test ID B108", - "value": "B108", - "url": "https://docs.openstack.org/bandit/latest/plugins/b108_hardcoded_tmp_directory.html" - } - ] - }, - ], - "remediations": [] -} -``` +SAST outputs a report file in JSON format. The report file contains details of all found vulnerabilities. +To download the report file, you can either: + +- Download the file from the CI/CD pipelines page. +- In the pipelines tab on merge requests, set [`artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. + +For information, see [Download job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). + +For details of the report file's schema, see +[SAST report file schema](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/sast-report-format.json). + +For an example SAST report file, see [`gl-secret-detection-report.json`](https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/master/qa/expect/secrets/gl-secret-detection-report.json) example. ## Running SAST in an offline environment @@ -945,7 +936,7 @@ registry.gitlab.com/security-products/sast/spotbugs:2 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../index.md#vulnerability-scanner-maintenance) with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index c5761a5743f..2ce2d59898f 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -63,7 +63,7 @@ as shown in the following table: | [Configure Secret Detection Scanners](#configuration) | **{check-circle}** | **{check-circle}** | | [Customize Secret Detection Settings](#customizing-settings) | **{check-circle}** | **{check-circle}** | | View [JSON Report](../sast/index.md#reports-json-format) | **{check-circle}** | **{check-circle}** | -| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | +| Presentation of JSON Report in merge request | **{dotted-circle}** | **{check-circle}** | | View identified secrets in the pipelines' **Security** tab | **{dotted-circle}** | **{check-circle}** | | [Interaction with Vulnerabilities](../vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | | [Access to Security Dashboard](../security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | @@ -182,14 +182,89 @@ Secret Detection can be customized by defining available CI/CD variables: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211387) in GitLab 13.5. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/339614) support for > passthrough chains. Expanded to include additional passthrough types of `file`, `git`, and `url` in GitLab 14.6. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/235359) support for overriding rules in GitLab 14.8. You can customize the default secret detection rules provided with GitLab. +Ruleset customization supports the following capabilities that can be used +simultaneously: + +- [Disabling predefined rules](index.md#disable-predefined-analyzer-rules). +- [Overriding predefined rules](index.md#override-predefined-analyzer-rules). +- Modifying the default behavior of the Secret Detection analyzer by [synthesizing and passing a custom configuration](index.md#synthesize-a-custom-configuration). Available for only `nodejs-scan`, `gosec`, and `semgrep`. + Customization allows replacing the default secret detection rules with rules that you define. To create a custom ruleset: 1. Create a `.gitlab` directory at the root of your project, if one doesn't already exist. 1. Create a custom ruleset file named `secret-detection-ruleset.toml` in the `.gitlab` directory. + +#### Disable predefined analyzer rules + +To disable analyzer rules: + +1. Set the `disabled` flag to `true` in the context of a `ruleset` section. + +1. In one or more `ruleset.identifier` subsections, list the rules that you want disabled. Every `ruleset.identifier` section has: + + - a `type` field, to name the predefined rule identifier. + - a `value` field, to name the rule to be disabled. + +##### Example: Disable predefined rules of Secret Detection analyzer + +In the following example, the disabled rules is assigned to `secrets` +by matching the `type` and `value` of identifiers: + +```toml +[secrets] + [[secrets.ruleset]] + disable = true + [secrets.ruleset.identifier] + type = "gitleaks_rule_id" + value = "RSA private key" +``` + +#### Override predefined analyzer rules + +To override rules: + +1. In one or more `ruleset.identifier` subsections, list the rules that you want to override. Every `ruleset.identifier` section has: + + - a `type` field, to name the predefined rule identifier that the Secret Detection analyzer uses. + - a `value` field, to name the rule to be overridden. + +1. In the `ruleset.override` context of a `ruleset` section, + provide the keys to override. Any combination of keys can be + overridden. Valid keys are: + + - description + - message + - name + - severity (valid options are: Critical, High, Medium, Low, Unknown, Info) + +##### Example: Override predefined rules of Secret Detection analyzer + +In the following example, rules +are matched by the `type` and `value` of identifiers and +then overridden: + +```toml +[secrets] + [[secrets.ruleset]] + [secrets.ruleset.identifier] + type = "gitleaks_rule_id" + value = "RSA private key" + [secrets.ruleset.override] + description = "OVERRIDDEN description" + message = "OVERRIDDEN message" + name = "OVERRIDDEN name" + severity = "Info" +``` + +#### Synthesize a custom configuration + +To create a custom configuration, you can use passthrough chains. + 1. In the `secret-detection-ruleset.toml` file, do one of the following: - Define a custom ruleset: @@ -239,31 +314,8 @@ From highest to lowest severity, the logging levels are: ## Post-processing and revocation -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4639) in GitLab 13.6. - -Upon detection of a secret, GitLab supports post-processing hooks. These can be used to take actions like notifying the cloud service who issued the secret. The cloud provider can confirm the credentials and take remediation actions like revoking or reissuing a new secret and notifying the creator of the secret. Post-processing workflows vary by supported cloud providers. - -GitLab currently supports post-processing for following service providers: - -- Amazon Web Services (AWS) - -Third party cloud and SaaS providers can [express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). Learn more about the [technical details of post-processing secrets](https://gitlab.com/groups/gitlab-org/-/epics/4639). - -NOTE: -Post-processing is currently limited to a project's default branch, see the above epic for future efforts to support additional branches. - -```mermaid -sequenceDiagram - autonumber - Rails->>+Sidekiq: gl-secret-detection-report.json - Sidekiq-->+Sidekiq: Ci::BuildFinishedWorker - Sidekiq-->+RevocationAPI: GET revocable keys types - RevocationAPI-->>-Sidekiq: OK - Sidekiq->>+RevocationAPI: POST revoke revocable keys - RevocationAPI-->>-Sidekiq: ACCEPTED - RevocationAPI-->>+Cloud Vendor: revoke revocable keys - Cloud Vendor-->>+RevocationAPI: ACCEPTED -``` +Upon detection of a secret, GitLab SaaS supports post-processing hooks. +For more information, see [Post-processing and revocation](post_processing.md). ## Full History Secret Detection @@ -316,7 +368,7 @@ registry.gitlab.com/security-products/secret-detection:3 The process for importing Docker images into a local offline Docker registry depends on **your network security policy**. Please consult your IT staff to find an accepted and approved -process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../vulnerabilities/index.md#vulnerability-scanner-maintenance) +process by which external resources can be imported or temporarily accessed. These scanners are [periodically updated](../index.md#vulnerability-scanner-maintenance) with new definitions, and you may be able to make occasional updates on your own. For details on saving and transporting Docker images as a file, see Docker's documentation on @@ -370,7 +422,7 @@ For information on this, see the [general Application Security troubleshooting s ### Error: `Couldn't run the gitleaks command: exit status 2` -If a pipeline is triggered from a Merge Request containing 60 commits while the `GIT_DEPTH` variable's +If a pipeline is triggered from a merge request containing 60 commits while the `GIT_DEPTH` variable's value is less than that, the Secret Detection job fails as the clone is not deep enough to contain all of the relevant commits. For information on the current default value, see the [pipeline configuration documentation](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). @@ -395,3 +447,12 @@ secret_detection: variables: GIT_DEPTH: 100 ``` + +### `secret-detection` job fails with `ERR fatal: ambiguous argument` message + +Your `secret-detection` job can fail with `ERR fatal: ambiguous argument` error if your +repository's default branch is unrelated to the branch the job was triggered for. +See issue [!352014](https://gitlab.com/gitlab-org/gitlab/-/issues/352014) for more details. + +To resolve the issue, make sure to correctly [set your default branch](../../project/repository/branches/default.md#change-the-default-branch-name-for-a-project) on your repository. You should set it to a branch +that has related history with the branch you run the `secret-detection` job on. diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md new file mode 100644 index 00000000000..972558c3b95 --- /dev/null +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -0,0 +1,179 @@ +--- +stage: Secure +group: Static Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Secret Detection post-processing and revocation **(FREE SAAS)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4639) in GitLab 13.6. + +GitLab supports running post-processing hooks after detecting a secret. These +hooks can perform actions, like notifying the cloud service that issued the secret. +The cloud provider can then confirm the credentials and take remediation actions, like: + +- Revoking a secret. +- Reissuing a secret. +- Notifying the creator of the secret. + +GitLab SaaS supports post-processing for Amazon Web Services (AWS). +Post-processing workflows vary by supported cloud providers. + +Post-processing is limited to a project's default branch. The epic +[Post-processing of leaked secrets](https://gitlab.com/groups/gitlab-org/-/epics/4639). +contains: + +- Technical details of post-processing secrets. +- Discussions of efforts to support additional branches. + +NOTE: +Post-processing is currently limited to a project's default branch + +## High-level architecture + +This diagram describes how a post-processing hook revokes a secret within the GitLab application: + +```mermaid +sequenceDiagram + autonumber + GitLab Rails->>+Sidekiq: gl-secret-detection-report.json + Sidekiq-->+Sidekiq: StoreSecurityReportsWorker + Sidekiq-->+RevocationAPI: GET revocable keys types + RevocationAPI-->>-Sidekiq: OK + Sidekiq->>+RevocationAPI: POST revoke revocable keys + RevocationAPI-->>-Sidekiq: ACCEPTED + RevocationAPI-->>+Cloud Vendor: revoke revocable keys + Cloud Vendor-->>+RevocationAPI: ACCEPTED +``` + +## Integrate your cloud provider service with GitLab Saas + +Third party cloud and SaaS providers can [express integration interest by filling out this form](https://forms.gle/wWpvrtLRK21Q2WJL9). + +### Implement a vendor revocation receiver service + +A vendor revocation receiver service integrates with a GitLab instance to receive +a web notification and respond to leaked token requests. + +To implement a receiver service to revoke leaked tokens: + +1. Create a publicly accessible HTTP service matching the corresponding API contract + below. Your service should be idempotent and rate-limited. +1. When a pipeline corresponding to its revocable token type (in the example, `my_api_token`) + completes, GitLab sends a request to your receiver service. +1. The included URL should be publicly accessible, and contain the commit where the + leaked token can be found. For example: + + ```plaintext + POST / HTTP/2 + Accept: */* + Content-Type: application/json + X-Gitlab-Token: MYSECRETTOKEN + + [ + {"type": "my_api_token", "token":"XXXXXXXXXXXXXXXX","url": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile1.java"} + ] + ``` + +## Implement a revocation service for self-managed + +Self-managed instances interested in using the revocation capabilities must: + +- Deploy the [RevocationAPI](#high-level-architecture). +- Configure the GitLab instance to use the RevocationAPI. + +A RevocationAPI must: + +- Match a minimal API specification. +- Provide two endpoints: + - Fetching revocable token types. + - Revoking leaked tokens. +- Be rate-limited and idempotent. + +Requests to the documented endpoints are authenticated via API tokens passed in +the `Authorization` header. Request and response bodies, if present, are +expected to have the content type `application/json`. + +All endpoints may return these responses: + +- `401 Unauthorized` +- `405 Method Not Allowed` +- `500 Internal Server Error` + +### `GET /v1/revocable_token_types` + +Returns the valid `type` values for use in the `revoke_tokens` endpoint. + +NOTE: +These values match the concatenation of [the `secrets` analyzer's](index.md) +[primary identifier](../../../development/integrations/secure.md#identifiers) by means +of concatenating the `primary_identifier.type` and `primary_identifier.value`. +In the case below, a finding identifier matches: + +```json +{"type": "gitleaks_rule_id", "name": "Gitleaks rule ID GitLab Personal Access Token", "value": "GitLab Personal Access Token"} +``` + +| Status Code | Description | +| ----- | --- | +| `200` | The response body contains the valid token `type` values. | + +Example response body: + +```json +{ + "types": ["gitleaks_rule_id_gitlab_personal_access_token"] +} +``` + +### `POST /v1/revoke_tokens` + +Accepts a list of tokens to be revoked by the appropriate provider. + +| Status Code | Description | +| ----- | --- | +| `204` | All submitted tokens have been accepted for eventual revocation. | +| `400` | The request body is invalid or one of the submitted token types is not supported. The request should not be retried. | +| `429` | The provider has received too many requests. The request should be retried later. | + +Example request body: + +```json +[{ + "type": "gitleaks_rule_id_gitlab_personal_access_token", + "token": "glpat--8GMtG8Mf4EnMJzmAWDU", + "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile1.java" +}, +{ + "type": "gitleaks_rule_id_gitlab_personal_access_token", + "token": "glpat--tG84EGK33nMLLDE70zU", + "location": "https://example.com/some-repo/blob/abcdefghijklmnop/compromisedfile2.java" +}] +``` + +### Configure GitLab to interface with RevocationAPI + +You must configure the following database settings in the GitLab instance: + +- `secret_detection_token_revocation_enabled` +- `secret_detection_token_revocation_url` +- `secret_detection_token_revocation_token` +- `secret_detection_revocation_token_types_url` + +For example, to configure these values in the +[Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session): + +```ruby +::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_token: 'MYSECRETTOKEN') +::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_url: 'https://example.gitlab.com/revocation_service/v1/revoke_tokens') +::Gitlab::CurrentSettings.update!(secret_detection_revocation_token_types_url: 'https://example.gitlab.com/revocation_service/v1/revocable_token_types') +::Gitlab::CurrentSettings.update!(secret_detection_token_revocation_enabled: true) +``` + +After you configure these values, completing a pipeline performs these actions: + +1. The revocation service is triggered once. +1. A request is made to `secret_detection_revocation_token_types_url` to fetch a + list of revocable tokens. +1. Any Secret Detection findings matching the results of the `token_types` request + are included in the subsequent revocation request. diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index 937ca33c3a1..a0cfd6d9d77 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -75,7 +75,7 @@ To view the total number of vulnerabilities per scan: Depending on the type of security scanner, you can download: -- A JSON artifact that contains the security scanner [report]('https://docs.gitlab.com/ee/development/integrations/secure.html#report'). +- A JSON artifact that contains the security scanner [report](../../../development/integrations/secure.md#report). - A CSV file that contains URLs and endpoints scanned by the security scanner. To download a security scan output: diff --git a/doc/user/application_security/threat_monitoring/index.md b/doc/user/application_security/threat_monitoring/index.md index ae5f6ba0fe1..390882d4326 100644 --- a/doc/user/application_security/threat_monitoring/index.md +++ b/doc/user/application_security/threat_monitoring/index.md @@ -7,7 +7,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Threat Monitoring **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14707) in GitLab 12.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14707) in GitLab 12.9. +> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. + +WARNING: +Threat Monitoring is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) +for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) +in GitLab 15.0. The **Threat Monitoring** page provides alerts and metrics for the GitLab application runtime security features. You can access diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 9aa8a0cd3cd..7b39002bac3 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -1,5 +1,4 @@ --- -type: reference, howto stage: Secure group: Threat Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments @@ -160,23 +159,3 @@ To manually apply the patch that GitLab generated for a vulnerability: 1. Ensure your local project has the same commit checked out that was used to generate the patch. 1. Run `git apply remediation.patch`. 1. Verify and commit the changes to your branch. - -## Vulnerability scanner maintenance - -The following vulnerability scanners and their databases are regularly updated: - -| Secure scanning tool | Vulnerabilities database updates | -|:----------------------------------------------------------------|----------------------------------| -| [Container Scanning](../container_scanning/index.md) | A job runs on a daily basis to build new images with the latest vulnerability database updates from the upstream scanner. | -| [Dependency Scanning](../dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for npm packages), and `gemnasium` (the GitLab tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated on a daily basis using [data from NVD, the `ruby-advisory-db` and the GitHub Security Advisory Database as data sources](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/SOURCES.md). See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). | -| [Dynamic Application Security Testing (DAST)](../dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/main/Dockerfile#L1). The scanning rules are downloaded at scan runtime. | -| [Static Application Security Testing (SAST)](../sast/index.md) | Relies exclusively on [the tools GitLab wraps](../sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | - -You do not have to update GitLab to benefit from the latest vulnerabilities definitions. -The security tools are released as Docker images. The vendored job definitions that enable them use -major release tags according to [semantic versioning](https://semver.org/). Each new release of the -tools overrides these tags. -The Docker images are updated to match the previous GitLab releases. Although -you automatically get the latest versions of the scanning tools, -there are some [known issues](https://gitlab.com/gitlab-org/gitlab/-/issues/9725) -with this approach. diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index f3e8e98bce3..89464064ea3 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -35,10 +35,10 @@ the following tables: ## SAST -| GitLab analyzer | Outputs severity levels? | Native severity level type | Native severity level example | -|--------------------------------------------------------------------------------------------------------|--------------------------|----------------------------|------------------------------------| -| [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | **{dotted-circle}** No | N/A | N/A | -| [`brakeman`](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | **{dotted-circle}** No | N/A | N/A | +| GitLab analyzer | Outputs severity levels? | Native severity level type | Native severity level example | +|----------------------------------------------------------------------------------------------------------|--------------------------|----------------------------|------------------------------------| +| [`security-code-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) | **{check-circle}** Yes | String | `CRITICAL`, `HIGH`, `MEDIUM` in [analyzer version 3.2.0 and later](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/blob/master/CHANGELOG.md#v320). In earlier versions, hardcoded to `Unknown`. | +| [`brakeman`](https://gitlab.com/gitlab-org/security-products/analyzers/brakeman) | **{check-circle}** Yes | String | `HIGH`, `MEDIUM`, `LOW` | | [`sobelow`](https://gitlab.com/gitlab-org/security-products/analyzers/sobelow) | **{check-circle}** Yes | N/A | Hardcodes all severity levels to `Unknown` | | [`nodejs-scan`](https://gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan) | **{check-circle}** Yes | String | `INFO`, `WARNING`, `ERROR` | | [`flawfinder`](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | **{check-circle}** Yes | Integer | `0`, `1`, `2`, `3`, `4`, `5` | diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index 05ff5c511f9..ba1455ab70a 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -7,7 +7,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Vulnerability Report **(ULTIMATE)** -The Vulnerability Report provides information about vulnerabilities from scans of the default branch. It is available for projects, groups, and the Security Center. +The Vulnerability Report provides information about vulnerabilities from scans of the default branch. It contains cumulative results of all successful jobs, regardless of whether the pipeline was successful. + +The scan results from a pipeline are only ingested after all the jobs in the pipeline complete. Partial results for a pipeline with jobs in progress can be seen in the pipeline security tab. + +The report is available for projects, groups, and the Security Center. At all levels, the Vulnerability Report contains: |
