Add latest changes from gitlab-org/gitlab@13-6-stable-eev13.6.0-rc42

3 files changed, 69 insertions, 47 deletions

diff --git a/doc/user/application_security/dast/img/dast_on_demand_v13_2.png b/doc/user/application_security/dast/img/dast_on_demand_v13_2.png
deleted file mode 100644
index 045221d713c..00000000000
--- a/doc/user/application_security/dast/img/dast_on_demand_v13_2.png
+++ /dev/null
diff --git a/doc/user/application_security/dast/img/dast_single_v13_0.png b/doc/user/application_security/dast/img/dast_single_v13_0.png
index 91ed4d916ab..1d528fa0ace 100644
--- a/doc/user/application_security/dast/img/dast_single_v13_0.png
+++ b/doc/user/application_security/dast/img/dast_single_v13_0.png
diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md
index fffaf4ad26b..2f1ed2faf90 100644
--- a/doc/user/application_security/dast/index.md
+++ b/doc/user/application_security/dast/index.md
@@ -161,6 +161,12 @@ headers whose values you want masked. For details on how to mask headers, see
 
 It's also possible to authenticate the user before performing the DAST checks.
 
+**Important:** It is highly recommended that you configure the scanner to authenticate to the application,
+or it will not be able to check most of the application for security risks, as most
+of your application is likely not accessible without authentication. It is also recommended
+that you periodically confirm the scanner's authentication is still working as this tends to break over
+time due to authentication changes to the application.
+
 Create masked variables to pass the credentials that DAST uses.
 To create masked variables for the username and password, see [Create a custom variable in the UI](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui).
 Note that the key of the username variable must be `DAST_USERNAME`
@@ -185,7 +191,7 @@ The results are saved as a
 that you can later download and analyze.
 Due to implementation limitations, we always take the latest DAST artifact available.
 
-DANGER: **Danger:**
+DANGER: **Warning:**
 **NEVER** run an authenticated scan against a production server. When an authenticated
 scan is run, it may perform *any* function that the authenticated user can. This
 includes actions like modifying and deleting data, submitting forms, and following links.
@@ -423,7 +429,51 @@ A URL scan allows you to specify which parts of a website are scanned by DAST.
 
 #### Define the URLs to scan
 
-To specify the paths to scan, add a comma-separated list of the paths to the `DAST_PATHS`
+URLs to scan can be specified by either of the following methods:
+
+- Use `DAST_PATHS_FILE` environment variable to specify the name of a file containing the paths.
+- Use `DAST_PATHS` environment variable to list the paths.
+
+##### Use DAST_PATHS_FILE environment variable
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6.
+
+To define the URLs to scan in a file, create a plain text file with one path per line.
+
+```txt
+page1.html
+/page2.html
+category/shoes/page1.html
+```
+
+To scan the URLs in that file, set the environment variable `DAST_PATHS_FILE` to the path of that file.
+
+```yaml
+include:
+  - template: DAST.gitlab-ci.yml
+
+variables:
+  DAST_PATHS_FILE: url_file.txt
+```
+
+By default, DAST scans do not clone the project repository. If the file is checked in to the project, instruct the DAST job to clone the project by setting GIT_STRATEGY to fetch. The file is expected to be in the `/zap/wrk` directory.
+
+```yaml
+dast:
+  script:
+    - mkdir -p /zap/wrk
+    - cp url_file.txt /zap/wrk/url_file.txt
+    - /analyze -t $DAST_WEBSITE
+  variables:
+    GIT_STRATEGY: fetch
+    DAST_PATHS_FILE: url_file.txt
+```
+
+##### Use DAST_PATHS environment variable
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4.
+
+To specify the paths to scan in an environment variable, add a comma-separated list of the paths to the `DAST_PATHS`
 environment variable. Note that you can only scan paths of a single host.
 
 ```yaml
@@ -434,10 +484,13 @@ variables:
   DAST_PATHS=/page1.html,/category1/page1.html,/page3.html
 ```
 
-When using `DAST_PATHS`, note the following:
+When using `DAST_PATHS` and `DAST_PATHS_FILE`, note the following:
 
+- `DAST_WEBSITE` must be defined when using either `DAST_PATHS_FILE` or `DAST_PATHS`. The paths listed in either will use `DAST_WEBSITE` to build the URLs to scan
+- Spidering is disabed when `DAST_PATHS` or `DAST_PATHS_FILE` are defined
+- `DAST_PATHS_FILE` and `DAST_PATHS` can not be used together
 - The `DAST_PATHS` environment variable has a limit of about 130kb. If you have a list or paths
-  greater than this, you should create multiple DAST jobs and split the paths over each job.
+  greater than this, use `DAST_PATHS_FILE`.
 
 #### Full Scan
 
@@ -476,6 +529,7 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia
 | `SECURE_ANALYZERS_PREFIX`   | URL | Set the Docker registry base address from which to download the analyzer. |
 | `DAST_WEBSITE`  | URL | The URL of the website to scan. `DAST_API_SPECIFICATION` must be specified if this is omitted. |
 | `DAST_API_SPECIFICATION`  | URL or string | The API specification to import. The specification can be hosted at a URL, or the name of a file present in the `/zap/wrk` directory. `DAST_WEBSITE` must be specified if this is omitted. |
+| `DAST_SPIDER_START_AT_HOST`  | boolean | Set to `false` to prevent DAST from resetting the target to its host before scanning. When `true`, non-host targets `http://test.site/some_path` will be reset to `http://test.site` before scan. Default: `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258805) in GitLab 13.6. |
 | `DAST_AUTH_URL` | URL | The URL of the page containing the sign-in HTML form on the target website. `DAST_USERNAME` and `DAST_PASSWORD` are submitted with the login form to create an authenticated scan. Not supported for API scans. |
 | `DAST_USERNAME` | string | The username to authenticate to in the website. |
 | `DAST_PASSWORD` | string | The password to authenticate to in the website. |
@@ -489,15 +543,16 @@ DAST can be [configured](#customizing-the-dast-settings) using environment varia
 | `DAST_API_HOST_OVERRIDE` | string | Used to override domains defined in API specification files. Only supported when importing the API specification from a URL. Example: `example.com:8080` |
 | `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://github.com/zaproxy/zaproxy/blob/develop/docs/scanners.md). For example, `HTTP Parameter Override` has a rule ID of `10026`. **Note:** In earlier versions of GitLab the excluded rules were executed but alerts they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. |
 | `DAST_REQUEST_HEADERS` | string | Set to a comma-separated list of request header names and values. Headers are added to every request made by DAST. For example, `Cache-control: no-cache,User-Agent: DAST/1.0` |
-| `DAST_DEBUG` | boolean | Enable debug message output. Default: `false` |
-| `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. |
-| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. |
-| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. |
-| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. |
-| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false` |
-| `DAST_USE_AJAX_SPIDER` | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false` |
-| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html` |
-| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. |
+| `DAST_DEBUG` | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_SPIDER_MINS` | number | The maximum duration of the spider scan in minutes. Set to `0` for unlimited. Default: One minute, or unlimited when the scan is a full scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_HTML_REPORT` | string | The filename of the HTML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_MARKDOWN_REPORT` | string | The filename of the Markdown report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1.|
+| `DAST_XML_REPORT` | string | The filename of the XML report written at the end of a scan. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_INCLUDE_ALPHA_VULNERABILITIES` | boolean | Set to `true` to include alpha passive and active scan rules. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_USE_AJAX_SPIDER` | boolean | Set to `true` to use the AJAX spider in addition to the traditional spider, useful for crawling sites that require JavaScript. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
+| `DAST_PATHS` | string | Set to a comma-separated list of URLs for DAST to scan. For example, `/page1.html,/category1/page3.html,/page2.html`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214120) in GitLab 13.4. |
+| `DAST_PATHS_FILE` | string | The file path containing the paths within `DAST_WEBSITE` to scan. The file must be plain text with one path per line and be within `/zap/wrk`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/258825) in GitLab 13.6. |
+| `DAST_ZAP_CLI_OPTIONS` | string | ZAP server command-line options. For example, `-Xmx3072m` would set the Java maximum memory allocation pool size. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. |
 | `DAST_ZAP_LOG_CONFIGURATION` | string | Set to a semicolon-separated list of additional log4j properties for the ZAP Server. For example, `log4j.logger.org.parosproxy.paros.network.HttpSender=DEBUG;log4j.logger.com.crawljax=DEBUG` |
 
 ### DAST command-line options
@@ -548,7 +603,7 @@ variables:
 ### Cloning the project's repository
 
 The DAST job does not require the project's repository to be present when running, so by default
-[`GIT_STRATEGY`](../../../ci/yaml/README.md#git-strategy) is set to `none`.
+[`GIT_STRATEGY`](../../../ci/runners/README.md#git-strategy) is set to `none`.
 
 ### Debugging DAST jobs
 
@@ -713,10 +768,6 @@ To delete a scanner profile:
 
 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2.
 > - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3.
-> - It's deployed behind a feature flag, enabled by default.
-> - It's enabled on GitLab.com.
-> - It's able to be enabled or disabled per-project.
-> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-on-demand-scans).
 
 An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger
 the scan. You must start it manually.
@@ -747,35 +798,6 @@ To run an on-demand DAST scan, you need:
 
 The on-demand DAST scan runs and the project's dashboard shows the results.
 
-### Enable or disable On-demand Scans
-
-The On-demand DAST Scans feature is enabled by default. You can disable on-demand scans
-instance-wide, or disable it for specific projects if you prefer.
-
-To run on-demand DAST scans, an administrator must enable the
-`security_on_demand_scans_feature_flag` feature flag.
-
-[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md)
-can disable or enable the feature flags.
-
-To disable On-demand DAST Scans:
-
-```ruby
-# Instance-wide
-Feature.disable(:security_on_demand_scans_feature_flag)
-# or by project
-Feature.disable(:security_on_demand_scans_feature_flag, Project.find(<project id>))
-```
-
-To enable On-demand DAST Scans:
-
-```ruby
-# Instance-wide
-Feature.enable(:security_on_demand_scans_feature_flag)
-# or by project
-Feature.enable(:security_on_demand_scans_feature_flag, Project.find(<project ID>))
-```
-
 ## Reports
 
 The DAST tool outputs a report file in JSON format by default. However, this tool can also generate reports in