Add latest changes from gitlab-org/gitlab@12-10-stable-ee

9 files changed, 260 insertions, 22 deletions

diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md
index 27b22fb925c..68ad2d427dd 100644
--- a/doc/user/application_security/container_scanning/index.md
+++ b/doc/user/application_security/container_scanning/index.md
@@ -187,6 +187,10 @@ using environment variables.
 
 ### Overriding the Container Scanning template
 
+CAUTION: **Deprecation:**
+Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
+is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
+
 If you want to override the job definition (for example, change properties like
 `variables`), you need to declare a `container_scanning` job after the
 template inclusion and specify any additional keys under it. For example:
@@ -212,11 +216,46 @@ If you want to whitelist specific vulnerabilities, you'll need to:
 
 ### Running Container Scanning in an offline environment
 
-Container Scanning can be executed on an offline GitLab Ultimate installation by using the following process:
+For self-managed GitLab instances in an environment with limited, restricted, or intermittent access
+to external resources through the internet, some adjustments are required for the Container Scanning job to
+successfully run. For more information, see [Offline environments](../offline_deployments/index.md).
+
+#### Requirements for offline Container Scanning
+
+To use Container Scanning in an offline environment, you need:
+
+- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements).
+- To configure a local Docker Container Registry with copies of the Container Scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/klar) images, found in the [Container Scanning container registry](https://gitlab.com/gitlab-org/security-products/analyzers/klar/container_registry).
+
+NOTE: **Note:**
+GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy),
+meaning the runner may try to pull remote images even if a local copy is available. Set GitLab
+Runner's [`pull_policy` to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy)
+in an offline environment if you prefer using only locally available Docker images.
+
+#### Make GitLab Container Scanning analyzer images available inside your Docker registry
 
-1. Host the following Docker images on a [local Docker container registry](../../packages/container_registry/index.md):
-   - [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db)
-   - GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar`
+For Container Scanning, import and host the following images from `registry.gitlab.com` to your
+offline [local Docker container registry](../../packages/container_registry/index.md):
+
+- [arminc/clair-db vulnerabilities database](https://hub.docker.com/r/arminc/clair-db)
+- GitLab klar analyzer: `registry.gitlab.com/gitlab-org/security-products/analyzers/klar`
+
+The process for importing Docker images into a local offline Docker registry depends on
+**your network security policy**. Please consult your IT staff to find an accepted and approved
+process by which external resources can be imported or temporarily accessed.
+
+Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database)
+with new definitions, so consider if you are able to make periodic updates yourself.
+You can read more specific steps on how to do this [below](#automating-container-scanning-vulnerability-database-updates-with-a-pipeline).
+
+For details on saving and transporting Docker images as a file, see Docker's documentation on
+[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/),
+[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/).
+
+#### Set Container Scanning CI job variables to use local Container Scanner analyzers
+
+Container Scanning can be executed on an offline GitLab Ultimate installation using the following process:
 
 1. [Override the container scanning template](#overriding-the-container-scanning-template) in your `.gitlab-ci.yml` file to refer to the Docker images hosted on your local Docker container registry:
 
@@ -234,7 +273,12 @@ Container Scanning can be executed on an offline GitLab Ultimate installation by
    self-signed certificate, then you must set `DOCKER_INSECURE: "true"` in the above
    `container_scanning` section of your `.gitlab-ci.yml`.
 
-It may be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to automatically build a new version of the vulnerabilities database on a preset schedule. You can use the following `.gitlab-yml.ci` as a template:
+#### Automating Container Scanning vulnerability database updates with a pipeline
+
+It can be worthwhile to set up a [scheduled pipeline](../../../ci/pipelines/schedules.md) to
+automatically build a new version of the vulnerabilities database on a preset schedule. Automating
+this with a pipeline means you won't have to do it manually each time. You can use the following
+`.gitlab-yml.ci` as a template:
 
 ```yaml
 image: docker:stable
diff --git a/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png b/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png
index c15a2da513c..9f277dcb578 100644
--- a/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png
+++ b/doc/user/application_security/dast/img/dast_urls_scanned_v12_10.png
diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md
index 6f51aaf4931..abf194aae48 100644
--- a/doc/user/application_security/dast/index.md
+++ b/doc/user/application_security/dast/index.md
@@ -376,6 +376,10 @@ configuration, the last mention of the variable will take precedence.
 
 ### Overriding the DAST template
 
+CAUTION: **Deprecation:**
+Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
+is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
+
 If you want to override the job definition (for example, change properties like
 `variables` or `dependencies`), you need to declare a `dast` job after the
 template inclusion and specify any additional keys under it. For example:
diff --git a/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png b/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png
new file mode 100644
index 00000000000..2755b42f1e4
--- /dev/null
+++ b/doc/user/application_security/dependency_list/img/dependency_list_v12_10.png
diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md
index b9c3b6521d6..73d2cfeaf00 100644
--- a/doc/user/application_security/dependency_list/index.md
+++ b/doc/user/application_security/dependency_list/index.md
@@ -17,32 +17,25 @@ sidebar. This information is sometimes referred to as a Software Bill of Materia
 
 ## Viewing dependencies
 
-![Dependency List](img/dependency_list_v12_4.png)
+![Dependency List](img/dependency_list_v12_10.png)
 
 Dependencies are displayed with the following information:
 
 | Field     | Description |
 | --------- | ----------- |
-| Status    | Displays whether or not the dependency has any known vulnerabilities |
-| Component | The dependency's name |
-| Version   | The exact locked version of the dependency your project uses |
+| Component | The dependency's name and version |
 | Packager  | The packager used to install the depedency |
 | Location  | A link to the packager-specific lockfile in your project that declared the dependency |
 | License   | Links to dependency's software licenses |
 
-Dependencies shown are initially sorted by their names. They can also be sorted
-by the packager they were installed by, or by the severity of their known
-vulnerabilities.
-
-There is a second list under the `Vulnerable components` tab displaying only
-those dependencies with known vulnerabilities. If there are none, this tab is
-disabled.
+Dependencies shown are initially sorted by the severity of their known vulnerabilities, if any. They
+can also be sorted by name or by the packager that installed them.
 
 ### Vulnerabilities
 
-If a dependency has known vulnerabilities, they can be viewed by clicking on the
-`Status` cell of that dependency. The severity and description of each
-vulnerability will then be displayed below it.
+If a dependency has known vulnerabilities, you can view them by clicking the arrow next to the
+dependency's name or the badge that indicates how many known vulnerabilities exist. For each
+vulnerability, its severity and description then appears below it.
 
 ## Licenses
 
diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md
index ae006178945..cda621e61a6 100644
--- a/doc/user/application_security/dependency_scanning/index.md
+++ b/doc/user/application_security/dependency_scanning/index.md
@@ -115,6 +115,10 @@ configuration, the last mention of the variable will take precedence.
 
 ### Overriding the Dependency Scanning template
 
+CAUTION: **Deprecation:**
+Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
+is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
+
 If you want to override the job definition (for example, change properties like
 `variables` or `dependencies`), you need to declare a `dependency_scanning` job
 after the template inclusion and specify any additional keys under it. For example:
@@ -175,6 +179,8 @@ The following variables are used for configuring specific analyzers (used for a
 | `DS_PIP_DEPENDENCY_PATH`                | `gemnasium-python` |                              | Path to load Python pip dependencies from. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12412) in GitLab 12.2) |
 | `DS_PYTHON_VERSION`                     | `retire.js`        |                              | Version of Python. If set to 2, dependencies are installed using Python 2.7 instead of Python 3.6. ([Introduced](https://gitlab.com/gitlab-org/gitlab/issues/12296) in GitLab 12.1)|
 | `MAVEN_CLI_OPTS`                        | `gemnasium-maven`  | `"-DskipTests --batch-mode"` | List of command line arguments that will be passed to `maven` by the analyzer. See an example for [using private repos](../index.md#using-private-maven-repos). |
+| `GRADLE_CLI_OPTS`                        | `gemnasium-maven`  | | List of command line arguments that will be passed to `gradle` by the analyzer. |
+| `SBT_CLI_OPTS`                        | `gemnasium-maven`  | | List of command-line arguments that the analyzer will pass to `sbt`. |
 | `BUNDLER_AUDIT_UPDATE_DISABLED`         | `bundler-audit`    | `"false"`                      | Disable automatic updates for the `bundler-audit` analyzer. Useful if you're running Dependency Scanning in an offline, air-gapped environment.|
 | `BUNDLER_AUDIT_ADVISORY_DB_URL`         | `bundler-audit`    | `https://github.com/rubysec/ruby-advisory-db` | URL of the advisory database used by bundler-audit. |
 | `BUNDLER_AUDIT_ADVISORY_DB_REF_NAME`    | `bundler-audit`    | `master`                     | Git ref for the advisory database specified by `BUNDLER_AUDIT_ADVISORY_DB_URL`. |
@@ -415,6 +421,181 @@ You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-product
 to find a vulnerability in the Gemnasium database.
 You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md).
 
+## Running Dependency Scanning in an offline environment
+
+For self-managed GitLab instances in an environment with limited, restricted, or intermittent access
+to external resources through the internet, some adjustments are required for dependency scannings jobs to run successfully. For more information, see [Offline environments](../offline_deployments/index.md).
+
+### Requirements for offline Dependency Scanning
+
+Here are the requirements for using Dependency Scanning in an offline environment:
+
+- [Disable Docker-In-Docker](#disabling-docker-in-docker-for-dependency-scanning)
+- GitLab Runner with the [`docker` or `kubernetes` executor](#requirements).
+- Docker Container Registry with locally available copies of dependency scanning [analyzer](https://gitlab.com/gitlab-org/security-products/analyzers) images.
+- Host an offline Git copy of the [gemnasium-db advisory database](https://gitlab.com/gitlab-org/security-products/gemnasium-db/)
+- _Only if scanning Ruby projects_: Host an offline Git copy of the [advisory database](https://github.com/rubysec/ruby-advisory-db).
+- _Only if scanning npm/yarn projects_: Host an offline copy of the [retire.js](https://github.com/RetireJS/retire.js/) [node](https://github.com/RetireJS/retire.js/blob/master/repository/npmrepository.json) and [js](https://github.com/RetireJS/retire.js/blob/master/repository/jsrepository.json) advisory databases.
+
+NOTE: **Note:**
+GitLab Runner has a [default `pull policy` of `always`](https://docs.gitlab.com/runner/executors/docker.html#using-the-always-pull-policy),
+meaning the runner will try to pull Docker images from the GitLab container registry even if a local
+copy is available. GitLab Runner's [`pull_policy` can be set to `if-not-present`](https://docs.gitlab.com/runner/executors/docker.html#using-the-if-not-present-pull-policy)
+in an offline environment if you prefer using only locally available Docker images. However, we
+recommend keeping the pull policy setting to `always` as it will better enable updated scanners to
+be utilized within your CI/CD pipelines.
+
+### Make GitLab Dependency Scanning analyzer images available inside your Docker registry
+
+For Dependency Scanning, import docker images ([supported languages and frameworks](#supported-languages-and-package-managers))
+from `registry.gitlab.com` to your offline docker registry. The Dependency Scanning analyzer
+docker images are:
+
+```plaintext
+registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2
+registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven:2
+registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2
+registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2
+registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2
+```
+
+The process for importing Docker images into a local offline Docker registry depends on
+**your network security policy**. Please consult your IT staff to find an accepted and approved
+process by which external resources can be imported or temporarily accessed. Note that these scanners are [updated periodically](../index.md#maintenance-and-update-of-the-vulnerabilities-database)
+with new definitions, so consider if you are able to make periodic updates yourself.
+
+For details on saving and transporting Docker images as a file, see Docker's documentation on
+[`docker save`](https://docs.docker.com/engine/reference/commandline/save/), [`docker load`](https://docs.docker.com/engine/reference/commandline/load/),
+[`docker export`](https://docs.docker.com/engine/reference/commandline/export/), and [`docker import`](https://docs.docker.com/engine/reference/commandline/import/).
+
+### Set Dependency Scanning CI config for "offline" use
+
+Below is a general `.gitlab-ci.yml` template to configure your environment for running Dependency
+Scanning offline:
+
+```yaml
+include:
+  - template: Dependency-Scanning.gitlab-ci.yml
+
+variables:
+  DS_DISABLE_DIND: "true"
+  DS_ANALYZER_IMAGE_PREFIX: "docker-registry.example.com/analyzers"
+```
+
+See explanations of the variables above in the [configuration section](#configuration).
+
+### Specific settings for languages and package managers
+
+For every language and package manager, add the following to the variables section of
+`.gitlab-ci.yml`:
+
+```yaml
+GEMNASIUM_DB_REMOTE_URL: "gitlab.example.com/gemnasium-db.git"
+```
+
+See the following sections for additional instructions on specific languages and package managers.
+
+#### JavaScript (npm and yarn) projects
+
+Add the following to the variables section of `.gitlab-ci.yml`:
+
+```yaml
+RETIREJS_JS_ADVISORY_DB: "example.com/jsrepository.json"
+RETIREJS_NODE_ADVISORY_DB: "example.com/npmrepository.json"
+```
+
+#### Ruby (gem) projects
+
+Add the following to the variables section of `.gitlab-ci.yml`:
+
+```yaml
+BUNDLER_AUDIT_ADVISORY_DB_REF_NAME: "master"
+BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git"
+```
+
+#### Java (Maven) projects
+
+When using a self-signed certificates, add the following to the variables section of`.gitlab-ci.yml`:
+
+```yaml
+MAVEN_CLI_OPTS="-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true"`
+```
+
+#### Java (Gradle) projects
+
+When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`:
+
+```yaml
+gemnasium-maven-dependency_scanning:
+  variables:
+    before_script:
+      - echo -n | openssl s_client -connect maven-repo.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt
+      - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt
+```
+
+This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image.
+
+#### Scala (sbt) projects
+
+When using self-signed certificates, add the following job section to the `.gitlab-ci.yml`:
+
+```yaml
+gemnasium-maven-dependency_scanning:
+  variables:
+    before_script:
+      - echo -n | openssl s_client -connect gitlab-airgap-test.us-west1-b.c.group-secure-a89fe7.internal:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/internal.crt
+      - keytool -importcert -file /tmp/internal.crt -cacerts -storepass changeit -noprompt
+```
+
+This adds the self-signed certificates of your maven repository to the Java Key Store of the analyzer's docker image.
+
+#### Python (pip) and Python (Pipfile) projects
+
+Add the following `pip.conf` to your repository to define your index URL and trust its self-signed
+certificate:
+
+```toml
+[global]
+index-url = https://pypi.example.com
+trusted-host = pypi.example.com
+```
+
+Add the following job section to `.gitlab-ci.yml`:
+
+```yaml
+gemnasium-python-dependency_scanning:
+  before_script:
+    - mkdir ~/.config/pip
+    - cp pip.conf ~/.config/pip/pip.conf
+```
+
+#### Python (setuptools)
+
+When using self-signed certificates for your private PyPi repo no extra job configuration (aside
+from the template `.gitlab-ci.yml` above) is needed. However, you must update your `setup.py` to
+ensure that it can reach your private repo. Here is an example configuration:
+
+1. Update `setup.py` to create a `dependency_links` attribute pointing at your private repo for each
+   dependency in the `install_requires` list:
+
+    ```python
+    install_requires=['pyparsing>=2.0.3'],
+    dependency_links=['https://pypi.example.com/simple/pyparsing'],
+    ```
+
+1. Fetch the certificate from your repository URL and add it to the project:
+
+    ```bash
+    echo -n | openssl s_client -connect pypi.example.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > internal.crt
+    ```
+
+1. Point `setup.py` at the newly downloaded certificate:
+
+    ```python
+    import setuptools.ssl_support
+    setuptools.ssl_support.cert_paths = ['internal.crt']
+    ```
+
 ## Troubleshooting
 
 ### Error response from daemon: error processing tar file: docker-tar: relocation error
diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md
index 011f95c7049..a6457d58fe2 100644
--- a/doc/user/application_security/sast/index.md
+++ b/doc/user/application_security/sast/index.md
@@ -143,6 +143,10 @@ the pipeline configuration, the last mention of the variable will take precedenc
 
 ### Overriding the SAST template
 
+CAUTION: **Deprecation:**
+Beginning in GitLab 13.0, the use of [`only` and `except`](../../../ci/yaml/README.md#onlyexcept-basic)
+is no longer supported. When overriding the template, you must use [`rules`](../../../ci/yaml/README.md#rules) instead.
+
 If you want to override the job definition (for example, change properties like
 `variables` or `dependencies`), you need to declare a `sast` job after the
 template inclusion and specify any additional keys under it. For example:
diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png
new file mode 100644
index 00000000000..07b41b471d4
--- /dev/null
+++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard_export_csv_v12.10.png
diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md
index 1eef6b9b696..42b28b7b9f2 100644
--- a/doc/user/application_security/security_dashboard/index.md
+++ b/doc/user/application_security/security_dashboard/index.md
@@ -34,13 +34,13 @@ To use the instance, group, project, or pipeline security dashboard:
 1. [GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or newer must be used.
    If you're using the shared Runners on GitLab.com, this is already the case.
 
-## Pipeline Security Dashboard
+## Pipeline Security
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/13496) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.3.
 
-At the pipeline level, the Security Dashboard displays the vulnerabilities present in the branch of the project the pipeline was run against.
+At the pipeline level, the Security section displays the vulnerabilities present in the branch of the project the pipeline was run against.
 
-Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security Dashboard.
+Visit the page for any pipeline which has run any of the [supported reports](#supported-reports). Click the **Security** tab to view the Security findings.
 
 ![Pipeline Security Dashboard](img/pipeline_security_dashboard_v12_6.png)
 
@@ -54,6 +54,18 @@ for your project from the last successful pipeline. Use it to find and fix vulne
 
 ![Project Security Dashboard](img/project_security_dashboard_v12_3.png)
 
+### Export vulnerabilities
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10.
+
+You can export all your project's vulnerabilities as CSV by clicking on the export button located at top right of the Project Security Dashboard. This will initiate the process, and once complete, the CSV report will be downloaded. The report will contain all vulnerabilities in the project as filters won't apply.
+
+NOTE: **Note:**
+It may take several minutes for the download to start if your project consists
+of thousands of vulnerabilities. Do not close the page until the download finishes.
+
+![CSV Export Button](img/project_security_dashboard_export_csv_v12.10.png)
+
 ## Group Security Dashboard
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/6709) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5.