diff options
Diffstat (limited to 'doc/user/application_security/api_fuzzing/index.md')
| -rw-r--r-- | doc/user/application_security/api_fuzzing/index.md | 191 |
1 files changed, 185 insertions, 6 deletions
diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 49311ccc7cd..fcb41b599ae 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -25,7 +25,7 @@ you can run fuzz tests as part your CI/CD workflow. - GraphQL - Form bodies, JSON, or XML - One of the following assets to provide APIs to test: - - OpenAPI v2 API definition + - OpenAPI v2 or v3 API definition - HTTP Archive (HAR) of API requests to test - Postman Collection v2.0 or v2.1 @@ -54,7 +54,7 @@ changes, other pipelines, or other scanners) during a scan could cause inaccurat There are three ways to perform scans. See the configuration section for the one you wish to use: -- [OpenAPI v2 specification](#openapi-specification) +- [OpenAPI v2 or v3 specification](#openapi-specification) - [HTTP Archive (HAR)](#http-archive-har) - [Postman Collection v2.0 or v2.1](#postman-collection) @@ -64,6 +64,12 @@ Examples of both configurations can be found here: - [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing-example/-/tree/har) - [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-fuzzing/postman-api-fuzzing-example) +WARNING: +GitLab 14.0 will require that you place API fuzzing configuration files (for example, +`gitlab-api-fuzzing-config.yml`) in your repository's `.gitlab` directory instead of your +repository's root. You can continue using your existing configuration files as they are, but +starting in GitLab 14.0, GitLab will not check your repository's root for configuration files. + ### OpenAPI Specification The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an @@ -71,6 +77,13 @@ API description format for REST APIs. This section shows you how to configure AP an OpenAPI specification to provide information about the target API to test. OpenAPI specifications are provided as a file system resource or URL. +API fuzzing uses an OpenAPI document to generate the request body. When a request body is required, +the body generation is limited to these body types: + +- `application/x-www-form-urlencoded` +- `multipart/form-data` +- `application/json` + Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: 1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) @@ -100,7 +113,7 @@ Follow these steps to configure API fuzzing in GitLab with an OpenAPI specificat FUZZAPI_PROFILE: Quick-10 ``` -1. Provide the location of the OpenAPI v2 specification. You can provide the specification as a file +1. Provide the location of the OpenAPI specification. You can provide the specification as a file or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable: ```yaml @@ -327,6 +340,60 @@ WARNING: the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting data. Only run fuzzing against a test server. +#### Postman variables + +Postman allows the developer to define placeholders that can be used in different parts of the +requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/). +You can use variables to store and reuse values in your requests and scripts. For example, you can +edit the collection to add variables to the document: + + + +You can then use the variables in sections such as URL, headers, and others: + + + +Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes) +(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at +the Environment scope: + + + +When you export a Postman collection, only Postman collection variables are exported into the +Postman file. For example, Postman does not export environment-scoped variables into the Postman +file. + +By default, the API fuzzer uses the Postman file to resolve Postman variable values. If a JSON file +is set in a GitLab CI environment variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON +file takes precedence to get Postman variable values. + +Although Postman can export environment variables into a JSON file, the format is not compatible +with the JSON expected by `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`. + +Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`: + +```yaml +include: + - template: API-Fuzzing.gitlab-ci.yml + +variables: + FUZZAPI_PROFILE: Quick-10 + FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json + FUZZAPI_TARGET_URL: http://test-deployment/ + FUZZAPI_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json +``` + +The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with +key-value pairs for properties. The keys are the variables' names, and the values are the variables' +values. For example: + + ```json + { + "base_url": "http://127.0.0.1/", + "token": "Token 84816165151" + } + ``` + ### Authentication Authentication is handled by providing the authentication token as a header or cookie. You can @@ -502,6 +569,7 @@ repository's root as `.gitlab-api-fuzzing.yml`. |[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. | |[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | |[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | +|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. | |[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | |[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | @@ -523,11 +591,19 @@ repository's root as `.gitlab-api-fuzzing.yml`. ### Overrides -API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests. +API Fuzzing provides a method to add or override specific items in your request, for example: + +- Headers +- Cookies +- Query string +- Form data +- JSON nodes +- XML nodes + You can use this to inject semantic version headers, authentication, and so on. The [authentication section](#authentication) includes examples of using overrides for that purpose. -Overrides use a JSON document to define the headers and cookies: +Overrides use a JSON document, where each type of override is represented by a JSON object: ```json { @@ -538,6 +614,22 @@ Overrides use a JSON document to define the headers and cookies: "cookies": { "cookie1": "value", "cookie2": "value" + }, + "query": { + "query-string1": "value", + "query-string2": "value" + }, + "body-form": { + "form-param1": "value", + "form-param1": "value", + }, + "body-json": { + "json-path1": "value", + "json-path2": "value", + }, + "body-xml" : { + "xpath1": "value", + "xpath2": "value", } } ``` @@ -565,7 +657,94 @@ Example of setting both a header and cookie: } ``` -You can provide this JSON document as a file or CI/CD variable. You may also provide a command +Example usage for setting a `body-form` override: + +```json +{ + "body-form": { + "username": "john.doe" + } +} +``` + +The override engine uses `body-form` when the request body has only form-data content. + +Example usage for setting a `body-json` override: + +```json +{ + "body-json": { + "$.credentials.access-token": "iddqd!42.$" + } +} +``` + +Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/) +expression. The JSON Path expression `$.credentials.access-token` identifies the node to be +overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body +has only [JSON](https://www.json.org/json-en.html) content. + +For example, if the body is set to the following JSON: + +```json +{ + "credentials" : { + "username" :"john.doe", + "access-token" : "non-valid-password" + } +} +``` + +It is changed to: + +```json +{ + "credentials" : { + "username" :"john.doe", + "access-token" : "iddqd!42.$" + } +} +``` + +Here's an example for setting a `body-xml` override. The first entry overrides an XML attribute and +the second entry overrides an XML element: + +```json +{ + "body-xml" : { + "/credentials/@isEnabled": "true", + "/credentials/access-token/text()" : "iddqd!42.$" + } +} +``` + +Note that each JSON property name in the object `body-xml` is set to an +[XPath v2](https://www.w3.org/TR/xpath20/) +expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override +with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the +element node to override with the value `iddqd!42.$`. The override engine uses `body-xml` when the +request body has only [XML](https://www.w3.org/XML/) +content. + +For example, if the body is set to the following XML: + +```xml +<credentials isEnabled="false"> + <username>john.doe</username> + <access-token>non-valid-password</access-token> +</credentials> +``` + +It is changed to: + +```xml +<credentials isEnabled="true"> + <username>john.doe</username> + <access-token>iddqd!42.$</access-token> +</credentials> +``` + +You can provide this JSON document as a file or environment variable. You may also provide a command to generate the JSON document. The command can run at intervals to support values that expire. #### Using a file |