summaryrefslogtreecommitdiff
path: root/doc/ci
diff options
context:
space:
mode:
Diffstat (limited to 'doc/ci')
-rw-r--r--doc/ci/docker/using_docker_images.md63
-rw-r--r--doc/ci/examples/README.md7
-rw-r--r--doc/ci/examples/code_climate.md22
-rw-r--r--doc/ci/examples/container_scanning.md55
-rw-r--r--doc/ci/examples/sast_docker.md56
-rw-r--r--doc/ci/yaml/README.md8
6 files changed, 142 insertions, 69 deletions
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md
index fb5bfe26bb0..bc5d3840368 100644
--- a/doc/ci/docker/using_docker_images.md
+++ b/doc/ci/docker/using_docker_images.md
@@ -58,7 +58,7 @@ your job and is linked to the Docker image that the `image` keyword defines.
This allows you to access the service image during build time.
The service image can run any application, but the most common use case is to
-run a database container, eg. `mysql`. It's easier and faster to use an
+run a database container, e.g., `mysql`. It's easier and faster to use an
existing image and run it as an additional container than install `mysql` every
time the project is built.
@@ -83,6 +83,67 @@ So, in order to access your database service you have to connect to the host
named `mysql` instead of a socket or `localhost`. Read more in [accessing the
services](#accessing-the-services).
+### How the health check of services works
+
+Services are designed to provide additional functionality which is **network accessible**.
+It may be a database like MySQL, or Redis, and even `docker:dind` which
+allows you to use Docker in Docker. It can be practically anything that is
+required for the CI/CD job to proceed and is accessed by network.
+
+To make sure this works, the Runner:
+
+1. checks which ports are exposed from the container by default
+1. starts a special container that waits for these ports to be accessible
+
+When the second stage of the check fails, either because there is no opened port in the
+service, or the service was not started properly before the timeout and the port is not
+responding, it prints the warning: `*** WARNING: Service XYZ probably didn't start properly`.
+
+In most cases it will affect the job, but there may be situations when the job
+will still succeed even if that warning was printed. For example:
+
+- The service was started a little after the warning was raised, and the job is
+ not using the linked service from the very beginning. In that case, when the
+ job needed to access the service, it may have been already there waiting for
+ connections.
+- The service container is not providing any networking service, but it's doing
+ something with the job's directory (all services have the job directory mounted
+ as a volume under `/builds`). In that case, the service will do its job, and
+ since the job is not trying to connect to it, it won't fail.
+
+### What services are not for
+
+As it was mentioned before, this feature is designed to provide **network accessible**
+services. A database is the simplest example of such a service.
+
+NOTE: **Note:**
+The services feature is not designed to, and will not add any software from the
+defined `services` image(s) to the job's container.
+
+For example, if you have the following `services` defined in your job, the `php`,
+`node` or `go` commands will **not** be available for your script, and thus
+the job will fail:
+
+```yaml
+job:
+ services:
+ - php:7
+ - node:latest
+ - golang:1.10
+ image: alpine:3.7
+ script:
+ - php -v
+ - node -v
+ - go version
+```
+
+If you need to have `php`, `node` and `go` available for your script, you should
+either:
+
+- choose an existing Docker image that contains all required tools, or
+- create your own Docker image, which will have all the required tools included
+ and use that in your job
+
### Accessing the services
Let's say that you need a Wordpress instance to test some API integration with
diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md
index f69729f602d..c1e258aedca 100644
--- a/doc/ci/examples/README.md
+++ b/doc/ci/examples/README.md
@@ -47,8 +47,11 @@ There's also a collection of repositories with [example projects](https://gitlab
## Static Application Security Testing (SAST)
-- **(Ultimate)** [Scan your code for vulnerabilities](https://docs.gitlab.com/ee/ci/examples/sast.html)
-- [Scan your Docker images for vulnerabilities](sast_docker.md)
+**(Ultimate)** [Scan your code for vulnerabilities](https://docs.gitlab.com/ee/ci/examples/sast.html)
+
+## Container Scanning
+
+[Scan your Docker images for vulnerabilities](container_scanning.md)
## Dynamic Application Security Testing (DAST)
diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md
index d7df53494ed..ec5e5afb8c6 100644
--- a/doc/ci/examples/code_climate.md
+++ b/doc/ci/examples/code_climate.md
@@ -16,18 +16,26 @@ codequality:
- docker:dind
script:
- docker pull codeclimate/codeclimate
- - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init
- - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true
+ - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/')
+ - docker run
+ --env SOURCE_CODE="$PWD" \
+ --volume "$PWD":/code \
+ --volume /var/run/docker.sock:/var/run/docker.sock \
+ "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code
artifacts:
paths: [codeclimate.json]
```
-This will create a `codequality` job in your CI pipeline and will allow you to
-download and analyze the report artifact in JSON format.
+The above example will create a `codequality` job in your CI/CD pipeline which
+will scan your source code for code quality issues. The report will be saved
+as an artifact that you can later download and analyze.
-For [GitLab Starter][ee] users, this information can be automatically
-extracted and shown right in the merge request widget. [Learn more on code quality
-diffs in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html).
+TIP: **Tip:**
+Starting with [GitLab Starter][ee] 9.3, this information will
+be automatically extracted and shown right in the merge request widget. To do
+so, the CI/CD job must be named `codequality` and the artifact path must be
+`codeclimate.json`.
+[Learn more on code quality diffs in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html).
[cli]: https://github.com/codeclimate/codeclimate
[dind]: ../docker/using_docker_build.md#use-docker-in-docker-executor
diff --git a/doc/ci/examples/container_scanning.md b/doc/ci/examples/container_scanning.md
new file mode 100644
index 00000000000..3437b63748a
--- /dev/null
+++ b/doc/ci/examples/container_scanning.md
@@ -0,0 +1,55 @@
+# Container Scanning with GitLab CI/CD
+
+You can check your Docker images (or more precisely the containers) for known
+vulnerabilities by using [Clair](https://github.com/coreos/clair) and
+[clair-scanner](https://github.com/arminc/clair-scanner), two open source tools
+for Vulnerability Static Analysis for containers.
+
+All you need is a GitLab Runner with the Docker executor (the shared Runners on
+GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`,
+called `sast:container`:
+
+```yaml
+sast:container:
+ image: docker:latest
+ variables:
+ DOCKER_DRIVER: overlay2
+ ## Define two new variables based on GitLab's CI/CD predefined variables
+ ## https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables
+ CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG
+ CI_APPLICATION_TAG: $CI_COMMIT_SHA
+ allow_failure: true
+ services:
+ - docker:dind
+ script:
+ - docker run -d --name db arminc/clair-db:latest
+ - docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1
+ - apk add -U wget ca-certificates
+ - docker pull ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG}
+ - wget https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64
+ - mv clair-scanner_linux_amd64 clair-scanner
+ - chmod +x clair-scanner
+ - touch clair-whitelist.yml
+ - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-sast-container-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true
+ artifacts:
+ paths: [gl-sast-container-report.json]
+```
+
+The above example will create a `sast:container` job in your CI/CD pipeline, pull
+the image from the [Container Registry](../../user/project/container_registry.md)
+(whose name is defined from the two `CI_APPLICATION_` variables) and scan it
+for possible vulnerabilities. The report will be saved as an artifact that you
+can later download and analyze.
+
+If you want to whitelist some specific vulnerabilities, you can do so by defining
+them in a [YAML file](https://github.com/arminc/clair-scanner/blob/master/README.md#example-whitelist-yaml-file),
+in our case its named `clair-whitelist.yml`.
+
+TIP: **Tip:**
+Starting with [GitLab Ultimate][ee] 10.4, this information will
+be automatically extracted and shown right in the merge request widget. To do
+so, the CI/CD job must be named `sast:container` and the artifact path must be
+`gl-sast-container-report.json`.
+[Learn more on container scanning results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html).
+
+[ee]: https://about.gitlab.com/products/
diff --git a/doc/ci/examples/sast_docker.md b/doc/ci/examples/sast_docker.md
index 57a9c4bcfc1..9f4a63e296d 100644
--- a/doc/ci/examples/sast_docker.md
+++ b/doc/ci/examples/sast_docker.md
@@ -1,55 +1 @@
-# Static Application Security Testing for Docker containers with GitLab CI/CD
-
-You can check your Docker images (or more precisely the containers) for known
-vulnerabilities by using [Clair](https://github.com/coreos/clair) and
-[clair-scanner](https://github.com/arminc/clair-scanner), two open source tools
-for Vulnerability Static Analysis for containers.
-
-All you need is a GitLab Runner with the Docker executor (the shared Runners on
-GitLab.com will work fine). You can then add a new job to `.gitlab-ci.yml`,
-called `sast:container`:
-
-```yaml
-sast:container:
- image: docker:latest
- variables:
- DOCKER_DRIVER: overlay2
- ## Define two new variables based on GitLab's CI/CD predefined variables
- ## https://docs.gitlab.com/ee/ci/variables/#predefined-variables-environment-variables
- CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG
- CI_APPLICATION_TAG: $CI_COMMIT_SHA
- allow_failure: true
- services:
- - docker:dind
- script:
- - docker run -d --name db arminc/clair-db:latest
- - docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.0.1
- - apk add -U wget ca-certificates
- - docker pull ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG}
- - wget https://github.com/arminc/clair-scanner/releases/download/v8/clair-scanner_linux_amd64
- - mv clair-scanner_linux_amd64 clair-scanner
- - chmod +x clair-scanner
- - touch clair-whitelist.yml
- - ./clair-scanner -c http://docker:6060 --ip $(hostname -i) -r gl-sast-container-report.json -l clair.log -w clair-whitelist.yml ${CI_APPLICATION_REPOSITORY}:${CI_APPLICATION_TAG} || true
- artifacts:
- paths: [gl-sast-container-report.json]
-```
-
-The above example will create a `sast:container` job in your CI/CD pipeline, pull
-the image from the [Container Registry](../../user/project/container_registry.md)
-(whose name is defined from the two `CI_APPLICATION_` variables) and scan it
-for possible vulnerabilities. The report will be saved as an artifact that you
-can later download and analyze.
-
-If you want to whitelist some specific vulnerabilities, you can do so by defining
-them in a [YAML file](https://github.com/arminc/clair-scanner/blob/master/README.md#example-whitelist-yaml-file),
-in our case its named `clair-whitelist.yml`.
-
-TIP: **Tip:**
-Starting with [GitLab Ultimate][ee] 10.4, this information will
-be automatically extracted and shown right in the merge request widget. To do
-so, the CI/CD job must be named `sast:container` and the artifact path must be
-`gl-sast-container-report.json`.
-[Learn more on application security testing results shown in merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/sast_docker.html).
-
-[ee]: https://about.gitlab.com/products/
+This document was moved to [another location](./container_scanning.md). \ No newline at end of file
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index ae200f9b6e2..accf6340398 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -966,7 +966,7 @@ tag including only the files that are untracked by Git:
```yaml
job:
artifacts:
- name: "${CI_JOB_NAME}_${CI_COMMIT_REF_NAME}"
+ name: "$CI_JOB_NAME-$CI_COMMIT_REF_NAME"
untracked: true
```
@@ -975,7 +975,7 @@ To create an archive with a name of the current [stage](#stages) and branch name
```yaml
job:
artifacts:
- name: "${CI_JOB_STAGE}_${CI_COMMIT_REF_NAME}"
+ name: "$CI_JOB_STAGE-$CI_COMMIT_REF_NAME"
untracked: true
```
@@ -987,7 +987,7 @@ If you use **Windows Batch** to run your shell scripts you need to replace
```yaml
job:
artifacts:
- name: "%CI_JOB_STAGE%_%CI_COMMIT_REF_NAME%"
+ name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%"
untracked: true
```
@@ -997,7 +997,7 @@ If you use **Windows PowerShell** to run your shell scripts you need to replace
```yaml
job:
artifacts:
- name: "$env:CI_JOB_STAGE_$env:CI_COMMIT_REF_NAME"
+ name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME"
untracked: true
```
View Lorry Controller Status