summaryrefslogtreecommitdiff
path: root/doc/ci/docker
diff options
context:
space:
mode:
authorEvan Read <eread@gitlab.com>2019-06-07 08:00:41 +0000
committerAchilleas Pipinellis <axil@gitlab.com>2019-06-07 08:00:41 +0000
commita1816ff0e60ad7546c9f4e28e86e502c951bb0ca (patch)
treebb010c77fd33f22a744aa08f56d0c5e8de882416 /doc/ci/docker
parenta80c6a68b542a33808315e7c2871cd48c1245391 (diff)
downloadgitlab-ce-a1816ff0e60ad7546c9f4e28e86e502c951bb0ca.tar.gz
SSoT audit fixes
Implements part of the single source of truth audit fixes for the CI section.
Diffstat (limited to 'doc/ci/docker')
-rw-r--r--doc/ci/docker/README.md4
-rw-r--r--doc/ci/docker/using_docker_build.md42
-rw-r--r--doc/ci/docker/using_docker_images.md63
-rw-r--r--doc/ci/docker/using_kaniko.md42
4 files changed, 98 insertions, 53 deletions
diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md
index 446f5b54f0c..f76471b50f2 100644
--- a/doc/ci/docker/README.md
+++ b/doc/ci/docker/README.md
@@ -1,9 +1,13 @@
---
comments: false
+type: index
---
# Docker integration
+GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable
+integration between the two.
+
The following documentation is available for using GitLab CI/CD with Docker:
- [Using Docker images](using_docker_images.md).
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md
index 6920ce17b46..d8068bbb7f0 100644
--- a/doc/ci/docker/using_docker_build.md
+++ b/doc/ci/docker/using_docker_build.md
@@ -1,3 +1,7 @@
+---
+type: concepts, howto
+---
+
# Building Docker images with GitLab CI/CD
GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects.
@@ -5,10 +9,10 @@ GitLab CI/CD allows you to use Docker Engine to build and test docker-based proj
One of the new trends in Continuous Integration/Deployment is to:
-1. Create an application image
-1. Run tests against the created image
-1. Push image to a remote registry
-1. Deploy to a server from the pushed image
+1. Create an application image.
+1. Run tests against the created image.
+1. Push image to a remote registry.
+1. Deploy to a server from the pushed image.
It's also useful when your application already has the `Dockerfile` that can be
used to create and test an image:
@@ -113,7 +117,7 @@ In order to do that, follow the steps:
The above command will create a `config.toml` entry similar to this:
- ```
+ ```toml
[[runners]]
url = "https://gitlab.com/"
token = TOKEN
@@ -227,7 +231,7 @@ In order to do that, follow the steps:
The above command will create a `config.toml` entry similar to this:
- ```
+ ```toml
[[runners]]
url = "https://gitlab.com/"
token = REGISTRATION_TOKEN
@@ -270,9 +274,9 @@ aware of the following implications:
create containers with specific names, they may conflict with each other.
- Sharing files and directories from the source repo into containers may not
work as expected since volume mounting is done in the context of the host
- machine, not the build container, e.g.:
+ machine, not the build container. For example:
- ```
+ ```sh
docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests
```
@@ -337,7 +341,7 @@ NOTE: **Note:**
The shared Runners on GitLab.com use the `overlay2` driver by default.
By default, when using `docker:dind`, Docker uses the `vfs` storage driver which
-copies the filesystem on every run. This is a very disk-intensive operation
+copies the filesystem on every run. This is a disk-intensive operation
which can be avoided if a different driver is used, for example `overlay2`.
### Requirements
@@ -345,13 +349,13 @@ which can be avoided if a different driver is used, for example `overlay2`.
1. Make sure a recent kernel is used, preferably `>= 4.2`.
1. Check whether the `overlay` module is loaded:
- ```
+ ```sh
sudo lsmod | grep overlay
```
If you see no result, then it isn't loaded. To load it use:
- ```
+ ```sh
sudo modprobe overlay
```
@@ -359,7 +363,7 @@ which can be avoided if a different driver is used, for example `overlay2`.
On Ubuntu systems, this is done by editing `/etc/modules`. Just add the
following line into it:
- ```
+ ```text
overlay
```
@@ -367,7 +371,7 @@ which can be avoided if a different driver is used, for example `overlay2`.
You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`:
-```
+```yaml
variables:
DOCKER_DRIVER: overlay2
```
@@ -571,3 +575,15 @@ deploy:
[docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
[2fa]: ../../user/profile/account/two_factor_authentication.md
[pat]: ../../user/profile/personal_access_tokens.md
+
+<!-- ## Troubleshooting
+
+Include any troubleshooting steps that you can foresee. If you know beforehand what issues
+one might have when setting this up, or when something is changed, or on upgrading, it's
+important to describe those, too. Think of things that may go wrong and include them here.
+This is important to minimize requests for support, and to avoid doc comments with
+questions that you know someone might ask.
+
+Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+If you have none to add when creating a doc, leave this section in place
+but commented out to help encourage others to add to it in the future. -->
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md
index 13c26bc5f47..d09efce7cc1 100644
--- a/doc/ci/docker/using_docker_images.md
+++ b/doc/ci/docker/using_docker_images.md
@@ -1,3 +1,7 @@
+---
+type: concepts, howto
+---
+
# Using Docker images
GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use
@@ -45,10 +49,10 @@ The `image` keyword is the name of the Docker image the Docker executor
will run to perform the CI tasks.
By default, the executor will only pull images from [Docker Hub][hub],
-but this can be configured in the `gitlab-runner/config.toml` by setting
+however this can be configured in the `gitlab-runner/config.toml` by setting
the [Docker pull policy][] to allow using local images.
-For more information about images and Docker Hub please read
+For more information about images and Docker Hub, please read
the [Docker Fundamentals][] documentation.
## What is a service
@@ -95,8 +99,8 @@ required for the CI/CD job to proceed and is accessed by network.
To make sure this works, the Runner:
-1. checks which ports are exposed from the container by default
-1. starts a special container that waits for these ports to be accessible
+1. Checks which ports are exposed from the container by default.
+1. Starts a special container that waits for these ports to be accessible.
When the second stage of the check fails, either because there is no opened port in the
service, or the service was not started properly before the timeout and the port is not
@@ -106,7 +110,7 @@ In most cases it will affect the job, but there may be situations when the job
will still succeed even if that warning was printed. For example:
- The service was started a little after the warning was raised, and the job is
- not using the linked service from the very beginning. In that case, when the
+ not using the linked service from the beginning. In that case, when the
job needed to access the service, it may have been already there waiting for
connections.
- The service container is not providing any networking service, but it's doing
@@ -143,9 +147,9 @@ job:
If you need to have `php`, `node` and `go` available for your script, you should
either:
-- choose an existing Docker image that contains all required tools, or
-- create your own Docker image, which will have all the required tools included
- and use that in your job
+- Choose an existing Docker image that contains all required tools.
+- Create your own Docker image, which will have all the required tools included
+ and use that in your job.
### Accessing the services
@@ -167,18 +171,18 @@ access to it from your build container under two hostnames to choose from:
- `tutum-wordpress`
- `tutum__wordpress`
->**Note:**
+NOTE: **Note:**
Hostnames with underscores are not RFC valid and may cause problems in 3rd party
applications.
The default aliases for the service's hostname are created from its image name
following these rules:
-- Everything after the colon (`:`) is stripped
+- Everything after the colon (`:`) is stripped.
- Slash (`/`) is replaced with double underscores (`__`) and the primary alias
- is created
+ is created.
- Slash (`/`) is replaced with a single dash (`-`) and the secondary alias is
- created (requires GitLab Runner v1.1.0 or higher)
+ created (requires GitLab Runner v1.1.0 or higher).
To override the default behavior, you can
[specify a service alias](#available-settings-for-services).
@@ -333,7 +337,7 @@ services:
```
The Runner will still start two containers using the `mysql:latest` image,
-but now each of them will also be accessible with the alias configured
+however now each of them will also be accessible with the alias configured
in `.gitlab-ci.yml` file.
### Setting a command for the service
@@ -408,8 +412,6 @@ you should check which one your Runner is using. Specifically:
The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`][entrypoint].
-----
-
Let's assume you have a `super/sql:experimental` image with some SQL database
inside it and you would like to use it as a base image for your job because you
want to execute some tests with this database binary. Let's also assume that
@@ -443,7 +445,7 @@ image:
Look for the `[runners.docker]` section:
-```
+```toml
[runners.docker]
image = "ruby:2.1"
services = ["mysql:latest", "postgres:latest"]
@@ -469,11 +471,11 @@ image which is private and requires you to login into a private container regist
Let's also assume that these are the login credentials:
-| Key | Value |
-|----------|---------------------------|
-| registry | registry.example.com:5000 |
-| username | my_username |
-| password | my_password |
+| Key | Value |
+|:---------|:----------------------------|
+| registry | `registry.example.com:5000` |
+| username | `my_username` |
+| password | `my_password` |
To configure access for `registry.example.com:5000`, follow these steps:
@@ -534,7 +536,8 @@ To configure access for `registry.example.com:5000`, follow these steps:
You can add configuration for as many registries as you want, adding more
registries to the `"auths"` hash as described above.
-NOTE: **Note:** The full `hostname:port` combination is required everywhere
+NOTE: **Note:**
+The full `hostname:port` combination is required everywhere
for the Runner to match the `DOCKER_AUTH_CONFIG`. For example, if
`registry.example.com:5000/namespace/image:tag` is specified in `.gitlab-ci.yml`,
then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`.
@@ -551,8 +554,9 @@ service containers.
For all possible configuration variables check the documentation of each image
provided in their corresponding Docker hub page.
-*Note: All variables will be passed to all services containers. It's not
-designed to distinguish which variable should go where.*
+NOTE: **Note:**
+All variables will be passed to all services containers. It's not
+designed to distinguish which variable should go where.
### PostgreSQL service example
@@ -582,8 +586,9 @@ time.
## How to debug a job locally
-*Note: The following commands are run without root privileges. You should be
-able to run Docker with your regular user account.*
+NOTE: **Note:**
+The following commands are run without root privileges. You should be
+able to run Docker with your regular user account.
First start with creating a file named `build_script`:
@@ -602,7 +607,7 @@ is specific to your project.
Then create some service containers:
-```
+```sh
docker run -d --name service-mysql mysql:latest
docker run -d --name service-postgres postgres:latest
```
@@ -614,7 +619,7 @@ respectively. They will both run in the background (`-d`).
Finally, create a build container by executing the `build_script` file we
created earlier:
-```
+```sh
docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script
```
@@ -626,7 +631,7 @@ piped using STDIN to the bash interpreter which in turn executes the
When you finish testing and no longer need the containers, you can remove them
with:
-```
+```sh
docker rm -f -v build service-mysql service-postgres
```
diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md
index f354cdb398e..50f1ac3d54a 100644
--- a/doc/ci/docker/using_kaniko.md
+++ b/doc/ci/docker/using_kaniko.md
@@ -1,3 +1,7 @@
+---
+type: howto
+---
+
# Building images with kaniko and GitLab CI/CD
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45512) in GitLab 11.2.
@@ -9,17 +13,18 @@ container images from a Dockerfile, inside a container or Kubernetes cluster.
kaniko solves two problems with using the
[docker-in-docker build](using_docker_build.md#use-docker-in-docker-executor) method:
-1. Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
- in order to function, which is a significant security concern.
-1. Docker-in-docker generally incurs a performance penalty and can be quite slow.
+- Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
+ in order to function, which is a significant security concern.
+- Docker-in-docker generally incurs a performance penalty and can be quite slow.
## Requirements
In order to utilize kaniko with GitLab, a [GitLab Runner](https://docs.gitlab.com/runner/)
-using either the [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html),
-[Docker](https://docs.gitlab.com/runner/executors/docker.html), or
-[Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html)
-executors is required.
+using one of the following executors is required:
+
+- [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html).
+- [Docker](https://docs.gitlab.com/runner/executors/docker.html).
+- [Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html).
## Building a Docker image with kaniko
@@ -34,14 +39,17 @@ few important details:
- A Docker `config.json` file needs to be created with the authentication
information for the desired container registry.
----
+In the following example, kaniko is used to:
+
+1. Build a Docker image.
+1. Then push it to [GitLab Container Registry](../../user/project/container_registry.md).
-In the following example, kaniko is used to build a Docker image and then push
-it to [GitLab Container Registry](../../user/project/container_registry.md).
The job will run only when a tag is pushed. A `config.json` file is created under
`/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the
[environment variables](../variables/README.md#predefined-environment-variables)
-GitLab CI/CD provides. In the last step, kaniko uses the `Dockerfile` under the
+GitLab CI/CD provides.
+
+In the last step, kaniko uses the `Dockerfile` under the
root directory of the project, builds the Docker image and pushes it to the
project's Container Registry while tagging it with the Git tag:
@@ -80,3 +88,15 @@ store:
...
-----END CERTIFICATE-----" >> /kaniko/ssl/certs/ca-certificates.crt
```
+
+<!-- ## Troubleshooting
+
+Include any troubleshooting steps that you can foresee. If you know beforehand what issues
+one might have when setting this up, or when something is changed, or on upgrading, it's
+important to describe those, too. Think of things that may go wrong and include them here.
+This is important to minimize requests for support, and to avoid doc comments with
+questions that you know someone might ask.
+
+Each scenario can be a third-level heading, e.g. `### Getting error message X`.
+If you have none to add when creating a doc, leave this section in place
+but commented out to help encourage others to add to it in the future. -->