summaryrefslogtreecommitdiff
path: root/doc/ci/docker
diff options
context:
space:
mode:
Diffstat (limited to 'doc/ci/docker')
-rw-r--r--doc/ci/docker/README.md3
-rw-r--r--doc/ci/docker/using_docker_build.md73
-rw-r--r--doc/ci/docker/using_docker_images.md28
-rw-r--r--doc/ci/docker/using_kaniko.md11
4 files changed, 71 insertions, 44 deletions
diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md
index f76471b50f2..4e7d9015e85 100644
--- a/doc/ci/docker/README.md
+++ b/doc/ci/docker/README.md
@@ -1,4 +1,7 @@
---
+stage: Verify
+group: Continuous Integration
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
comments: false
type: index
---
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md
index f992af6c8a5..65b9c03186b 100644
--- a/doc/ci/docker/using_docker_build.md
+++ b/doc/ci/docker/using_docker_build.md
@@ -1,10 +1,13 @@
---
+stage: Verify
+group: Continuous Integration
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: concepts, howto
---
# Building Docker images with GitLab CI/CD
-GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects.
+GitLab CI/CD allows you to use Docker Engine to build and test Docker-based projects.
One of the new trends in Continuous Integration/Deployment is to:
@@ -91,15 +94,15 @@ NOTE: **Note:**
By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions.
For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful).
-### Use docker-in-docker workflow with Docker executor
+### Use Docker-in-Docker workflow with Docker executor
-The second approach is to use the special docker-in-docker (dind)
+The second approach is to use the special Docker-in-Docker (dind)
[Docker image](https://hub.docker.com/_/docker/) with all tools installed
(`docker`) and run the job script in context of that
image in privileged mode.
NOTE: **Note:**
-`docker-compose` is not part of docker-in-docker (dind). To use `docker-compose` in your
+`docker-compose` is not part of Docker-in-Docker (dind). To use `docker-compose` in your
CI builds, follow the `docker-compose`
[installation instructions](https://docs.docker.com/compose/install/).
@@ -113,19 +116,19 @@ out the official Docker documentation on
Docker-in-Docker works well, and is the recommended configuration, but it is
not without its own challenges:
-- When using docker-in-docker, each job is in a clean environment without the past
+- When using Docker-in-Docker, each job is in a clean environment without the past
history. Concurrent jobs work fine because every build gets its own
instance of Docker engine so they won't conflict with each other. But this
also means that jobs can be slower because there's no caching of layers.
- By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is
the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver)
for details.
-- Since the `docker:19.03.8-dind` container and the Runner container don't share their
+- Since the `docker:19.03.11-dind` container and the Runner container don't share their
root filesystem, the job's working directory can be used as a mount point for
child containers. For example, if you have files you want to share with a
child container, you may create a subdirectory under `/builds/$CI_PROJECT_PATH`
and use it as your mount point (for a more thorough explanation, check [issue
- #41227](https://gitlab.com/gitlab-org/gitlab-foss/issues/41227)):
+ #41227](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41227)):
```yaml
variables:
@@ -139,7 +142,7 @@ not without its own challenges:
An example project using this approach can be found here: <https://gitlab.com/gitlab-examples/docker>.
In the examples below, we are using Docker images tags to specify a
-specific version, such as `docker:19.03.8`. If tags like `docker:stable`
+specific version, such as `docker:19.03.11`. If tags like `docker:stable`
are used, you have no control over what version is going to be used and this
can lead to unpredictable behavior, especially when new versions are
released.
@@ -151,12 +154,12 @@ Requires GitLab Runner 11.11 or later, but is not supported if GitLab
Runner is installed using the [Helm
chart](https://docs.gitlab.com/runner/install/kubernetes.html). See the
[related
-issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/issues/83) for
+issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for
details.
The Docker daemon supports connection over TLS and it's done by default
-for Docker 19.03.8 or higher. This is the **suggested** way to use the
-docker-in-docker service and
+for Docker 19.03.11 or higher. This is the **suggested** way to use the
+Docker-in-Docker service and
[GitLab.com Shared Runners](../../user/gitlab_com/index.md#shared-runners)
support this.
@@ -171,19 +174,19 @@ support this.
--registration-token REGISTRATION_TOKEN \
--executor docker \
--description "My Docker Runner" \
- --docker-image "docker:19.03.8" \
+ --docker-image "docker:19.03.11" \
--docker-privileged \
--docker-volumes "/certs/client"
```
The above command will register a new Runner to use the special
- `docker:19.03.8` image, which is provided by Docker. **Notice that it's
+ `docker:19.03.11` image, which is provided by Docker. **Notice that it's
using the `privileged` mode to start the build and service
- containers.** If you want to use [docker-in-docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) mode, you always
+ containers.** If you want to use [Docker-in-Docker](https://www.docker.com/blog/docker-can-now-run-within-docker/) mode, you always
have to use `privileged = true` in your Docker containers.
This will also mount `/certs/client` for the service and build
- container, which is needed for the docker client to use the
+ container, which is needed for the Docker client to use the
certificates inside of that directory. For more information how
Docker with TLS works check <https://hub.docker.com/_/docker/#tls>.
@@ -196,7 +199,7 @@ support this.
executor = "docker"
[runners.docker]
tls_verify = false
- image = "docker:19.03.8"
+ image = "docker:19.03.11"
privileged = true
disable_cache = false
volumes = ["/certs/client", "/cache"]
@@ -206,10 +209,10 @@ support this.
```
1. You can now use `docker` in the build script (note the inclusion of the
- `docker:19.03.8-dind` service):
+ `docker:19.03.11-dind` service):
```yaml
- image: docker:19.03.8
+ image: docker:19.03.11
variables:
# When using dind service, we need to instruct docker, to talk with
@@ -234,7 +237,7 @@ support this.
DOCKER_TLS_CERTDIR: "/certs"
services:
- - docker:19.03.8-dind
+ - docker:19.03.11-dind
before_script:
- docker info
@@ -261,7 +264,7 @@ Assuming that the Runner `config.toml` is similar to:
executor = "docker"
[runners.docker]
tls_verify = false
- image = "docker:19.03.8"
+ image = "docker:19.03.11"
privileged = true
disable_cache = false
volumes = ["/cache"]
@@ -271,10 +274,10 @@ Assuming that the Runner `config.toml` is similar to:
```
You can now use `docker` in the build script (note the inclusion of the
-`docker:19.03.8-dind` service):
+`docker:19.03.11-dind` service):
```yaml
-image: docker:19.03.8
+image: docker:19.03.11
variables:
# When using dind service we need to instruct docker, to talk with the
@@ -295,7 +298,7 @@ variables:
DOCKER_TLS_CERTDIR: ""
services:
- - docker:19.03.8-dind
+ - docker:19.03.11-dind
before_script:
- docker info
@@ -315,7 +318,7 @@ container so that Docker is available in the context of that image.
NOTE: **Note:**
If you bind the Docker socket [when using GitLab Runner 11.11 or
newer](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261),
-you can no longer use `docker:19.03.8-dind` as a service because volume bindings
+you can no longer use `docker:19.03.11-dind` as a service because volume bindings
are done to the services as well, making these incompatible.
In order to do that, follow the steps:
@@ -330,12 +333,12 @@ In order to do that, follow the steps:
--registration-token REGISTRATION_TOKEN \
--executor docker \
--description "My Docker Runner" \
- --docker-image "docker:19.03.8" \
+ --docker-image "docker:19.03.11" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock
```
The above command will register a new Runner to use the special
- `docker:19.03.8` image which is provided by Docker. **Notice that it's using
+ `docker:19.03.11` image which is provided by Docker. **Notice that it's using
the Docker daemon of the Runner itself, and any containers spawned by Docker
commands will be siblings of the Runner rather than children of the Runner.**
This may have complications and limitations that are unsuitable for your workflow.
@@ -349,7 +352,7 @@ In order to do that, follow the steps:
executor = "docker"
[runners.docker]
tls_verify = false
- image = "docker:19.03.8"
+ image = "docker:19.03.11"
privileged = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
@@ -358,11 +361,11 @@ In order to do that, follow the steps:
```
1. You can now use `docker` in the build script (note that you don't need to
- include the `docker:19.03.8-dind` service as when using the Docker in Docker
+ include the `docker:19.03.11-dind` service as when using the Docker in Docker
executor):
```yaml
- image: docker:19.03.8
+ image: docker:19.03.11
before_script:
- docker info
@@ -377,7 +380,7 @@ In order to do that, follow the steps:
While the above method avoids using Docker in privileged mode, you should be
aware of the following implications:
-- By sharing the docker daemon, you are effectively disabling all
+- By sharing the Docker daemon, you are effectively disabling all
the security mechanisms of containers and exposing your host to privilege
escalation which can lead to container breakout. For example, if a project
ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner
@@ -392,9 +395,9 @@ aware of the following implications:
docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests
```
-## Making docker-in-docker builds faster with Docker layer caching
+## Making Docker-in-Docker builds faster with Docker layer caching
-When using docker-in-docker, Docker will download all layers of your image every
+When using Docker-in-Docker, Docker will download all layers of your image every
time you create a build. Recent versions of Docker (Docker 1.13 and above) can
use a pre-existing image as a cache during the `docker build` step, considerably
speeding up the build process.
@@ -416,10 +419,10 @@ any image that's used with the `--cache-from` argument must first be pulled
Here's a `.gitlab-ci.yml` file showing how Docker caching can be used:
```yaml
-image: docker:19.03.8
+image: docker:19.03.11
services:
- - docker:19.03.8-dind
+ - docker:19.03.11-dind
variables:
# Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled
@@ -514,7 +517,7 @@ Once you've built a Docker image, you can push it up to the built-in
## Troubleshooting
-### docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
+### `docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?`
This is a common error when you are using
[Docker in Docker](#use-docker-in-docker-workflow-with-docker-executor)
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md
index 51139da2d16..2448bb536ab 100644
--- a/doc/ci/docker/using_docker_images.md
+++ b/doc/ci/docker/using_docker_images.md
@@ -1,4 +1,7 @@
---
+stage: Verify
+group: Runner
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: concepts, howto
---
@@ -364,7 +367,7 @@ For example, the following two definitions are equal:
| `alias` | no | 9.4 |Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. |
NOTE: **Note:**
-Alias support for the Kubernetes executor was [introduced](https://gitlab.com/gitlab-org/gitlab-runner/issues/2229) in GitLab Runner 12.8, and is only available for Kubernetes version 1.7 or later.
+Alias support for the Kubernetes executor was [introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2229) in GitLab Runner 12.8, and is only available for Kubernetes version 1.7 or later.
### Starting multiple services from the same image
@@ -543,7 +546,7 @@ runtime.
of credentials on runner's host. We recommend to upgrade your Runner to
at least version **1.8** if you want to use private registries.
- Not available for [Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html),
- follow <https://gitlab.com/gitlab-org/gitlab-runner/issues/2673> for
+ follow <https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2673> for
details.
### Using statically-defined credentials
@@ -591,7 +594,7 @@ There are two ways to determine the value of `DOCKER_AUTH_CONFIG`:
```
- **Second way -** In some setups, it's possible that Docker client
- will use the available system keystore to store the result of `docker
+ will use the available system key store to store the result of `docker
login`. In that case, it's impossible to read `~/.docker/config.json`,
so you will need to prepare the required base64-encoded version of
`${username}:${password}` and create the Docker configuration JSON manually.
@@ -709,7 +712,7 @@ To configure credentials store, follow these steps:
```
- Or, if you are running self-managed Runners, add the above JSON to
- `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this config file
+ `${GITLAB_RUNNER_HOME}/.docker/config.json`. GitLab Runner will read this configuration file
and will use the needed helper for this specific repository.
NOTE: **Note:** `credsStore` is used to access ALL the registries.
@@ -727,6 +730,9 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`.
+1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials).
+ Make sure that GitLab Runner can access the credentials.
+
1. Make GitLab Runner use it. There are two ways to accomplish this. Either:
- Create a [variable](../variables/README.md#gitlab-cicd-environment-variables)
@@ -741,9 +747,21 @@ To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow th
}
```
+ This configures Docker to use the credential helper for a specific registry.
+
+ or
+
+ ```json
+ {
+ "credsStore": "ecr-login"
+ }
+ ```
+
+ This configures Docker to use the credential helper for all Amazon ECR registries.
+
- Or, if you are running self-managed Runners,
add the above JSON to `${GITLAB_RUNNER_HOME}/.docker/config.json`.
- GitLab Runner will read this config file and will use the needed helper for this
+ GitLab Runner will read this configuration file and will use the needed helper for this
specific repository.
1. You can now use any private image from `aws_account_id.dkr.ecr.region.amazonaws.com` defined in
diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md
index 587f1f91f72..d53430400ec 100644
--- a/doc/ci/docker/using_kaniko.md
+++ b/doc/ci/docker/using_kaniko.md
@@ -1,21 +1,24 @@
---
+stage: Verify
+group: Continuous Integration
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: howto
---
# Building images with kaniko and GitLab CI/CD
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/45512) in GitLab 11.2. Requires GitLab Runner 11.2 and above.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45512) in GitLab 11.2. Requires GitLab Runner 11.2 and above.
[kaniko](https://github.com/GoogleContainerTools/kaniko) is a tool to build
container images from a Dockerfile, inside a container or Kubernetes cluster.
kaniko solves two problems with using the
-[docker-in-docker
+[Docker-in-Docker
build](using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor) method:
-- Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
+- Docker-in-Docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
in order to function, which is a significant security concern.
-- Docker-in-docker generally incurs a performance penalty and can be quite slow.
+- Docker-in-Docker generally incurs a performance penalty and can be quite slow.
## Requirements
View Lorry Controller Status