diff options
Diffstat (limited to 'doc/ci/docker/using_docker_images.md')
| -rw-r--r-- | doc/ci/docker/using_docker_images.md | 203 |
1 files changed, 203 insertions, 0 deletions
diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md new file mode 100644 index 00000000000..ef449cd45bc --- /dev/null +++ b/doc/ci/docker/using_docker_images.md @@ -0,0 +1,203 @@ +# Using Docker Images +GitLab CI can use [Docker Engine](https://www.docker.com/) to build projects. + +Docker is an open-source project that allows to use predefined images to run applications +in independent "containers" that are run within a single Linux instance. +[Docker Hub](https://registry.hub.docker.com/) have rich database of built images that can be used to build applications. + +Docker when used with GitLab CI runs each build in separate and isolated container using predefined image and always from scratch. +It makes it easier to have simple and reproducible build environment that can also be run on your workstation. +This allows you to test all commands from your shell, rather than having to test them on a CI server. + +### Register Docker runner +To use GitLab Runner with Docker you need to register new runner to use `docker` executor: + +```bash +gitlab-ci-multi-runner register \ + --url "https://ci.gitlab.com/" \ + --registration-token "PROJECT_REGISTRATION_TOKEN" \ + --description "docker-ruby-2.1" \ + --executor "docker" \ + --docker-image ruby:2.1 \ + --docker-postgres latest \ + --docker-mysql latest +``` + +**The registered runner will use `ruby:2.1` image and will run two services (`postgres:latest` and `mysql:latest`) that will be accessible for time of the build.** + +### What is image? +The image is the name of any repository that is present in local Docker Engine or any repository that can be found at [Docker Hub](https://registry.hub.docker.com/). +For more information about the image and Docker Hub please read the [Docker Fundamentals](https://docs.docker.com/introduction/understanding-docker/). + +### What is service? +Service is just another image that is run for time of your build and is linked to your build. This allows you to access the service image during build time. +The service image can run any application, but most common use case is to run some database container, ie.: `mysql`. +It's easier and faster to use existing image, run it as additional container than install `mysql` every time project is built. + +#### How is service linked to the build? +There's good document that describes how Docker linking works: [Linking containers together](https://docs.docker.com/userguide/dockerlinks/). +To summarize: if you add `mysql` as service to your application, the image will be used to create container that is linked to build container. +The service container for MySQL will be accessible under hostname `mysql`. +So, **to access your database service you have to connect to host: `mysql` instead of socket or `localhost`**. + +### How to use other images as services? +You are not limited to have only database services. +You can hand modify `config.toml` to add any image as service found at [Docker Hub](https://registry.hub.docker.com/). +Look for `[runners.docker]` section: +``` +[runners.docker] + image = "ruby:2.1" + services = ["mysql:latest", "postgres:latest"] +``` + +For example you need `wordpress` instance to test some API integration with `Wordpress`. +You can for example use this image: [tutum/wordpress](https://registry.hub.docker.com/u/tutum/wordpress/). +This is image that have fully preconfigured `wordpress` and have `MySQL` server built-in: +``` +[runners.docker] + image = "ruby:2.1" + services = ["mysql:latest", "postgres:latest", "tutum/wordpress:latest"] +``` + +Next time when you run your application the `tutum/wordpress` will be started +and you will have access to it from your build container under hostname: `tutum_wordpress`. + +Alias hostname for the service is made from the image name: +1. Everything after `:` is stripped, +2. '/' is replaced to `_`. + +### Configuring services +Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. + +GitLab Runner 0.5.0 and up passes all YAML-defined variables to created service containers. + +1. To configure database name for [postgres](https://registry.hub.docker.com/u/library/postgres/) service, +you need to set POSTGRES_DB. + + ```yaml + services: + - postgres + + variables: + POSTGRES_DB: gitlab + ``` + +1. To use [mysql](https://registry.hub.docker.com/u/library/mysql/) service with empty password for time of build, +you need to set MYSQL_ALLOW_EMPTY_PASSWORD. + + ```yaml + services: + - mysql + + variables: + MYSQL_ALLOW_EMPTY_PASSWORD: yes + ``` + +For other possible configuration variables check the +https://registry.hub.docker.com/u/library/mysql/ or https://registry.hub.docker.com/u/library/postgres/ +or README page for any other Docker image. + +**Note: All variables will passed to all service containers. It's not designed to distinguish which variable should go where.** + +### Overwrite image and services +It's possible to overwrite `docker-image` and specify services from `.gitlab-ci.yml`. +If you add to your YAML the `image` and the `services` these parameters +be used instead of the ones that were specified during runner's registration. +``` +image: ruby:2.2 +services: + - postgres:9.3 +before_install: + - bundle install + +test: + script: + - bundle exec rake spec +``` + +It's possible to define image and service per-job: +``` +before_install: + - bundle install + +test:2.1: + image: ruby:2.1 + services: + - postgres:9.3 + script: + - bundle exec rake spec + +test:2.2: + image: ruby:2.2 + services: + - postgres:9.4 + script: + - bundle exec rake spec +``` + +#### How to enable overwriting? +To enable overwriting you have to **enable it first** (it's disabled by default for security reasons). +You can do that by hand modifying runner configuration: `config.toml`. +Please go to section where is `[runners.docker]` definition for your runner. +Add `allowed_images` and `allowed_services` to specify what images are allowed to be picked from `.gitlab-ci.yml`: +``` +[runners.docker] + image = "ruby:2.1" + allowed_images = ["ruby:*", "python:*"] + allowed_services = ["mysql:*", "redis:*"] +``` +This enables you to use in your `.gitlab-ci.yml` any image that matches above wildcards. +You will be able to pick only `ruby` and `python` images. +The same rule can be applied to limit services. + +If you are courageous enough, you can make it fully open and accept everything: +``` +[runners.docker] + image = "ruby:2.1" + allowed_images = ["*", "*/*"] + allowed_services = ["*", "*/*"] +``` + +**It the feature is not enabled, or image isn't allowed the error message will be put into the build log.** + +### How Docker integration works +1. Create any service container: `mysql`, `postgresql`, `mongodb`, `redis`. +1. Create cache container to store all volumes as defined in `config.toml` and `Dockerfile` of build image (`ruby:2.1` as in above example). +1. Create build container and link any service container to build container. +1. Start build container and send build script to the container. +1. Run build script. +1. Checkout code in: `/builds/group-name/project-name/`. +1. Run any step defined in `.gitlab-ci.yml`. +1. Check exit status of build script. +1. Remove build container and all created service containers. + +### How to debug a build locally +1. Create a file with build script: +```bash +$ cat <<EOF > build_script +git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner +cd /builds/gitlab-org/gitlab-ci-multi-runner +make <- or any other build step +EOF +``` + +1. Create service containers: +``` +$ docker run -d -n service-mysql mysql:latest +$ docker run -d -n service-postgres postgres:latest +``` +This will create two service containers (MySQL and PostgreSQL). + +1. Create a build container and execute script in its context: +``` +$ cat build_script | docker run -n build -i -l mysql:service-mysql -l postgres:service-postgres ruby:2.1 /bin/bash +``` +This will create build container that has two service containers linked. +The build_script is piped using STDIN to bash interpreter which executes the build script in container. + +1. At the end remove all containers: +``` +docker rm -f -v build service-mysql service-postgres +``` +This will forcefully (the `-f` switch) remove build container and service containers +and all volumes (the `-v` switch) that were created with the container creation. |