diff options
Diffstat (limited to 'doc/ci')
46 files changed, 1283 insertions, 143 deletions
diff --git a/doc/ci/README.md b/doc/ci/README.md index 6b90940c047..dd14698e9cd 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,6 +1,6 @@ -## GitLab CI Documentation +# GitLab CI Documentation -### CI User documentation +## CI User documentation - [Get started with GitLab CI](quick_start/README.md) - [CI examples for various languages](examples/README.md) @@ -11,7 +11,8 @@ - [Configure a Runner, the application that runs your builds](runners/README.md) - [Use Docker images with GitLab Runner](docker/using_docker_images.md) - [Use CI to build Docker images](docker/using_docker_build.md) -- [Use variables in your `.gitlab-ci.yml`](variables/README.md) +- [CI Variables](variables/README.md) - Learn how to use variables defined in + your `.gitlab-ci.yml` or secured ones defined in your project's settings - [Use SSH keys in your build environment](ssh_keys/README.md) - [Trigger builds through the API](triggers/README.md) - [Build artifacts](../user/project/builds/artifacts.md) @@ -20,4 +21,12 @@ - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) - [CI/CD pipelines settings](../user/project/pipelines/settings.md) -- [**New CI build permissions model**](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. +- [Review Apps](review_apps/index.md) +- [Git submodules](git_submodules.md) Using Git submodules in your CI jobs +- [Auto deploy](autodeploy/index.md) + +## Breaking changes + +- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md) + Read about what changed in GitLab 8.12 and how that affects your builds. + There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/autodeploy/img/auto_deploy_button.png b/doc/ci/autodeploy/img/auto_deploy_button.png Binary files differnew file mode 100644 index 00000000000..423e76a6cda --- /dev/null +++ b/doc/ci/autodeploy/img/auto_deploy_button.png diff --git a/doc/ci/autodeploy/img/auto_deploy_dropdown.png b/doc/ci/autodeploy/img/auto_deploy_dropdown.png Binary files differnew file mode 100644 index 00000000000..957870ec8c7 --- /dev/null +++ b/doc/ci/autodeploy/img/auto_deploy_dropdown.png diff --git a/doc/ci/autodeploy/index.md b/doc/ci/autodeploy/index.md new file mode 100644 index 00000000000..630207ffa09 --- /dev/null +++ b/doc/ci/autodeploy/index.md @@ -0,0 +1,40 @@ +# Auto deploy + +> [Introduced][mr-8135] in GitLab 8.15. + +Auto deploy is an easy way to configure GitLab CI for the deployment of your +application. GitLab Community maintains a list of `.gitlab-ci.yml` +templates for various infrastructure providers and deployment scripts +powering them. These scripts are responsible for packaging your application, +setting up the infrastructure and spinning up necessary services (for +example a database). + +You can use [project services][project-services] to store credentials to +your infrastructure provider and they will be available during the +deployment. + +## Supported templates + +The list of supported auto deploy templates is available [here][auto-deploy-templates]. + +## Configuration + +1. Enable a deployment [project service][project-services] to store your +credentials. For example, if you want to deploy to OpenShift you have to +enable [Kubernetes service][kubernetes-service]. +1. Configure GitLab Runner to use Docker or Kubernetes executor with +[privileged mode enabled][docker-in-docker]. +1. Navigate to the "Project" tab and click "Set up auto deploy" button. +  +1. Select a template. +  +1. Commit your changes and create a merge request. +1. Test your deployment configuration using a [Review App][review-app] that was +created automatically for you. + +[mr-8135]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8135 +[project-services]: ../../project_services/project_services.md +[auto-deploy-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml/tree/master/autodeploy +[kubernetes-service]: ../../project_services/kubernetes.md +[docker-in-docker]: ../docker/using_docker_build.md#use-docker-in-docker-executor +[review-app]: ../review_apps/index.md diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 89088cf9b83..28141cced3b 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -270,12 +270,16 @@ which can be avoided if a different driver is used, for example `overlay`. ## Using the GitLab Container Registry -> **Note:** -This feature requires GitLab 8.8 and GitLab Runner 1.2. - -Once you've built a Docker image, you can push it up to the built-in [GitLab Container Registry](../../user/project/container_registry.md). For example, if you're using -docker-in-docker on your runners, this is how your `.gitlab-ci.yml` could look: +> **Notes:** +- This feature requires GitLab 8.8 and GitLab Runner 1.2. +- Starting from GitLab 8.12, if you have 2FA enabled in your account, you need + to pass a personal access token instead of your password in order to login to + GitLab's Container Registry. +Once you've built a Docker image, you can push it up to the built-in +[GitLab Container Registry](../../user/project/container_registry.md). For example, +if you're using docker-in-docker on your runners, this is how your `.gitlab-ci.yml` +could look like: ```yaml build: @@ -354,10 +358,20 @@ deploy: ``` Some things you should be aware of when using the Container Registry: -* You must log in to the container registry before running commands. Putting this in `before_script` will run it before each build job. -* Using `docker build --pull` makes sure that Docker fetches any changes to base images before building just in case your cache is stale. It takes slightly longer, but means you don’t get stuck without security patches to base images. -* Doing an explicit `docker pull` before each `docker run` makes sure to fetch the latest image that was just built. This is especially important if you are using multiple runners that cache images locally. Using the git SHA in your image tag makes this less necessary since each build will be unique and you shouldn't ever have a stale image, but it's still possible if you re-build a given commit after a dependency has changed. -* You don't want to build directly to `latest` in case there are multiple builds happening simultaneously. + +- You must log in to the container registry before running commands. Putting + this in `before_script` will run it before each build job. +- Using `docker build --pull` makes sure that Docker fetches any changes to base + images before building just in case your cache is stale. It takes slightly + longer, but means you don’t get stuck without security patches to base images. +- Doing an explicit `docker pull` before each `docker run` makes sure to fetch + the latest image that was just built. This is especially important if you are + using multiple runners that cache images locally. Using the git SHA in your + image tag makes this less necessary since each build will be unique and you + shouldn't ever have a stale image, but it's still possible if you re-build a + given commit after a dependency has changed. +- You don't want to build directly to `latest` in case there are multiple builds + happening simultaneously. [docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/ [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities diff --git a/doc/ci/environments.md b/doc/ci/environments.md index e070302fb82..98cd29c9567 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,69 +3,578 @@ >**Note:** Introduced in GitLab 8.9. -## Environments +During the development of software, there can be many stages until it's ready +for public consumption. You sure want to first test your code and then deploy it +in a testing or staging environment before you release it to the public. That +way you can prevent bugs not only in your software, but in the deployment +process as well. -Environments are places where code gets deployed, such as staging or production. -CI/CD [Pipelines] usually have one or more [jobs] that deploy to an environment. -Defining environments in a project's `.gitlab-ci.yml` lets developers track -[deployments] to these environments. +GitLab CI is capable of not only testing or building your projects, but also +deploying them in your infrastructure, with the added benefit of giving you a +way to track your deployments. In other words, you can always know what is +currently being deployed or has been deployed on your servers. -## Deployments +## Overview -Deployments are created when [jobs] deploy versions of code to [environments]. +With environments, you can control the Continuous Deployment of your software +all within GitLab. All you need to do is define them in your project's +[`.gitlab-ci.yml`][yaml] as we will explore below. GitLab provides a full +history of your deployments per every environment. -### Checkout deployments locally +Environments are like tags for your CI jobs, describing where code gets deployed. +Deployments are created when [jobs] deploy versions of code to environments, +so every environment can have one or more deployments. GitLab keeps track of +your deployments, so you always know what is currently being deployed on your +servers. If you have a deployment service such as [Kubernetes][kubernetes-service] +enabled for your project, you can use it to assist with your deployments, and +can even access a web terminal for your environment from within GitLab! -Since 8.13, a reference in the git repository is saved for each deployment. So -knowing what the state is of your current environments is only a `git fetch` -away. +To better understand how environments and deployments work, let's consider an +example. We assume that you have already created a project in GitLab and set up +a Runner. The example will cover the following: -In your git config, append the `[remote "<your-remote>"]` block with an extra -fetch line: +- We are developing an application +- We want to run tests and build our app on all branches +- Our default branch is `master` +- We deploy the app only when a pipeline on `master` branch is run + +Let's see how it all ties together. + +## Defining environments + +Let's consider the following `.gitlab-ci.yml` example: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master ``` -fetch = +refs/environments/*:refs/remotes/origin/environments/* + +We have defined 3 [stages](yaml/README.md#stages): + +- test +- build +- deploy + +The jobs assigned to these stages will run in this order. If a job fails, then +the builds that are assigned to the next stage won't run, rendering the pipeline +as failed. In our case, the `test` job will run first, then the `build` and +lastly the `deploy_staging`. With this, we ensure that first the tests pass, +then our app is able to be built successfully, and lastly we deploy to the +staging server. + +The `environment` keyword is just a hint for GitLab that this job actually +deploys to this environment's `name`. It can also have a `url` which, as we +will later see, is exposed in various places within GitLab. Each time a job that +has an environment specified and succeeds, a deployment is recorded, remembering +the Git SHA and environment name. + +>**Note:** +Starting with GitLab 8.15, the environment name is exposed to the Runner in +two forms: `$CI_ENVIRONMENT_NAME`, and `$CI_ENVIRONMENT_SLUG`. The first is +the name given in `.gitlab-ci.yml` (with any variables expanded), while the +second is a "cleaned-up" version of the name, suitable for use in URLs, DNS, +etc. + +To sum up, with the above `.gitlab-ci.yml` we have achieved that: + +- All branches will run the `test` and `build` jobs. +- The `deploy_staging` job will run [only](yaml/README.md#only) on the `master` + branch which means all merge requests that are created from branches don't + get to deploy to the staging server +- When a merge request is merged, all jobs will run and the `deploy_staging` + in particular will deploy our code to a staging server while the deployment + will be recorded in an environment named `staging`. + +Let's now see how that information is exposed within GitLab. + +## Viewing the current status of an environment + +The environment list under your project's **Pipelines ➔ Environments**, is +where you can find information of the last deployment status of an environment. + +Here's how the Environments page looks so far. + + + +There's a bunch of information there, specifically you can see: + +- The environment's name with a link to its deployments +- The last deployment ID number and who performed it +- The build ID of the last deployment with its respective job name +- The commit information of the last deployment such as who committed, to what + branch and the Git SHA of the commit +- The exact time the last deployment was performed +- A button that takes you to the URL that you have defined under the + `environment` keyword in `.gitlab-ci.yml` +- A button that re-deploys the latest deployment, meaning it runs the job + defined by the environment name for that specific commit + +>**Notes:** +- While you can create environments manually in the web interface, we recommend + that you define your environments in `.gitlab-ci.yml` first. They will + be automatically created for you after the first deploy. +- The environments page can only be viewed by Reporters and above. For more + information on the permissions, see the [permissions documentation][permissions]. +- Only deploys that happen after your `.gitlab-ci.yml` is properly configured + will show up in the "Environment" and "Last deployment" lists. + +The information shown in the Environments page is limited to the latest +deployments, but as you may have guessed an environment can have multiple +deployments. + +## Viewing the deployment history of an environment + +GitLab keeps track of your deployments, so you always know what is currently +being deployed on your servers. That way you can have the full history of your +deployments per every environment right in your browser. Clicking on an +environment will show the history of its deployments. Assuming you have deployed +multiple times already, here's how a specific environment's page looks like. + + + +We can see the same information as when in the Environments page, but this time +all deployments are shown. As you may have noticed, apart from the **Re-deploy** +button there are now **Rollback** buttons for each deployment. Let's see how +that works. + +## Rolling back changes + +You can't control everything, so sometimes things go wrong. When that unfortunate +time comes GitLab has you covered. Simply by clicking the **Rollback** button +that can be found in the deployments page +(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the +job with the commit associated with it. + +>**Note:** +Bear in mind that your mileage will vary and it's entirely up to how you define +the deployment process in the job's `script` whether the rollback succeeds or not. +GitLab CI is just following orders. + +Thankfully that was the staging server that we had to rollback, and since we +learn from our mistakes, we decided to not make the same again when we deploy +to the production server. Enter manual actions for deployments. + +## Manually deploying to environments + +Turning a job from running automatically to a manual action is as simple as +adding `when: manual` to it. To expand on our previous example, let's add +another job that this time deploys our app to a production server and is +tracked by a `production` environment. The `.gitlab-ci.yml` looks like this +so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -## Defining environments +The `when: manual` action exposes a play button in GitLab's UI and the +`deploy_prod` job will only be triggered if and when we click that play button. +You can find it in the pipeline, build, environment, and deployment views. + +| Pipelines | Single pipeline | Environments | Deployments | Builds | +| --------- | ----------------| ------------ | ----------- | -------| +|  |  |  |  |  | + +Clicking on the play button in either of these places will trigger the +`deploy_prod` job, and the deployment will be recorded under a new +environment named `production`. + +>**Note:** +Remember that if your environment's name is `production` (all lowercase), then +it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). +Double the benefit! + +## Web terminals + +>**Note:** +Web terminals were added in GitLab 8.15 and are only available to project +masters and owners. + +If you deploy to your environments with the help of a deployment service (e.g., +the [Kubernetes](../project_services/kubernetes.md) service), GitLab can open +a terminal session to your environment! This is a very powerful feature that +allows you to debug issues without leaving the comfort of your web browser. To +enable it, just follow the instructions given in the service documentation. + +Once enabled, your environments will gain a "terminal" button: + + + +You can also access the terminal button from the page for a specific environment: + + + +Wherever you find it, clicking the button will take you to a separate page to +establish the terminal session: + + + +This works just like any other terminal - you'll be in the container created +by your deployment, so you can run shell commands and get responses in real +time, check the logs, try out configuration or code tweaks, etc. You can open +multiple terminals to the same environment - they each get their own shell +session - and even a multiplexer like `screen` or `tmux`! + +>**Note:** +Container-based deployments often lack basic tools (like an editor), and may +be stopped or restarted at any time. If this happens, you will lose all your +changes! Treat this as a debugging tool, not a comprehensive online IDE. You +can use [Koding](../administration/integration/koding.md) for online +development. + +--- + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. -You can create and delete environments manually in the web interface, but we -recommend that you define your environments in `.gitlab-ci.yml` first, which -will automatically create environments for you after the first deploy. +## Dynamic environments -The `environment` is just a hint for GitLab that this job actually deploys to -this environment. Each time the job succeeds, a deployment is recorded, -remembering the git SHA and environment. +As the name suggests, it is possible to create environments on the fly by just +declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is +the basis of [Review apps](review_apps/index.md). -Add something like this to your `.gitlab-ci.yml`: +GitLab Runner exposes various [environment variables][variables] when a job runs, +and as such, you can use them as environment names. Let's add another job in +our example which will deploy to all branches except `master`: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_SLUG.review.example.com + only: + - branches + except: + - master ``` -production: + +Let's break it down in pieces. The job's name is `deploy_review` and it runs +on the `deploy` stage. The `script` at this point is fictional, you'd have to +use your own based on your deployment. Then, we set the `environment` with the +`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting +one. Since the [environment name][env-name] can contain slashes (`/`), we can +use this pattern to distinguish between dynamic environments and the regular +ones. + +So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` +which takes the value of the branch name. Since `$CI_BUILD_REF_NAME` itself may +also contain `/`, or other characters that would be invalid in a domain name or +URL, we use `$CI_ENVIRONMENT_SLUG` in the `environment:url` so that the +environment can get a specific and distinct URL for each branch. In this case, +given a `$CI_BUILD_REF_NAME` of `100-Do-The-Thing`, the URL will be something +like `https://review-100-do-the-4f99a2.example.com`. Again, the way you set up +the web server to serve these requests is based on your setup. + +You could also use `$CI_BUILD_REF_SLUG` in `environment:url`, e.g.: +`https://$CI_BUILD_REF_SLUG.review.example.com`. We use `$CI_ENVIRONMENT_SLUG` +here because it is guaranteed to be unique, but if you're using a workflow like +[GitLab Flow][gitlab-flow], collisions are very unlikely, and you may prefer +environment names to be more closely based on the branch name - the example +above would give you an URL like `https://100-do-the-thing.review.example.com` + +Last but not least, we tell the job to run [`only`][only] on branches +[`except`][only] master. + +>**Note:** +You are not bound to use the same prefix or only slashes in the dynamic +environments' names (`/`), but as we will see later, this will enable the +[grouping similar environments](#grouping-similar-environments) feature. + +The whole `.gitlab-ci.yml` looks like this so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com + only: + - branches + except: + - master + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: stage: deploy - script: dpl... - environment: production + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -See full [documentation](yaml/README.md#environment). +A more realistic example would include copying files to a location where a +webserver (NGINX) could then read and serve. The example below will copy the +`public` directory to `/srv/nginx/$CI_BUILD_REF_SLUG/public`: -## Seeing environment status +```yaml +review_app: + stage: deploy + script: + - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_SLUG + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_SLUG.example.com +``` -You can find the environment list under **Pipelines > Environments** for your -project. You'll see the git SHA and date of the last deployment to each -environment defined. +It is assumed that the user has already setup NGINX and GitLab Runner in the +server this job will run on. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Be sure to check out the [limitations](#limitations) section for some edge +cases regarding naming of your branches and Review Apps. + +--- + +The development workflow would now be: + +- Developer creates a branch locally +- Developer makes changes, commits and pushes the branch to GitLab +- Developer creates a merge request + +Behind the scenes: + +- GitLab Runner picks up the changes and starts running the jobs +- The jobs run sequentially as defined in `stages` + - First, the tests pass + - Then, the build begins and successfully also passes + - Lastly, the app is deployed to an environment with a name specific to the + branch + +So now, every branch gets its own environment and is deployed to its own place +with the added benefit of having a [history of deployments](#viewing-the-deployment-history-of-an-environment) +and also being able to [rollback changes](#rolling-back-changes) if needed. +Let's briefly see where URL that's defined in the environments is exposed. + +## Making use of the environment URL + +The environment URL is exposed in a few places within GitLab. + +| In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | +| -------------------- | ------------ | ----------- | +|  |  |  | + +If a merge request is eventually merged to the default branch (in our case +`master`) and that branch also deploys to an environment (in our case `staging` +and/or `production`) you can see this information in the merge request itself. + + + +--- + +We now have a full development cycle, where our app is tested, built, deployed +as a Review app, deployed to a staging server once the merge request is merged, +and finally manually deployed to the production server. What we just described +is a single workflow, but imagine tens of developers working on a project +at the same time. They each push to their branches, and dynamic environments are +created all the time. In that case, we probably need to do some clean up. Read +next how environments can be stopped. + +## Stopping an environment -## Seeing deployment history +By stopping an environment, you are effectively terminating its recording of the +deployments that happen in it. -Clicking on an environment will show the history of deployments. +A branch is associated with an environment when the CI pipeline that is created +for this branch, was recently deployed to this environment. You can think of +the CI pipeline as the glue between the branch and the environment: +`branch ➔ CI pipeline ➔ environment`. + +There is a special case where environments can be manually stopped. That can +happen if you provide another job for that matter. The syntax is a little +tricky since a job calls another job to do the job. + +Consider the following example where the `deploy_review` calls the `stop_review` +to clean up and stop the environment: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com + on_stop: stop_review + only: + - branches + except: + - master + +stop_review: + variables: + GIT_STRATEGY: none + script: + - echo "Remove review app" + when: manual + environment: + name: review/$CI_BUILD_REF_NAME + action: stop +``` + +Setting the [`GIT_STRATEGY`][git-strategy] to `none` is necessary on the +`stop_review` job so that the [GitLab Runner] won't try to checkout the code +after the branch is deleted. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Starting with GitLab 8.14, dynamic environments will be stopped automatically +when their associated branch is deleted. + +When you have an environment that has a stop action defined (typically when +the environment describes a review app), GitLab will automatically trigger a +stop action when the associated branch is deleted. + +You can read more in the [`.gitlab-ci.yml` reference][onstop]. + +## Grouping similar environments + +> [Introduced][ce-7015] in GitLab 8.14. + +As we've seen in the [dynamic environments](#dynamic-environments), you can +prepend their name with a word, then followed by a `/` and finally the branch +name which is automatically defined by the `CI_BUILD_REF_NAME` variable. + +In short, environments that are named like `type/foo` are presented under a +group named `type`. + +In our minimal example, we name the environments `review/$CI_BUILD_REF_NAME` +where `$CI_BUILD_REF_NAME` is the branch name: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME +``` + +In that case, if you visit the Environments page, and provided the branches +exist, you should see something like: + + + +## Checkout deployments locally + +Since 8.13, a reference in the git repository is saved for each deployment, so +knowing the state of your current environments is only a `git fetch` away. + +In your git config, append the `[remote "<your-remote>"]` block with an extra +fetch line: + +``` +fetch = +refs/environments/*:refs/remotes/origin/environments/* +``` + +## Limitations + +1. You are limited to use only the [CI predefined variables][variables] in the + `environment: name`. If you try to re-use variables defined inside `script` + as part of the environment name, it will not work. + +## Further reading + +Below are some links you may find interesting: + +- [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment) +- [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +- [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs +[yaml]: yaml/README.md +[kubernetes-service]: ../project_services/kubernetes.md] [environments]: #environments [deployments]: #deployments +[permissions]: ../user/permissions.md +[variables]: variables/README.md +[env-name]: yaml/README.md#environment-name +[only]: yaml/README.md#only-and-except +[onstop]: yaml/README.md#environment-on_stop +[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 +[gitlab-flow]: ../workflow/gitlab_flow.md +[gitlab runner]: https://docs.gitlab.com/runner/ +[git-strategy]: yaml/README.md#git-strategy diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index 175e9d79904..82ffb841729 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -40,7 +40,7 @@ repository with the following content: #!/bin/bash # We need to install dependencies only for Docker -[[ ! -e /.dockerenv ]] && [[ ! -e /.dockerinit ]] && exit 0 +[[ ! -e /.dockerenv ]] && exit 0 set -xe diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 08c10d391ea..42f15a27f12 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -34,7 +34,7 @@ production: This project has three jobs: 1. `test` - used to test Rails application, 2. `staging` - used to automatically deploy staging environment every push to `master` branch -3. `production` - used to automatically deploy production environmnet for every created tag +3. `production` - used to automatically deploy production environment for every created tag ### Store API keys You'll need to create two variables in `Project > Variables`: diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index 7412fdbbc78..85f8849fa99 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -1,11 +1,11 @@ -## Test a Scala application +# Test and deploy to Heroku a Scala application This example demonstrates the integration of Gitlab CI with Scala applications using SBT. Checkout the example [project](https://gitlab.com/gitlab-examples/scala-sbt) and [build status](https://gitlab.com/gitlab-examples/scala-sbt/builds). -### Add `.gitlab-ci.yml` file to project +## Add `.gitlab-ci.yml` file to project The following `.gitlab-ci.yml` should be added in the root of your repository to trigger CI: @@ -13,10 +13,14 @@ repository to trigger CI: ``` yaml image: java:8 +stages: + - test + - deploy + before_script: - apt-get update -y - apt-get install apt-transport-https -y - # Install SBT + ## Install SBT - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823 - apt-get update -y @@ -24,8 +28,17 @@ before_script: - sbt sbt-version test: + stage: test script: - sbt clean coverage test coverageReport + +deploy: + stage: deploy + script: + - apt-get update -yq + - apt-get install rubygems ruby-dev -y + - gem install dpl + - dpl --provider=heroku --app=gitlab-play-sample-app --api-key=$HEROKU_API_KEY ``` The `before_script` installs [SBT](http://www.scala-sbt.org/) and @@ -33,15 +46,31 @@ displays the version that is being used. The `test` stage executes SBT to compile and test the project. [scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT plugin to measure test coverage. +The `deploy` stage automatically deploys the project to Heroku using dpl. You can use other versions of Scala and SBT by defining them in `build.sbt`. -### Display test coverage in build +## Display test coverage in build Add the `Coverage was \[\d+.\d+\%\]` regular expression in the -**Settings > Edit Project > Test coverage parsing** project setting to -retrieve the test coverage rate from the build trace and have it +**Settings ➔ Edit Project ➔ Test coverage parsing** project setting to +retrieve the [test coverage] rate from the build trace and have it displayed with your builds. **Builds** must be enabled for this option to appear. + +## Heroku application + +A Heroku application is required. You can create one through the +[Dashboard](https://dashboard.heroku.com/). Substitute `gitlab-play-sample-app` +in the `.gitlab-ci.yml` file with your application's name. + +## Heroku API key + +You can look up your Heroku API key in your +[account](https://dashboard.heroku.com/account). Add a secure [variable] with +this value in **Project ➔ Variables** with key `HEROKU_API_KEY`. + +[variable]: ../variables/README.md#user-defined-variables-secure-variables +[test coverage]: ../../user/project/pipelines/settings.md#test-coverage-report-badge diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md new file mode 100644 index 00000000000..869743ce80a --- /dev/null +++ b/doc/ci/git_submodules.md @@ -0,0 +1,97 @@ +# Using Git submodules with GitLab CI + +> **Notes:** +- GitLab 8.12 introduced a new [CI build permissions model][newperms] and you + are encouraged to upgrade your GitLab instance if you haven't done already. + If you are **not** using GitLab 8.12 or higher, you would need to work your way + around submodules in order to access the sources of e.g., `gitlab.com/group/project` + with the use of [SSH keys](ssh_keys/README.md). +- With GitLab 8.12 onward, your permissions are used to evaluate what a CI build + can access. More information about how this system works can be found in the + [Build permissions model](../user/permissions.md#builds-permissions). +- The HTTP(S) Git protocol [must be enabled][gitpro] in your GitLab instance. + +## Configuring the `.gitmodules` file + +If dealing with [Git submodules][gitscm], your project will probably have a file +named `.gitmodules`. + +Let's consider the following example: + +1. Your project is located at `https://gitlab.com/secret-group/my-project`. +1. To checkout your sources you usually use an SSH address like + `git@gitlab.com:secret-group/my-project.git`. +1. Your project depends on `https://gitlab.com/group/project`, which you want + to include as a submodule. + +If you are using GitLab 8.12+ and your submodule is on the same GitLab server, +you must update your `.gitmodules` file to use **relative URLs**. +Since Git allows the usage of relative URLs for your `.gitmodules` configuration, +this easily allows you to use HTTP(S) for cloning all your CI builds and SSH +for all your local checkouts. The `.gitmodules` would look like: + +```ini +[submodule "project"] + path = project + url = ../../group/project.git +``` + +The above configuration will instruct Git to automatically deduce the URL that +should be used when cloning sources. Whether you use HTTP(S) or SSH, Git will use +that same channel and it will allow to make all your CI builds use HTTP(S) +(because GitLab CI only uses HTTP(S) for cloning your sources), and all your local +clones will continue using SSH. + +For all other submodules not located on the same GitLab server, use the full +HTTP(S) protocol URL: + +```ini +[submodule "project-x"] + path = project-x + url = https://gitserver.com/group/project-x.git +``` + +Once `.gitmodules` is correctly configured, you can move on to +[configuring your `.gitlab-ci.yml`](#using-git-submodules-in-your-ci-jobs). + +## Using Git submodules in your CI jobs + +There are a few steps you need to take in order to make submodules work +correctly with your CI builds: + +1. First, make sure you have used [relative URLs](#configuring-the-gitmodules-file) + for the submodules located in the same GitLab server. +1. Next, if you are using `gitlab-ci-multi-runner` v1.10+, you can set the + `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell + the runner to fetch your submodules before the build: + ```yaml + variables: + GIT_SUBMODULE_STRATEGY: recursive + ``` + See the [`.gitlab-ci.yml` reference](yaml/README.md#git-submodule-strategy) + for more details about `GIT_SUBMODULE_STRATEGY`. + +1. If you are using an older version of `gitlab-ci-multi-runner`, then use + `git submodule sync/update` in `before_script`: + + ```yaml + before_script: + - git submodule sync --recursive + - git submodule update --init --recursive + ``` + + `--recursive` should be used in either both or none (`sync/update`) depending on + whether you have recursive submodules. + +The rationale to set the `sync` and `update` in `before_script` is because of +the way Git submodules work. On a fresh Runner workspace, Git will set the +submodule URL including the token in `.git/config` +(or `.git/modules/<submodule>/config`) based on `.gitmodules` and the current +remote URL. On subsequent builds on the same Runner, `.git/config` is cached +and already contains a full URL for the submodule, corresponding to the previous +build, and to **a token from a previous build**. `sync` allows to force updating +the full URL. + +[gitpro]: ../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols +[gitscm]: https://git-scm.com/book/en/v2/Git-Tools-Submodules "Git submodules documentation" +[newperms]: ../user/project/new_ci_build_permissions_model.md diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png Binary files differindex 35780e277ae..2d7eec8a949 100644 --- a/doc/ci/img/builds_tab.png +++ b/doc/ci/img/builds_tab.png diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differnew file mode 100644 index 00000000000..7ded0c97b72 --- /dev/null +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differnew file mode 100644 index 00000000000..5c031ad0d9d --- /dev/null +++ b/doc/ci/img/environments_available_staging.png diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differnew file mode 100644 index 00000000000..0f42b368c5b --- /dev/null +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differnew file mode 100644 index 00000000000..44010f6aa6f --- /dev/null +++ b/doc/ci/img/environments_link_url.png diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differnew file mode 100644 index 00000000000..4f90143527a --- /dev/null +++ b/doc/ci/img/environments_link_url_deployments.png diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differnew file mode 100644 index 00000000000..64f134e0b0d --- /dev/null +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differnew file mode 100644 index 00000000000..e7cf63a1031 --- /dev/null +++ b/doc/ci/img/environments_manual_action_builds.png diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differnew file mode 100644 index 00000000000..2b3f6f3edad --- /dev/null +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differnew file mode 100644 index 00000000000..e0c07604e7f --- /dev/null +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differnew file mode 100644 index 00000000000..82bbae88027 --- /dev/null +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differnew file mode 100644 index 00000000000..36337cb1870 --- /dev/null +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differnew file mode 100644 index 00000000000..7bff84362a3 --- /dev/null +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_terminal_button_on_index.png b/doc/ci/img/environments_terminal_button_on_index.png Binary files differnew file mode 100644 index 00000000000..6f05b2aa343 --- /dev/null +++ b/doc/ci/img/environments_terminal_button_on_index.png diff --git a/doc/ci/img/environments_terminal_button_on_show.png b/doc/ci/img/environments_terminal_button_on_show.png Binary files differnew file mode 100644 index 00000000000..9469fab99ab --- /dev/null +++ b/doc/ci/img/environments_terminal_button_on_show.png diff --git a/doc/ci/img/environments_terminal_page.png b/doc/ci/img/environments_terminal_page.png Binary files differnew file mode 100644 index 00000000000..fde1bf325a6 --- /dev/null +++ b/doc/ci/img/environments_terminal_page.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differnew file mode 100644 index 00000000000..821352188ef --- /dev/null +++ b/doc/ci/img/environments_view.png diff --git a/doc/ci/img/features_settings.png b/doc/ci/img/features_settings.png Binary files differindex 38d7036f606..c159253d1c9 100644 --- a/doc/ci/img/features_settings.png +++ b/doc/ci/img/features_settings.png diff --git a/doc/ci/img/pipelines.png b/doc/ci/img/pipelines.png Binary files differnew file mode 100644 index 00000000000..5937e9d99c8 --- /dev/null +++ b/doc/ci/img/pipelines.png diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 7d100a4fd93..f91b9d350f7 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -11,6 +11,8 @@ concurrent [Runners]), and if they all succeed, the pipeline moves on to the next stage. If one of the builds fails, the next stage is not (usually) executed. + + ## Builds Builds are individual runs of [jobs]. Not to be confused with a `build` job or @@ -34,6 +36,37 @@ Clicking on a pipeline will show the builds that were run for that pipeline. Clicking on an individual build will show you its build trace, and allow you to cancel the build, retry it, or erase the build trace. +## How the pipeline duration is calculated + +Total running time for a given pipeline would exclude retries and pending +(queue) time. We could reduce this problem down to finding the union of +periods. + +So each job would be represented as a `Period`, which consists of +`Period#first` as when the job started and `Period#last` as when the +job was finished. A simple example here would be: + +* A (1, 3) +* B (2, 4) +* C (6, 7) + +Here A begins from 1, and ends to 3. B begins from 2, and ends to 4. +C begins from 6, and ends to 7. Visually it could be viewed as: + +``` +0 1 2 3 4 5 6 7 + AAAAAAA + BBBBBBB + CCCC +``` + +The union of A, B, and C would be (1, 4) and (6, 7), therefore the +total running time should be: + +``` +(4 - 1) + (7 - 6) => 4 +``` + ## Badges Build status and test coverage report badges are available. You can find their diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png Binary files differindex b53a6cd86b0..87643d62d58 100644 --- a/doc/ci/quick_start/img/build_log.png +++ b/doc/ci/quick_start/img/build_log.png diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png Binary files differindex 47862761ffe..d287ae3064f 100644 --- a/doc/ci/quick_start/img/builds_status.png +++ b/doc/ci/quick_start/img/builds_status.png diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png Binary files differindex a53562ce328..29c2fea5d6d 100644 --- a/doc/ci/quick_start/img/new_commit.png +++ b/doc/ci/quick_start/img/new_commit.png diff --git a/doc/ci/quick_start/img/pipelines_status.png b/doc/ci/quick_start/img/pipelines_status.png Binary files differindex 6bc97bb739c..53ccc49bd66 100644 --- a/doc/ci/quick_start/img/pipelines_status.png +++ b/doc/ci/quick_start/img/pipelines_status.png diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png Binary files differindex 23261123b18..5ce6fe8e17c 100644 --- a/doc/ci/quick_start/img/runners_activated.png +++ b/doc/ci/quick_start/img/runners_activated.png diff --git a/doc/ci/quick_start/img/single_commit_status_pending.png b/doc/ci/quick_start/img/single_commit_status_pending.png Binary files differindex ccf3ac957bb..91fc9011847 100644 --- a/doc/ci/quick_start/img/single_commit_status_pending.png +++ b/doc/ci/quick_start/img/single_commit_status_pending.png diff --git a/doc/ci/quick_start/img/status_pending.png b/doc/ci/quick_start/img/status_pending.png Binary files differindex 9feacf0c961..cbd44a189d3 100644 --- a/doc/ci/quick_start/img/status_pending.png +++ b/doc/ci/quick_start/img/status_pending.png diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr.png b/doc/ci/review_apps/img/review_apps_preview_in_mr.png Binary files differnew file mode 100644 index 00000000000..0300392f24b --- /dev/null +++ b/doc/ci/review_apps/img/review_apps_preview_in_mr.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md new file mode 100644 index 00000000000..c679ea4e298 --- /dev/null +++ b/doc/ci/review_apps/index.md @@ -0,0 +1,124 @@ +# Getting started with Review Apps + +> +- [Introduced][ce-21971] in GitLab 8.12. Further additions were made in GitLab + 8.13 and 8.14. +- Inspired by [Heroku's Review Apps][heroku-apps] which itself was inspired by + [Fourchette]. + +The basis of Review Apps is the [dynamic environments] which allow you to create +a new environment (dynamically) for each one of your branches. + +A Review App can then be visible as a link when you visit the [merge request] +relevant to the branch. That way, you are able to see live all changes introduced +by the merge request changes. Reviewing anything, from performance to interface +changes, becomes much easier with a live environment and as such, Review Apps +can make a huge impact on your development flow. + +They mostly make sense to be used with web applications, but you can use them +any way you'd like. + +## Overview + +Simply put, a Review App is a mapping of a branch with an environment as there +is a 1:1 relation between them. + +Here's an example of what it looks like when viewing a merge request with a +dynamically set environment. + + + +In the image above you can see that the `add-new-line` branch was successfully +built and deployed under a dynamic environment and can be previewed with an +also dynamically URL. + +The details of the Review Apps implementation depend widely on your real +technology stack and on your deployment process. The simplest case is to +deploy a simple static HTML website, but it will not be that straightforward +when your app is using a database for example. To make a branch be deployed +on a temporary instance and booting up this instance with all required software +and services automatically on the fly is not a trivial task. However, it is +doable, especially if you use Docker, or at least a configuration management +tool like Chef, Puppet, Ansible or Salt. + +## Prerequisites + +To get a better understanding of Review Apps, you must first learn how +environments and deployments work. The following docs will help you grasp that +knowledge: + +1. First, learn about [environments][] and their role in the development workflow. +1. Then make a small stop to learn about [CI variables][variables] and how they + can be used in your CI jobs. +1. Next, explore the [`environment` syntax][yaml-env] as defined in `.gitlab-ci.yml`. + This will be your primary reference when you are finally comfortable with + how environments work. +1. Additionally, find out about [manual actions][] and how you can use them to + deploy to critical environments like production with the push of a button. +1. And as a last step, follow the [example tutorials](#examples) which will + guide you step by step to set up the infrastructure and make use of + Review Apps. + +## Configuration + +The configuration of Review apps depends on your technology stack and your +infrastructure. Read the [dynamic environments] documentation to understand +how to define and create them. + +## Creating and destroying Review Apps + +The creation and destruction of a Review App is defined in `.gitlab-ci.yml` +at a job level under the `environment` keyword. + +Check the [environments] documentation how to do so. + +## A simple workflow + +The process of adding Review Apps in your workflow would look like: + +1. Set up the infrastructure to host and deploy the Review Apps. +1. [Install][install-runner] and [configure][conf-runner] a Runner that does + the deployment. +1. Set up a job in `.gitlab-ci.yml` that uses the predefined + [predefined CI environment variable][variables] `${CI_BUILD_REF_NAME}` to + create dynamic environments and restrict it to run only on branches. +1. Optionally set a job that [manually stops][manual-env] the Review Apps. + +From there on, you would follow the branched Git flow: + +1. Push a branch and let the Runner deploy the Review App based on the `script` + definition of the dynamic environment job. +1. Wait for the Runner to build and/or deploy your web app. +1. Click on the link that's present in the MR related to the branch and see the + changes live. + +## Limitations + +Check the [environments limitations](../environments.md#limitations). + +## Examples + +A list of examples used with Review Apps can be found below: + +- [Use with NGINX][app-nginx] - Use NGINX and the shell executor of GitLab Runner + to deploy a simple HTML website. + +And below is a soon to be added examples list: + +- Use with Amazon S3 +- Use on Heroku with dpl +- Use with OpenShift/kubernetes + +[app-nginx]: https://gitlab.com/gitlab-examples/review-apps-nginx +[ce-21971]: https://gitlab.com/gitlab-org/gitlab-ce/issues/21971 +[dynamic environments]: ../environments.md#dynamic-environments +[environments]: ../environments.md +[fourchette]: https://github.com/rainforestapp/fourchette +[heroku-apps]: https://devcenter.heroku.com/articles/github-integration-review-apps +[manual actions]: ../environments.md#manual-actions +[merge request]: ../../user/project/merge_requests.md +[variables]: ../variables/README.md +[yaml-env]: ../yaml/README.md#environment +[install-runner]: https://docs.gitlab.com/runner/install/ +[conf-runner]: https://docs.gitlab.com/runner/commands/ +[manual-env]: ../environments.md#stopping-an-environment diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 84048f1d25f..efca05af7b8 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -6,7 +6,7 @@ GitLab 8.12 has a completely redesigned build permissions system. Read all about the [new model and its implications](../../user/project/new_ci_build_permissions_model.md#build-triggers). -Triggers can be used to force a rebuild of a specific branch, tag or commit, +Triggers can be used to force a rebuild of a specific `ref` (branch or tag) with an API call. ## Add a trigger @@ -29,6 +29,10 @@ irreversible. ## Trigger a build +> **Note**: +Valid refs are only the branches and tags. If you pass a commit SHA as a ref, +it will not trigger a build. + To trigger a build you need to send a `POST` request to GitLab's API endpoint: ``` @@ -36,8 +40,8 @@ POST /projects/:id/trigger/builds ``` The required parameters are the trigger's `token` and the Git `ref` on which -the trigger will be performed. Valid refs are the branch, the tag or the commit -SHA. The `:id` of a project can be found by [querying the API](../../api/projects.md) +the trigger will be performed. Valid refs are the branch and the tag. The `:id` +of a project can be found by [querying the API](../../api/projects.md) or by visiting the **Triggers** page which provides self-explanatory examples. When a rebuild is triggered, the information is exposed in GitLab's UI under @@ -58,6 +62,22 @@ below. See the [Examples](#examples) section for more details on how to actually trigger a rebuild. +## Trigger a build from webhook + +> Introduced in GitLab 8.14. + +To trigger a build from webhook of another project you need to add the following +webhook url for Push and Tag push events: + +``` +https://gitlab.example.com/api/v3/projects/:id/ref/:ref/trigger/builds?token=TOKEN +``` + +> **Note**: +- `ref` should be passed as part of url in order to take precedence over `ref` + from webhook body that designates the branchref that fired the trigger in the source repository. +- `ref` should be url encoded if contains slashes. + ## Pass build variables to a trigger You can pass any number of arbitrary variables in the trigger API call and they @@ -169,6 +189,14 @@ curl --request POST \ https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` +### Using webhook to trigger builds + +You can add the following webhook to another project in order to trigger a build: + +``` +https://gitlab.example.com/api/v3/projects/9/ref/master/trigger/builds?token=TOKEN&variables[UPLOAD_TO_S3]=true +``` + ### Using cron to trigger nightly builds Whether you craft a script or just run cURL directly, you can trigger builds diff --git a/doc/ci/triggers/img/builds_page.png b/doc/ci/triggers/img/builds_page.png Binary files differindex c2cf4b1852c..fded5839f76 100644 --- a/doc/ci/triggers/img/builds_page.png +++ b/doc/ci/triggers/img/builds_page.png diff --git a/doc/ci/triggers/img/trigger_single_build.png b/doc/ci/triggers/img/trigger_single_build.png Binary files differindex fa86f0fee3d..c4a5550d640 100644 --- a/doc/ci/triggers/img/trigger_single_build.png +++ b/doc/ci/triggers/img/trigger_single_build.png diff --git a/doc/ci/triggers/img/trigger_variables.png b/doc/ci/triggers/img/trigger_variables.png Binary files differindex b2fcc65d304..65fe1ea9ab6 100644 --- a/doc/ci/triggers/img/trigger_variables.png +++ b/doc/ci/triggers/img/trigger_variables.png diff --git a/doc/ci/triggers/img/triggers_page.png b/doc/ci/triggers/img/triggers_page.png Binary files differindex 438f285ae2d..56d13905ce6 100644 --- a/doc/ci/triggers/img/triggers_page.png +++ b/doc/ci/triggers/img/triggers_page.png diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index a4c3a731a20..d3b9611b02e 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,22 +1,31 @@ -## Variables +# Variables -When receiving a build from GitLab CI, the runner prepares the build environment. -It starts by setting a list of **predefined variables** (Environment Variables) and a list of **user-defined variables** +When receiving a build from GitLab CI, the [Runner] prepares the build environment. +It starts by setting a list of **predefined variables** (environment variables) +and a list of **user-defined variables**. -The variables can be overwritten. They take precedence over each other in this order: -1. Trigger variables -1. Secure variables -1. YAML-defined job-level variables -1. YAML-defined global variables -1. Predefined variables +## Priority of variables -For example, if you define: -1. `API_TOKEN=SECURE` as Secure Variable -1. `API_TOKEN=YAML` as YAML-defined variable +The variables can be overwritten and they take precedence over each other in +this order: -The `API_TOKEN` will take the Secure Variable value: `SECURE`. +1. [Trigger variables][triggers] (take precedence over all) +1. [Secret variables](#secret-variables) +1. YAML-defined [job-level variables](../yaml/README.md#job-variables) +1. YAML-defined [global variables](../yaml/README.md#variables) +1. [Deployment variables](#deployment-variables) +1. [Predefined variables](#predefined-variables-environment-variables) (are the + lowest in the chain) -### Predefined variables (Environment Variables) +For example, if you define `API_TOKEN=secure` as a secret variable and +`API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value +`secure` as the secret variables are higher in the chain. + +## Predefined variables (Environment variables) + +Some of the predefined environment variables are available only if a minimum +version of [GitLab Runner][runner] is used. Consult the table below to find the +version of Runner required. | Variable | GitLab | Runner | Description | |-------------------------|--------|--------|-------------| @@ -32,6 +41,7 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`. | **CI_BUILD_NAME** | all | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | | **CI_BUILD_STAGE** | all | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | **CI_BUILD_REF_NAME** | all | all | The branch or tag name for which project is built | +| **CI_BUILD_REF_SLUG** | 8.15 | all | `$CI_BUILD_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | | **CI_BUILD_REPO** | all | all | The URL to clone the Git repository | | **CI_BUILD_TRIGGERED** | all | 0.5 | The flag to indicate that build was [triggered] | | **CI_BUILD_MANUAL** | 8.12 | all | The flag to indicate that build was manually started | @@ -43,16 +53,20 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`. | **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the build is run | +| **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this build | +| **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | | **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | **CI_RUNNER_ID** | 8.10 | 0.5 | The unique id of runner being used | | **CI_RUNNER_DESCRIPTION** | 8.10 | 0.5 | The description of the runner as saved in GitLab | | **CI_RUNNER_TAGS** | 8.10 | 0.5 | The defined runner tags | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | +| **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a build | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a build | +| **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a build | | **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the build | | **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the build | -**Some of the variables are only available when using runner with at least defined version.** Example values: @@ -60,7 +74,7 @@ Example values: export CI_BUILD_ID="50" export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_BUILD_REF_NAME="master" -export CI_BUILD_REPO="https://gitab-ci-token:abcde-1234ABCD5678ef@gitlab.com/gitlab-org/gitlab-ce.git" +export CI_BUILD_REPO="https://gitab-ci-token:abcde-1234ABCD5678ef@example.com/gitlab-org/gitlab-ce.git" export CI_BUILD_TAG="1.0.0" export CI_BUILD_NAME="spec:other" export CI_BUILD_STAGE="test" @@ -73,9 +87,9 @@ export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_NAME="gitlab-ce" export CI_PROJECT_NAMESPACE="gitlab-org" export CI_PROJECT_PATH="gitlab-org/gitlab-ce" -export CI_PROJECT_URL="https://gitlab.com/gitlab-org/gitlab-ce" -export CI_REGISTRY="registry.gitlab.com" -export CI_REGISTRY_IMAGE="registry.gitlab.com/gitlab-org/gitlab-ce" +export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-ce" +export CI_REGISTRY="registry.example.com" +export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-ce" export CI_RUNNER_ID="10" export CI_RUNNER_DESCRIPTION="my runner" export CI_RUNNER_TAGS="docker, linux" @@ -84,32 +98,80 @@ export CI_SERVER_NAME="GitLab" export CI_SERVER_REVISION="70606bf" export CI_SERVER_VERSION="8.9.0" export GITLAB_USER_ID="42" -export GITLAB_USER_EMAIL="alexzander@sporer.com" +export GITLAB_USER_EMAIL="user@example.com" ``` -### YAML-defined variables -**This feature requires GitLab Runner 0.5.0 or higher and GitLab CI 7.14 or higher ** +## `.gitlab-ci.yaml` defined variables + +>**Note:** +This feature requires GitLab Runner 0.5.0 or higher and GitLab CI 7.14 or higher. -GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in build environment. -The variables are stored in repository and are meant to store non-sensitive project configuration, ie. RAILS_ENV or DATABASE_URL. +GitLab CI allows you to add to `.gitlab-ci.yml` variables that are set in the +build environment. The variables are hence saved in the repository, and they +are meant to store non-sensitive project configuration, e.g., `RAILS_ENV` or +`DATABASE_URL`. + +For example, if you set the variable below globally (not inside a job), it will +be used in all executed commands and scripts: ```yaml variables: DATABASE_URL: "postgres://postgres@postgres/my_database" ``` -These variables can be later used in all executed commands and scripts. +The YAML-defined variables are also set to all created +[service containers](../docker/using_docker_images.md), thus allowing to fine +tune them. + +Variables can be defined at a global level, but also at a job level. To turn off +global defined variables in your job, define an empty array: + +```yaml +job_name: + variables: [] +``` + +## Secret variables -The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +>**Notes:** +- This feature requires GitLab Runner 0.4.0 or higher. +- Be aware that secret variables are not masked, and their values can be shown + in the build logs if explicitly asked to do so. If your project is public or + internal, you can set the pipelines private from your project's Pipelines + settings. Follow the discussion in issue [#13784][ce-13784] for masking the + secret variables. -Variables can be defined at a global level, but also at a job level. +GitLab CI allows you to define per-project **secret variables** that are set in +the build environment. The secret variables are stored out of the repository +(`.gitlab-ci.yml`) and are securely passed to GitLab Runner making them +available in the build environment. It's the recommended method to use for +storing things like passwords, secret keys and credentials. -More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md). +Secret variables can be added by going to your project's +**Settings ➔ Variables ➔ Add variable**. -#### Debug tracing +Once you set them, they will be available for all subsequent builds. +## Deployment variables + +>**Note:** +This feature requires GitLab CI 8.15 or higher. + +[Project services](../../project_services/project_services.md) that are +responsible for deployment configuration may define their own variables that +are set in the build environment. These variables are only defined for +[deployment builds](../environments.md). Please consult the documentation of +the project services that you are using to learn which variables they define. + +An example project service that defines deployment variables is +[Kubernetes Service](../../project_services/kubernetes.md). + +## Debug tracing + +> Introduced in GitLab Runner 1.7. +> > **WARNING:** Enabling debug tracing can have severe security implications. The - output **will** contain the content of all your secure variables and any other + output **will** contain the content of all your secret variables and any other secrets! The output **will** be uploaded to the GitLab server and made visible in build traces! @@ -124,55 +186,162 @@ trace, resulting in a verbose build trace listing all commands that were run, variables that were set, etc. Before enabling this, you should ensure builds are visible to -[team members only](../../../user/permissions.md#project-features). You should -also [erase](../pipelines.md#seeing-build-traces) all generated build traces +[team members only](../../user/permissions.md#project-features). You should +also [erase](../pipelines.md#seeing-build-status) all generated build traces before making them visible again. To enable debug traces, set the `CI_DEBUG_TRACE` variable to `true`: ```yaml -job1: +job_name: variables: CI_DEBUG_TRACE: "true" ``` -The [example project](https://gitlab.com/gitlab-examples/ci-debug-trace) -demonstrates a working configuration, including build trace examples. - -### User-defined variables (Secure Variables) -**This feature requires GitLab Runner 0.4.0 or higher** +Example truncated output with debug trace set to true: -GitLab CI allows you to define per-project **Secure Variables** that are set in -the build environment. -The secure variables are stored out of the repository (the `.gitlab-ci.yml`). -The variables are securely passed to GitLab Runner and are available in the -build environment. -It's desired method to use them for storing passwords, secret keys or whatever -you want. +```bash +... + +export CI_SERVER_TLS_CA_FILE="/builds/gitlab-examples/ci-debug-trace.tmp/CI_SERVER_TLS_CA_FILE" +if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then + echo $'\''\x1b[32;1mFetching changes...\x1b[0;m'\'' + $'\''cd'\'' "/builds/gitlab-examples/ci-debug-trace" + $'\''git'\'' "config" "fetch.recurseSubmodules" "false" + $'\''rm'\'' "-f" ".git/index.lock" + $'\''git'\'' "clean" "-ffdx" + $'\''git'\'' "reset" "--hard" + $'\''git'\'' "remote" "set-url" "origin" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git" + $'\''git'\'' "fetch" "origin" "--prune" "+refs/heads/*:refs/remotes/origin/*" "+refs/tags/*:refs/tags/*" +else + $'\''mkdir'\'' "-p" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template" + $'\''rm'\'' "-r" "-f" "/builds/gitlab-examples/ci-debug-trace" + $'\''git'\'' "config" "-f" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template/config" "fetch.recurseSubmodules" "false" + echo $'\''\x1b[32;1mCloning repository...\x1b[0;m'\'' + $'\''git'\'' "clone" "--no-checkout" "https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git" "/builds/gitlab-examples/ci-debug-trace" "--template" "/builds/gitlab-examples/ci-debug-trace.tmp/git-template" + $'\''cd'\'' "/builds/gitlab-examples/ci-debug-trace" +fi +echo $'\''\x1b[32;1mChecking out dd648b2e as master...\x1b[0;m'\'' +$'\''git'\'' "checkout" "-f" "-q" "dd648b2e48ce6518303b0bb580b2ee32fadaf045" +' ++++ hostname +++ echo 'Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb...' +Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb... +++ export CI=true +++ CI=true +++ export CI_DEBUG_TRACE=false +++ CI_DEBUG_TRACE=false +++ export CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ export CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ export CI_BUILD_REF_NAME=master +++ CI_BUILD_REF_NAME=master +++ export CI_BUILD_ID=7046507 +++ CI_BUILD_ID=7046507 +++ export CI_BUILD_REPO=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git +++ CI_BUILD_REPO=https://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@example.com/gitlab-examples/ci-debug-trace.git +++ export CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx +++ CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx +++ export CI_PROJECT_ID=1796893 +++ CI_PROJECT_ID=1796893 +++ export CI_PROJECT_DIR=/builds/gitlab-examples/ci-debug-trace +++ CI_PROJECT_DIR=/builds/gitlab-examples/ci-debug-trace +++ export CI_SERVER=yes +++ CI_SERVER=yes +++ export 'CI_SERVER_NAME=GitLab CI' +++ CI_SERVER_NAME='GitLab CI' +++ export CI_SERVER_VERSION= +++ CI_SERVER_VERSION= +++ export CI_SERVER_REVISION= +++ CI_SERVER_REVISION= +++ export GITLAB_CI=true +++ GITLAB_CI=true +++ export CI=true +++ CI=true +++ export GITLAB_CI=true +++ GITLAB_CI=true +++ export CI_BUILD_ID=7046507 +++ CI_BUILD_ID=7046507 +++ export CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx +++ CI_BUILD_TOKEN=xxxxxxxxxxxxxxxxxxxx +++ export CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ export CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ CI_BUILD_BEFORE_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 +++ export CI_BUILD_REF_NAME=master +++ CI_BUILD_REF_NAME=master +++ export CI_BUILD_NAME=debug_trace +++ CI_BUILD_NAME=debug_trace +++ export CI_BUILD_STAGE=test +++ CI_BUILD_STAGE=test +++ export CI_SERVER_NAME=GitLab +++ CI_SERVER_NAME=GitLab +++ export CI_SERVER_VERSION=8.14.3-ee +++ CI_SERVER_VERSION=8.14.3-ee +++ export CI_SERVER_REVISION=82823 +++ CI_SERVER_REVISION=82823 +++ export CI_PROJECT_ID=17893 +++ CI_PROJECT_ID=17893 +++ export CI_PROJECT_NAME=ci-debug-trace +++ CI_PROJECT_NAME=ci-debug-trace +++ export CI_PROJECT_PATH=gitlab-examples/ci-debug-trace +++ CI_PROJECT_PATH=gitlab-examples/ci-debug-trace +++ export CI_PROJECT_NAMESPACE=gitlab-examples +++ CI_PROJECT_NAMESPACE=gitlab-examples +++ export CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace +++ CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace +++ export CI_PIPELINE_ID=52666 +++ CI_PIPELINE_ID=52666 +++ export CI_RUNNER_ID=1337 +++ CI_RUNNER_ID=1337 +++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com +++ CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com +++ export 'CI_RUNNER_TAGS=shared, docker, linux, ruby, mysql, postgres, mongo, git-annex' +++ CI_RUNNER_TAGS='shared, docker, linux, ruby, mysql, postgres, mongo, git-annex' +++ export CI_REGISTRY=registry.example.com +++ CI_REGISTRY=registry.example.com +++ export CI_DEBUG_TRACE=true +++ CI_DEBUG_TRACE=true +++ export GITLAB_USER_ID=42 +++ GITLAB_USER_ID=42 +++ export GITLAB_USER_EMAIL=user@example.com +++ GITLAB_USER_EMAIL=axilleas@axilleas.me +++ export VERY_SECURE_VARIABLE=imaverysecurevariable +++ VERY_SECURE_VARIABLE=imaverysecurevariable +++ mkdir -p /builds/gitlab-examples/ci-debug-trace.tmp +++ echo -n '-----BEGIN CERTIFICATE----- +MIIFQzCCBCugAwIBAgIRAL/ElDjuf15xwja1ZnCocWAwDQYJKoZIhvcNAQELBQAw' + +... +``` -**The value of the variable can be shown in build log if explicitly asked to do so.** -If your project is public or internal you can make the builds private. +## Using the CI variables in your job scripts -Secure Variables can added by going to `Project > Variables > Add Variable`. +All variables are set as environment variables in the build environment, and +they are accessible with normal methods that are used to access such variables. +In most cases `bash` or `sh` is used to execute the build script. -They will be available for all subsequent builds. +To access the variables (predefined and user-defined) in a `bash`/`sh` environment, +prefix the variable name with the dollar sign (`$`): -### Use variables -The variables are set as environment variables in build environment and are accessible with normal methods that are used to access such variables. -In most cases the **bash** is used to execute build script. -To access variables (predefined and user-defined) in bash environment, prefix the variable name with `$`: ``` job_name: script: - echo $CI_BUILD_ID ``` -You can also list all environment variables with `export` command, -but be aware that this will also expose value of all **Secure Variables** in build log: +You can also list all environment variables with the `export` command, +but be aware that this will also expose the values of all the secret variables +you set, in the build log: + ``` job_name: script: - export ``` +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 +[runner]: https://docs.gitlab.com/runner/ [triggered]: ../triggers/README.md +[triggers]: ../triggers/README.md#pass-build-variables-to-a-trigger diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5c0e1c44e3f..75a0897eb15 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -406,14 +406,20 @@ except master. ### job variables It is possible to define build variables using a `variables` keyword on a job -level. It works basically the same way as its global-level equivalent but -allows you to define job-specific build variables. +level. It works basically the same way as its [global-level equivalent](#variables) +but allows you to define job-specific build variables. When the `variables` keyword is used on a job level, it overrides global YAML -build variables and predefined variables. +build variables and predefined variables. To turn off global defined variables +in your job, define an empty array: -Build variables priority is defined in -[variables documentation](../variables/README.md). +```yaml +job_name: + variables: [] +``` + +Build variables priority is defined in the +[variables documentation][variables]. ### tags @@ -541,6 +547,8 @@ same manual action multiple times. An example usage of manual actions is deployment to production. +Read more at the [environments documentation][env-manual]. + ### environment > Introduced in GitLab 8.9. @@ -552,28 +560,14 @@ An example usage of manual actions is deployment to production. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name can contain: - -- letters -- digits -- spaces -- `-` -- `_` -- `/` -- `$` -- `{` -- `}` - -Common names are `qa`, `staging`, and `production`, but you can use whatever -name works with your workflow. - In its simplest form, the `environment` keyword can be defined like: ``` deploy to production: stage: deploy script: git push production HEAD:master - environment: production + environment: + name: production ``` In the above example, the `deploy to production` job will be marked as doing a @@ -588,6 +582,21 @@ Before GitLab 8.11, the name of an environment could be defined as a string like `environment: production`. The recommended way now is to define it under the `name` keyword. +The `environment` name can contain: + +- letters +- digits +- spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` + +Common names are `qa`, `staging`, and `production`, but you can use whatever +name works with your workflow. + Instead of defining the name of the environment right after the `environment` keyword, it is also possible to define it as a separate value. For that, use the `name` keyword under `environment`: @@ -626,7 +635,12 @@ deploy to production: #### environment:on_stop -> [Introduced][ce-6669] in GitLab 8.13. +> +**Notes:** +- [Introduced][ce-6669] in GitLab 8.13. +- Starting with GitLab 8.14, when you have an environment that has a stop action + defined, GitLab will automatically trigger a stop action when the associated + branch is deleted. Closing (stoping) environments can be achieved with the `on_stop` keyword defined under `environment`. It declares a different job that runs in order to close @@ -676,6 +690,7 @@ The `stop_review_app` job is **required** to have the following keywords defined #### dynamic environments > [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. + `$CI_ENVIRONMENT_SLUG` was [introduced][ce-7983] in GitLab 8.15 `environment` can also represent a configuration hash with `name` and `url`. These parameters can use any of the defined [CI variables](#variables) @@ -688,15 +703,17 @@ deploy as review app: stage: deploy script: make deploy environment: - name: review-apps/$CI_BUILD_REF_NAME - url: https://$CI_BUILD_REF_NAME.review.example.com/ + name: review/$CI_BUILD_REF_NAME + url: https://$CI_ENVIRONMENT_SLUG.example.com/ ``` The `deploy as review app` job will be marked as deployment to dynamically -create the `review-apps/$CI_BUILD_REF_NAME` environment, which `$CI_BUILD_REF_NAME` -is an [environment variable][variables] set by the Runner. If for example the -`deploy as review app` job was run in a branch named `pow`, this environment -should be accessible under `https://pow.review.example.com/`. +create the `review/$CI_BUILD_REF_NAME` environment, where `$CI_BUILD_REF_NAME` +is an [environment variable][variables] set by the Runner. The +`$CI_ENVIRONMENT_SLUG` variable is based on the environment name, but suitable +for inclusion in URLs. In this case, if the `deploy as review app` job was run +in a branch named `pow`, this environment would be accessible with an URL like +`https://review-pow-aaaaaa.example.com/`. This of course implies that the underlying server which hosts the application is properly configured. @@ -745,6 +762,15 @@ artifacts: - binaries/ ``` +To disable artifact passing, define the job with empty [dependencies](#dependencies): + +```yaml +job: + stage: build + script: make build + dependencies: [] +``` + You may want to create artifacts only for tagged releases to avoid filling the build server storage with temporary build artifacts. @@ -1008,6 +1034,66 @@ variables: GIT_STRATEGY: none ``` +## Git Submodule Strategy + +> Requires GitLab Runner v1.10+. + +The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git +submodules are included when fetching the code before a build. Like +`GIT_STRATEGY`, it can be set in either the global [`variables`](#variables) +section or the [`variables`](#job-variables) section for individual jobs. + +There are three posible values: `none`, `normal`, and `recursive`: + +- `none` means that submodules will not be included when fetching the project + code. This is the default, which matches the pre-v1.10 behavior. + +- `normal` means that only the top-level submodules will be included. It is + equivalent to: + ``` + $ git submodule sync + $ git submodule update --init + ``` + +- `recursive` means that all submodules (including submodules of submodules) + will be included. It is equivalent to: + ``` + $ git submodule sync --recursive + $ git submodule update --init --recursive + ``` + +Note that for this feature to work correctly, the submodules must be configured +(in `.gitmodules`) with either: +- the HTTP(S) URL of a publicly-accessible repository, or +- a relative path to another repository on the same GitLab server. See the + [Git submodules](../git_submodules.md) documentation. + + +## Build stages attempts + +> Introduced in GitLab, it requires GitLab Runner v1.9+. + +You can set the number for attempts the running build will try to execute each +of the following stages: + +| Variable | Description | +|-------------------------|-------------| +| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a build | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a build | +| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a build | + +The default is one single attempt. + +Example: + +``` +variables: + GET_SOURCES_ATTEMPTS: "3" +``` + +You can set them in the global [`variables`](#variables) section or the [`variables`](#job-variables) +section for individual jobs. + ## Shallow cloning > Introduced in GitLab 8.9 as an experimental feature. May change in future @@ -1210,8 +1296,10 @@ capitalization, the commit will be created but the builds will be skipped. Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. +[env-manual]: ../environments.md#manually-deploying-to-environments [examples]: ../examples/README.md [ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 [environment]: ../environments.md [ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 [variables]: ../variables/README.md +[ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 |
