summaryrefslogtreecommitdiff
path: root/doc/ci/environments.md
blob: ef04c53736743427ae7391ad776b29ee0b5969ee (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
# Introduction to environments and deployments

>**Note:**
Introduced in GitLab 8.9.

During the development of software, there can be many stages until it's ready
for public consumption. You sure want to first test your code and then deploy it
in a testing or staging environment before you release it to the public. That
way you can prevent bugs not only in your software, but in the deployment
process as well.

GitLab CI is capable of not only testing or building your projects, but also
deploying them in your infrastructure, with the added benefit of giving you a
way to track your deployments. In other words, you can always know what is
currently being deployed or has been deployed on your servers.

## Overview

With environments, you can control the Continuous Deployment of your software
all within GitLab. All you need to do is define them in your project's
[`.gitlab-ci.yml`][yaml] as we will explore below. GitLab provides a full
history of your deployments per every environment.

Environments are like tags for your CI jobs, describing where code gets deployed.
Deployments are created when [jobs] deploy versions of code to environments,
so every environment can have one or more deployments. GitLab keeps track of
your deployments, so you always know what is currently being deployed on your
servers. If you have a deployment service such as [Kubernetes][kubernetes-service]
enabled for your project, you can use it to assist with your deployments, and
can even access a web terminal for your environment from within GitLab!

To better understand how environments and deployments work, let's consider an
example. We assume that you have already created a project in GitLab and set up
a Runner. The example will cover the following:

- We are developing an application
- We want to run tests and build our app on all branches
- Our default branch is `master`
- We deploy the app only when a pipeline on `master` branch is run

Let's see how it all ties together.

## Defining environments

Let's consider the following `.gitlab-ci.yml` example:

```yaml
stages:
  - test
  - build
  - deploy

test:
  stage: test
  script: echo "Running tests"

build:
  stage: build
  script: echo "Building the app"

deploy_staging:
  stage: deploy
  script:
    - echo "Deploy to staging server"
  environment:
    name: staging
    url: https://staging.example.com
  only:
  - master
```

We have defined 3 [stages](yaml/README.md#stages):

- test
- build
- deploy

The jobs assigned to these stages will run in this order. If a job fails, then
the builds that are assigned to the next stage won't run, rendering the pipeline
as failed. In our case, the `test` job will run first, then the `build` and
lastly the `deploy_staging`. With this, we ensure that first the tests pass,
then our app is able to be built successfully, and lastly we deploy to the
staging server.

The `environment` keyword is just a hint for GitLab that this job actually
deploys to this environment's `name`. It can also have a `url` which, as we
will later see, is exposed in various places within GitLab. Each time a job that
has an environment specified and succeeds, a deployment is recorded, remembering
the Git SHA and environment name.

>**Note:**
Starting with GitLab 8.15, the environment name is exposed to the Runner in
two forms: `$CI_ENVIRONMENT_NAME`, and `$CI_ENVIRONMENT_SLUG`. The first is
the name given in `.gitlab-ci.yml` (with any variables expanded), while the
second is a "cleaned-up" version of the name, suitable for use in URLs, DNS,
etc.

To sum up, with the above `.gitlab-ci.yml` we have achieved that:

- All branches will run the `test` and `build` jobs.
- The `deploy_staging` job will run [only](yaml/README.md#only) on the `master`
  branch which means all merge requests that are created from branches don't
  get to deploy to the staging server
- When a merge request is merged, all jobs will run and the `deploy_staging`
  in particular will deploy our code to a staging server while the deployment
  will be recorded in an environment named `staging`.

Let's now see how that information is exposed within GitLab.

## Viewing the current status of an environment

The environment list under your project's **Pipelines ➔ Environments**, is
where you can find information of the last deployment status of an environment.

Here's how the Environments page looks so far.

![Staging environment view](img/environments_available_staging.png)

There's a bunch of information there, specifically you can see:

- The environment's name with a link to its deployments
- The last deployment ID number and who performed it
- The build ID of the last deployment with its respective job name
- The commit information of the last deployment such as who committed, to what
  branch and the Git SHA of the commit
- The exact time the last deployment was performed
- A button that takes you to the URL that you have defined under the
  `environment` keyword in `.gitlab-ci.yml`
- A button that re-deploys the latest deployment, meaning it runs the job
  defined by the environment name for that specific commit

>**Notes:**
- While you can create environments manually in the web interface, we recommend
  that you define your environments in `.gitlab-ci.yml` first. They will
  be automatically created for you after the first deploy.
- The environments page can only be viewed by Reporters and above. For more
  information on the permissions, see the [permissions documentation][permissions].
- Only deploys that happen after your `.gitlab-ci.yml` is properly configured
  will show up in the "Environment" and "Last deployment" lists.

The information shown in the Environments page is limited to the latest
deployments, but as you may have guessed an environment can have multiple
deployments.

## Viewing the deployment history of an environment

GitLab keeps track of your deployments, so you always know what is currently
being deployed on your servers. That way you can have the full history of your
deployments per every environment right in your browser. Clicking on an
environment will show the history of its deployments. Assuming you have deployed
multiple times already, here's how a specific environment's page looks like.

![Deployments](img/deployments_view.png)

We can see the same information as when in the Environments page, but this time
all deployments are shown. As you may have noticed, apart from the **Re-deploy**
button there are now **Rollback** buttons for each deployment. Let's see how
that works.

## Rolling back changes

You can't control everything, so sometimes things go wrong. When that unfortunate
time comes GitLab has you covered. Simply by clicking the **Rollback** button
that can be found in the deployments page
(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the
job with the commit associated with it.

>**Note:**
Bear in mind that your mileage will vary and it's entirely up to how you define
the deployment process in the job's `script` whether the rollback succeeds or not.
GitLab CI is just following orders.

Thankfully that was the staging server that we had to rollback, and since we
learn from our mistakes, we decided to not make the same again when we deploy
to the production server. Enter manual actions for deployments.

## Manually deploying to environments

Turning a job from running automatically to a manual action is as simple as
adding `when: manual` to it. To expand on our previous example, let's add
another job that this time deploys our app to a production server and is
tracked by a `production` environment. The `.gitlab-ci.yml` looks like this
so far:

```yaml
stages:
  - test
  - build
  - deploy

test:
  stage: test
  script: echo "Running tests"

build:
  stage: build
  script: echo "Building the app"

deploy_staging:
  stage: deploy
  script:
    - echo "Deploy to staging server"
  environment:
    name: staging
    url: https://staging.example.com
  only:
  - master

deploy_prod:
  stage: deploy
  script:
    - echo "Deploy to production server"
  environment:
    name: production
    url: https://example.com
  when: manual
  only:
  - master
```

The `when: manual` action exposes a play button in GitLab's UI and the
`deploy_prod` job will only be triggered if and when we click that play button.
You can find it in the pipeline, build, environment, and deployment views.

| Pipelines | Single pipeline | Environments | Deployments | Builds |
| --------- | ----------------| ------------ | ----------- | -------|
| ![Pipelines manual action](img/environments_manual_action_pipelines.png) | ![Pipelines manual action](img/environments_manual_action_single_pipeline.png) | ![Environments manual action](img/environments_manual_action_environments.png) | ![Deployments manual action](img/environments_manual_action_deployments.png) | ![Builds manual action](img/environments_manual_action_builds.png) |

Clicking on the play button in either of these places will trigger the
`deploy_prod` job, and the deployment will be recorded under a new
environment named `production`.

>**Note:**
Remember that if your environment's name is `production` (all lowercase), then
it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md).
Double the benefit!

## Web terminals

>**Note:**
Web terminals were added in GitLab 8.15 and are only available to project
masters and owners.

If you deploy to your environments with the help of a deployment service (e.g.,
the [Kubernetes](../project_services/kubernetes.md) service), GitLab can open
a terminal session to your environment! This is a very powerful feature that
allows you to debug issues without leaving the comfort of your web browser. To
enable it, just follow the instructions given in the service documentation.

Once enabled, your environments will gain a "terminal" button:

![Terminal button on environment index](img/environments_terminal_button_on_index.png)

You can also access the terminal button from the page for a specific environment:

![Terminal button for an environment](img/environments_terminal_button_on_show.png)

Wherever you find it, clicking the button will take you to a separate page to
establish the terminal session:

![Terminal page](img/environments_terminal_page.png)

This works just like any other terminal - you'll be in the container created
by your deployment, so you can run shell commands and get responses in real
time, check the logs, try out configuration or code tweaks, etc. You can open
multiple terminals to the same environment - they each get their own shell
session -  and even a multiplexer like `screen` or `tmux`!  

>**Note:**
Container-based deployments often lack basic tools (like an editor), and may
be stopped or restarted at any time. If this happens, you will lose all your
changes! Treat this as a debugging tool, not a comprehensive online IDE. You
can use [Koding](../administration/integration/koding.md) for online
development.

---

While this is fine for deploying to some stable environments like staging or
production, what happens for branches? So far we haven't defined anything
regarding deployments for branches other than `master`. Dynamic environments
will help us achieve that.

## Dynamic environments

As the name suggests, it is possible to create environments on the fly by just
declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is
the basis of [Review apps](review_apps/index.md).

GitLab Runner exposes various [environment variables][variables] when a job runs,
and as such, you can use them as environment names. Let's add another job in
our example which will deploy to all branches except `master`:

```yaml
deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_BUILD_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.example.com
  only:
    - branches
  except:
    - master
```

Let's break it down in pieces. The job's name is `deploy_review` and it runs
on the `deploy` stage. The `script` at this point is fictional, you'd have to
use your own based on your deployment. Then, we set the `environment` with the
`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting
one. Since the [environment name][env-name] can contain slashes (`/`), we can
use this pattern to distinguish between dynamic environments and the regular
ones.

So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME`
which takes the value of the branch name. Since `$CI_BUILD_REF_NAME` itself may
also contain `/`, or other characters that would be invalid in a domain name or
URL, we use `$CI_ENVIRONMENT_SLUG` in the `environment:url` so that the
environment can get a specific and distinct URL for each branch. In this case,
given a `$CI_BUILD_REF_NAME` of `100-Do-The-Thing`, the URL will be something
like `https://100-do-the-4f99a2.example.com`. Again, the way you set up
the web server to serve these requests is based on your setup.

You could also use `$CI_BUILD_REF_SLUG` in `environment:url`, e.g.:
`https://$CI_BUILD_REF_SLUG.example.com`. We use `$CI_ENVIRONMENT_SLUG`
here because it is guaranteed to be unique, but if you're using a workflow like
[GitLab Flow][gitlab-flow], collisions are very unlikely, and you may prefer
environment names to be more closely based on the branch name - the example
above would give you an URL like `https://100-do-the-thing.example.com`

Last but not least, we tell the job to run [`only`][only] on branches
[`except`][only] master.

>**Note:**
You are not bound to use the same prefix or only slashes in the dynamic
environments' names (`/`), but as we will see later, this will enable the
[grouping similar environments](#grouping-similar-environments) feature.

The whole `.gitlab-ci.yml` looks like this so far:

```yaml
stages:
  - test
  - build
  - deploy

test:
  stage: test
  script: echo "Running tests"

build:
  stage: build
  script: echo "Building the app"

deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_BUILD_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.example.com
  only:
    - branches
  except:
    - master

deploy_staging:
  stage: deploy
  script:
    - echo "Deploy to staging server"
  environment:
    name: staging
    url: https://staging.example.com
  only:
  - master

deploy_prod:
  stage: deploy
  script:
    - echo "Deploy to production server"
  environment:
    name: production
    url: https://example.com
  when: manual
  only:
  - master
```

A more realistic example would include copying files to a location where a
webserver (NGINX) could then read and serve. The example below will copy the
`public` directory to `/srv/nginx/$CI_BUILD_REF_SLUG/public`:

```yaml
review_app:
  stage: deploy
  script:
    - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_SLUG
  environment:
    name: review/$CI_BUILD_REF_NAME
    url: https://$CI_BUILD_REF_SLUG.example.com
```

It is assumed that the user has already setup NGINX and GitLab Runner in the
server this job will run on.

>**Note:**
Be sure to check out the [limitations](#limitations) section for some edge
cases regarding naming of your branches and Review Apps.

---

The development workflow would now be:

- Developer creates a branch locally
- Developer makes changes, commits and pushes the branch to GitLab
- Developer creates a merge request

Behind the scenes:

- GitLab Runner picks up the changes and starts running the jobs
- The jobs run sequentially as defined in `stages`
  - First, the tests pass
  - Then, the build begins and successfully also passes
  - Lastly, the app is deployed to an environment with a name specific to the
    branch

So now, every branch gets its own environment and is deployed to its own place
with the added benefit of having a [history of deployments](#viewing-the-deployment-history-of-an-environment)
and also being able to [rollback changes](#rolling-back-changes) if needed.
Let's briefly see where URL that's defined in the environments is exposed.

## Making use of the environment URL

The environment URL is exposed in a few places within GitLab.

| In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button |
| -------------------- | ------------ | ----------- |
| ![Environment URL in merge request](img/environments_mr_review_app.png) | ![Environment URL in environments](img/environments_link_url.png) | ![Environment URL in deployments](img/environments_link_url_deployments.png) |

If a merge request is eventually merged to the default branch (in our case
`master`) and that branch also deploys to an environment (in our case `staging`
and/or `production`) you can see this information in the merge request itself.

![Environment URLs in merge request](img/environments_link_url_mr.png)

---

We now have a full development cycle, where our app is tested, built, deployed
as a Review app, deployed to a staging server once the merge request is merged,
and finally manually deployed to the production server. What we just described
is a single workflow, but imagine tens of developers working on a project
at the same time. They each push to their branches, and dynamic environments are
created all the time. In that case, we probably need to do some clean up. Read
next how environments can be stopped.

## Stopping an environment

By stopping an environment, you are effectively terminating its recording of the
deployments that happen in it.

A branch is associated with an environment when the CI pipeline that is created
for this branch, was recently deployed to this environment. You can think of
the CI pipeline as the glue between the branch and the environment:
`branch ➔ CI pipeline ➔ environment`.

There is a special case where environments can be manually stopped. That can
happen if you provide another job for that matter. The syntax is a little
tricky since a job calls another job to do the job.

Consider the following example where the `deploy_review` calls the `stop_review`
to clean up and stop the environment:

```yaml
deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_BUILD_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.example.com
    on_stop: stop_review
  only:
    - branches
  except:
    - master

stop_review:
  variables:
    GIT_STRATEGY: none
  script:
    - echo "Remove review app"
  when: manual
  environment:
    name: review/$CI_BUILD_REF_NAME
    action: stop
```

Setting the [`GIT_STRATEGY`][git-strategy] to `none` is necessary on the
`stop_review` job so that the [GitLab Runner] won't try to checkout the code
after the branch is deleted.

>**Note:**
Starting with GitLab 8.14, dynamic environments will be stopped automatically
when their associated branch is deleted.

When you have an environment that has a stop action defined (typically when
the environment describes a review app), GitLab will automatically trigger a
stop action when the associated branch is deleted.

You can read more in the [`.gitlab-ci.yml` reference][onstop].

## Grouping similar environments

> [Introduced][ce-7015] in GitLab 8.14.

As we've seen in the [dynamic environments](#dynamic-environments), you can
prepend their name with a word, then followed by a `/` and finally the branch
name which is automatically defined by the `CI_BUILD_REF_NAME` variable.

In short, environments that are named like `type/foo` are presented under a
group named `type`.

In our minimal example, we name the environments `review/$CI_BUILD_REF_NAME`
where `$CI_BUILD_REF_NAME` is the branch name:

```yaml
deploy_review:
  stage: deploy
  script:
    - echo "Deploy a review app"
  environment:
    name: review/$CI_BUILD_REF_NAME
```

In that case, if you visit the Environments page, and provided the branches
exist, you should see something like:

![Environment groups](img/environments_dynamic_groups.png)

## Checkout deployments locally

Since 8.13, a reference in the git repository is saved for each deployment, so
knowing the state of your current environments is only a `git fetch` away.

In your git config, append the `[remote "<your-remote>"]` block with an extra
fetch line:

```
fetch = +refs/environments/*:refs/remotes/origin/environments/*
```

## Limitations

1. You are limited to use only the [CI predefined variables][variables] in the
   `environment: name`. If you try to re-use variables defined inside `script`
   as part of the environment name, it will not work.

## Further reading

Below are some links you may find interesting:

- [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment)
- [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/)
- [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md)

[Pipelines]: pipelines.md
[jobs]: yaml/README.md#jobs
[yaml]: yaml/README.md
[kubernetes-service]: ../project_services/kubernetes.md]
[environments]: #environments
[deployments]: #deployments
[permissions]: ../user/permissions.md
[variables]: variables/README.md
[env-name]: yaml/README.md#environment-name
[only]: yaml/README.md#only-and-except
[onstop]: yaml/README.md#environment-on_stop
[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015
[gitlab-flow]: ../workflow/gitlab_flow.md
[gitlab runner]: https://docs.gitlab.com/runner/
[git-strategy]: yaml/README.md#git-strategy