diff options
author | Rémy Coutable <remy@rymai.me> | 2016-06-15 13:48:09 +0000 |
---|---|---|
committer | Rémy Coutable <remy@rymai.me> | 2016-06-15 13:48:09 +0000 |
commit | be098459147f0cd4323db7e0cca44160b5236ff4 (patch) | |
tree | d6d25003eb5ffda365191a958be85be5f681fbf3 | |
parent | a4a85c269a4f3165914915ede4a9a7642057d105 (diff) | |
parent | 6ace6d940a90e70f89392c3be7d9e538b6cec04c (diff) | |
download | gitlab-ce-be098459147f0cd4323db7e0cca44160b5236ff4.tar.gz |
Merge branch 'environments-and-deployments' into 'master'
Add environments and deployments
This MR is a continuation of https://gitlab.com/gitlab-org/gitlab-ce/issues/17009.
The current implementation is as follow:
1. We have two new tables: `environments` and `deployments`.
2. We have a new tab: `Environments` under `Pipelines` where you can see all you environments and add a new one.
3. We add a new option to `.gitlab-ci.yml` to track where we should create a deployment for environment.
4. If environment in `.gitlab-ci.yml` is specified it will create a deployment. **If environment does not exist it will be created.** (this got changed)
5. The deployment is always successful and shows the time of the action, in that case a build that presumably should do deployment. In the future we could extend deployment with statuses: success, failure. We could extend deployments with information that this is partial or full deployment.
6. User have to create environments that he will track first.
7. User can remove environments.
8. User can retry/rollback past deployment (in that case we retry past build). The new build when succeeds it will create a new deployment.
9. Currently environment have only one parameter: `name`. In the future it should have: `variables`, `credentials` and possibly `runners` and maybe other resources.
10. Currently deployment have this parameters: `sha`, `ref`, `deployable (in this case a build)`, `user (who triggered a deployment)`, `created_at`.
The `.gitlab-ci.yml`:
```
deploy to production:
stage: deploy
script: dpl travis...
environment: production
```
What needs to be done:
- [x] Write initial implementation
- [x] Improve implementation (@ayufan)
- [x] Write tests (@ayufan)
- [x] Improve UX of the forms (cc @markpundsack) - reviewed by @markpundsack
- [x] Improve implementation of the views (cc @jschatz1) - done by @iamphill
- [x] Write .gitlab-ci.yml documentation for `environments` - done by @ayufan
- [ ] Write user documentation (@ayufan and @markpundsack)
See merge request !4605
46 files changed, 880 insertions, 40 deletions
diff --git a/CHANGELOG b/CHANGELOG index 6f29b578a95..eb583af9f95 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -27,6 +27,7 @@ v 8.9.0 (unreleased) - Add a metric for the number of new Redis connections created by a transaction - Redesign navigation for project pages - Fix groups API to list only user's accessible projects + - Add Environments and Deployments - Redesign account and email confirmation emails - Don't fail builds for projects that are deleted - `git clone https://host/namespace/project` now works, in addition to using the `.git` suffix diff --git a/app/assets/stylesheets/pages/environments.scss b/app/assets/stylesheets/pages/environments.scss new file mode 100644 index 00000000000..e160d676e35 --- /dev/null +++ b/app/assets/stylesheets/pages/environments.scss @@ -0,0 +1,5 @@ +.environments { + .commit-title { + margin: 0; + } +} diff --git a/app/controllers/projects/builds_controller.rb b/app/controllers/projects/builds_controller.rb index 14c82826342..ef3051d7519 100644 --- a/app/controllers/projects/builds_controller.rb +++ b/app/controllers/projects/builds_controller.rb @@ -51,7 +51,7 @@ class Projects::BuildsController < Projects::ApplicationController return render_404 end - build = Ci::Build.retry(@build) + build = Ci::Build.retry(@build, current_user) redirect_to build_path(build) end diff --git a/app/controllers/projects/commit_controller.rb b/app/controllers/projects/commit_controller.rb index 20637fa46fe..6751737d15e 100644 --- a/app/controllers/projects/commit_controller.rb +++ b/app/controllers/projects/commit_controller.rb @@ -46,7 +46,7 @@ class Projects::CommitController < Projects::ApplicationController def retry_builds ci_builds.latest.failed.each do |build| if build.retryable? - Ci::Build.retry(build) + Ci::Build.retry(build, current_user) end end diff --git a/app/controllers/projects/environments_controller.rb b/app/controllers/projects/environments_controller.rb new file mode 100644 index 00000000000..4b433796161 --- /dev/null +++ b/app/controllers/projects/environments_controller.rb @@ -0,0 +1,49 @@ +class Projects::EnvironmentsController < Projects::ApplicationController + layout 'project' + before_action :authorize_read_environment! + before_action :authorize_create_environment!, only: [:new, :create] + before_action :authorize_update_environment!, only: [:destroy] + before_action :environment, only: [:show, :destroy] + + def index + @environments = project.environments + end + + def show + @deployments = environment.deployments.order(id: :desc).page(params[:page]) + end + + def new + @environment = project.environments.new + end + + def create + @environment = project.environments.create(create_params) + + if @environment.persisted? + redirect_to namespace_project_environment_path(project.namespace, project, @environment) + else + render 'new' + end + end + + def destroy + if @environment.destroy + flash[:notice] = 'Environment was successfully removed.' + else + flash[:alert] = 'Failed to remove environment.' + end + + redirect_to namespace_project_environments_path(project.namespace, project) + end + + private + + def create_params + params.require(:environment).permit(:name) + end + + def environment + @environment ||= project.environments.find(params[:id]) + end +end diff --git a/app/controllers/projects/pipelines_controller.rb b/app/controllers/projects/pipelines_controller.rb index cac440ae53e..127bd1a4318 100644 --- a/app/controllers/projects/pipelines_controller.rb +++ b/app/controllers/projects/pipelines_controller.rb @@ -32,7 +32,7 @@ class Projects::PipelinesController < Projects::ApplicationController end def retry - pipeline.retry_failed + pipeline.retry_failed(current_user) redirect_back_or_default default: namespace_project_pipelines_path(project.namespace, project) end diff --git a/app/helpers/gitlab_routing_helper.rb b/app/helpers/gitlab_routing_helper.rb index 3a43e936aee..5386ddadc62 100644 --- a/app/helpers/gitlab_routing_helper.rb +++ b/app/helpers/gitlab_routing_helper.rb @@ -42,6 +42,10 @@ module GitlabRoutingHelper namespace_project_pipelines_path(project.namespace, project, *args) end + def project_environments_path(project, *args) + namespace_project_environments_path(project.namespace, project, *args) + end + def project_builds_path(project, *args) namespace_project_builds_path(project.namespace, project, *args) end diff --git a/app/helpers/projects_helper.rb b/app/helpers/projects_helper.rb index d30dd66202b..3b4e431a491 100644 --- a/app/helpers/projects_helper.rb +++ b/app/helpers/projects_helper.rb @@ -140,6 +140,10 @@ module ProjectsHelper nav_tabs << :container_registry end + if can?(current_user, :read_environment, project) + nav_tabs << :environments + end + if can?(current_user, :admin_project, project) nav_tabs << :settings end diff --git a/app/models/ability.rb b/app/models/ability.rb index 647a73aa1ce..9c58b956007 100644 --- a/app/models/ability.rb +++ b/app/models/ability.rb @@ -9,7 +9,6 @@ class Ability when CommitStatus then commit_status_abilities(user, subject) when Project then project_abilities(user, subject) when Issue then issue_abilities(user, subject) - when ExternalIssue then external_issue_abilities(user, subject) when Note then note_abilities(user, subject) when ProjectSnippet then project_snippet_abilities(user, subject) when PersonalSnippet then personal_snippet_abilities(user, subject) @@ -19,6 +18,7 @@ class Ability when GroupMember then group_member_abilities(user, subject) when ProjectMember then project_member_abilities(user, subject) when User then user_abilities + when ExternalIssue, Deployment, Environment then project_abilities(user, subject.project) else [] end.concat(global_abilities(user)) end @@ -230,6 +230,8 @@ class Ability :read_build, :read_container_image, :read_pipeline, + :read_environment, + :read_deployment ] end @@ -248,6 +250,8 @@ class Ability :push_code, :create_container_image, :update_container_image, + :create_environment, + :create_deployment ] end @@ -265,6 +269,8 @@ class Ability @project_master_rules ||= project_dev_rules + [ :push_code_to_protected_branches, :update_project_snippet, + :update_environment, + :update_deployment, :admin_milestone, :admin_project_snippet, :admin_project_member, @@ -275,7 +281,9 @@ class Ability :admin_commit_status, :admin_build, :admin_container_image, - :admin_pipeline + :admin_pipeline, + :admin_environment, + :admin_deployment ] end @@ -319,6 +327,8 @@ class Ability unless project.builds_enabled rules += named_abilities('build') rules += named_abilities('pipeline') + rules += named_abilities('environment') + rules += named_abilities('deployment') end unless project.container_registry_enabled @@ -513,10 +523,6 @@ class Ability end end - def external_issue_abilities(user, subject) - project_abilities(user, subject.project) - end - private def restricted_public_level? diff --git a/app/models/ci/build.rb b/app/models/ci/build.rb index 89a1f8b3f57..764d8e4e136 100644 --- a/app/models/ci/build.rb +++ b/app/models/ci/build.rb @@ -40,7 +40,7 @@ module Ci new_build.save end - def retry(build) + def retry(build, user = nil) new_build = Ci::Build.new(status: 'pending') new_build.ref = build.ref new_build.tag = build.tag @@ -54,6 +54,7 @@ module Ci new_build.stage = build.stage new_build.stage_idx = build.stage_idx new_build.trigger_request = build.trigger_request + new_build.user = user new_build.save MergeRequests::AddTodoWhenBuildFailsService.new(build.project, nil).close(new_build) new_build @@ -75,6 +76,17 @@ module Ci build.update_coverage build.execute_hooks end + + after_transition any => [:success] do |build| + if build.environment.present? + service = CreateDeploymentService.new(build.project, build.user, + environment: build.environment, + sha: build.sha, + ref: build.ref, + tag: build.tag) + service.execute(build) + end + end end def retryable? @@ -85,10 +97,6 @@ module Ci !self.pipeline.statuses.latest.include?(self) end - def retry - Ci::Build.retry(self) - end - def depends_on_builds # Get builds of the same type latest_builds = self.pipeline.builds.latest diff --git a/app/models/ci/pipeline.rb b/app/models/ci/pipeline.rb index 9b5b46f4928..4bbfb4cc806 100644 --- a/app/models/ci/pipeline.rb +++ b/app/models/ci/pipeline.rb @@ -76,8 +76,10 @@ module Ci builds.running_or_pending.each(&:cancel) end - def retry_failed - builds.latest.failed.select(&:retryable?).each(&:retry) + def retry_failed(user) + builds.latest.failed.select(&:retryable?).each do |build| + Ci::Build.retry(build, user) + end end def latest? @@ -161,6 +163,10 @@ module Ci git_commit_message =~ /(\[ci skip\])/ if git_commit_message end + def environments + builds.where.not(environment: nil).success.pluck(:environment).uniq + end + private def update_state diff --git a/app/models/deployment.rb b/app/models/deployment.rb new file mode 100644 index 00000000000..e498ca96e3c --- /dev/null +++ b/app/models/deployment.rb @@ -0,0 +1,29 @@ +class Deployment < ActiveRecord::Base + include InternalId + + belongs_to :project, required: true, validate: true + belongs_to :environment, required: true, validate: true + belongs_to :user + belongs_to :deployable, polymorphic: true + + validates :sha, presence: true + validates :ref, presence: true + + delegate :name, to: :environment, prefix: true + + def commit + project.commit(sha) + end + + def commit_title + commit.try(:title) + end + + def short_sha + Commit.truncate_sha(sha) + end + + def last? + self == environment.last_deployment + end +end diff --git a/app/models/environment.rb b/app/models/environment.rb new file mode 100644 index 00000000000..ac3a571a1f3 --- /dev/null +++ b/app/models/environment.rb @@ -0,0 +1,16 @@ +class Environment < ActiveRecord::Base + belongs_to :project, required: true, validate: true + + has_many :deployments + + validates :name, + presence: true, + uniqueness: { scope: :project_id }, + length: { within: 0..255 }, + format: { with: Gitlab::Regex.environment_name_regex, + message: Gitlab::Regex.environment_name_regex_message } + + def last_deployment + deployments.last + end +end diff --git a/app/models/project.rb b/app/models/project.rb index 0d2e612436a..fdbc84474ed 100644 --- a/app/models/project.rb +++ b/app/models/project.rb @@ -127,6 +127,8 @@ class Project < ActiveRecord::Base has_many :runners, through: :runner_projects, source: :runner, class_name: 'Ci::Runner' has_many :variables, dependent: :destroy, class_name: 'Ci::Variable', foreign_key: :gl_project_id has_many :triggers, dependent: :destroy, class_name: 'Ci::Trigger', foreign_key: :gl_project_id + has_many :environments, dependent: :destroy + has_many :deployments, dependent: :destroy accepts_nested_attributes_for :variables, allow_destroy: true diff --git a/app/services/ci/create_builds_service.rb b/app/services/ci/create_builds_service.rb index 64bcdac5c65..3a74ae094e8 100644 --- a/app/services/ci/create_builds_service.rb +++ b/app/services/ci/create_builds_service.rb @@ -29,7 +29,8 @@ module Ci :options, :allow_failure, :stage, - :stage_idx) + :stage_idx, + :environment) build_attrs.merge!(ref: @pipeline.ref, tag: @pipeline.tag, diff --git a/app/services/create_deployment_service.rb b/app/services/create_deployment_service.rb new file mode 100644 index 00000000000..efeb9df9527 --- /dev/null +++ b/app/services/create_deployment_service.rb @@ -0,0 +1,18 @@ +require_relative 'base_service' + +class CreateDeploymentService < BaseService + def execute(deployable = nil) + environment = project.environments.find_or_create_by( + name: params[:environment] + ) + + project.deployments.create( + environment: environment, + ref: params[:ref], + tag: params[:tag], + sha: params[:sha], + user: current_user, + deployable: deployable + ) + end +end diff --git a/app/views/layouts/nav/_project.html.haml b/app/views/layouts/nav/_project.html.haml index 718acb424b2..a851cae4b56 100644 --- a/app/views/layouts/nav/_project.html.haml +++ b/app/views/layouts/nav/_project.html.haml @@ -42,7 +42,7 @@ Code - if project_nav_tab? :pipelines - = nav_link(controller: :pipelines) do + = nav_link(controller: [:pipelines, :builds, :environments]) do = link_to project_pipelines_path(@project), title: 'Pipelines', class: 'shortcuts-pipelines' do %span Pipelines diff --git a/app/views/projects/deployments/_commit.html.haml b/app/views/projects/deployments/_commit.html.haml new file mode 100644 index 00000000000..0f9d9512d88 --- /dev/null +++ b/app/views/projects/deployments/_commit.html.haml @@ -0,0 +1,12 @@ +%div.branch-commit + - if deployment.ref + = link_to deployment.ref, namespace_project_commits_path(@project.namespace, @project, deployment.ref), class: "monospace" + · + = link_to deployment.short_sha, namespace_project_commit_path(@project.namespace, @project, deployment.sha), class: "commit-id monospace" + + %p.commit-title + %span + - if commit_title = deployment.commit_title + = link_to_gfm commit_title, namespace_project_commit_path(@project.namespace, @project, deployment.sha), class: "commit-row-message" + - else + Cant find HEAD commit for this branch diff --git a/app/views/projects/deployments/_deployment.html.haml b/app/views/projects/deployments/_deployment.html.haml new file mode 100644 index 00000000000..d08dd92f1f6 --- /dev/null +++ b/app/views/projects/deployments/_deployment.html.haml @@ -0,0 +1,23 @@ +%tr.deployment + %td + %strong= "##{deployment.iid}" + + %td + = render 'projects/deployments/commit', deployment: deployment + + %td + - if deployment.deployable + = link_to namespace_project_build_path(@project.namespace, @project, deployment.deployable) do + = "#{deployment.deployable.name} (##{deployment.deployable.id})" + + %td + #{time_ago_with_tooltip(deployment.created_at)} + + %td + - if can?(current_user, :create_deployment, deployment) && deployment.deployable + .pull-right + = link_to retry_namespace_project_build_path(@project.namespace, @project, deployment.deployable), method: :post, class: 'btn btn-build' do + - if deployment.last? + Retry + - else + Rollback diff --git a/app/views/projects/environments/_environment.html.haml b/app/views/projects/environments/_environment.html.haml new file mode 100644 index 00000000000..eafa246d05f --- /dev/null +++ b/app/views/projects/environments/_environment.html.haml @@ -0,0 +1,17 @@ +- last_deployment = environment.last_deployment + +%tr.environment + %td + %strong + = link_to environment.name, namespace_project_environment_path(@project.namespace, @project, environment) + + %td + - if last_deployment + = render 'projects/deployments/commit', deployment: last_deployment + - else + %p.commit-title + No deployments yet + + %td + - if last_deployment + #{time_ago_with_tooltip(last_deployment.created_at)} diff --git a/app/views/projects/environments/_form.html.haml b/app/views/projects/environments/_form.html.haml new file mode 100644 index 00000000000..c07f4bd510c --- /dev/null +++ b/app/views/projects/environments/_form.html.haml @@ -0,0 +1,7 @@ += form_for @environment, url: namespace_project_environments_path(@project.namespace, @project), html: { class: 'col-lg-9' } do |f| + = form_errors(@environment) + .form-group + = f.label :name, 'Name', class: 'label-light' + = f.text_field :name, required: true, class: 'form-control' + = f.submit 'Create environment', class: 'btn btn-create' + = link_to 'Cancel', namespace_project_environments_path(@project.namespace, @project), class: 'btn btn-cancel' diff --git a/app/views/projects/environments/_header_title.html.haml b/app/views/projects/environments/_header_title.html.haml new file mode 100644 index 00000000000..e056fccad5d --- /dev/null +++ b/app/views/projects/environments/_header_title.html.haml @@ -0,0 +1 @@ +- header_title project_title(@project, "Environments", project_environments_path(@project)) diff --git a/app/views/projects/environments/index.html.haml b/app/views/projects/environments/index.html.haml new file mode 100644 index 00000000000..ae9e77e7d89 --- /dev/null +++ b/app/views/projects/environments/index.html.haml @@ -0,0 +1,23 @@ +- @no_container = true +- page_title "Environments" += render "projects/pipelines/head" + +%div{ class: (container_class) } + - if can?(current_user, :create_environment, @project) + .top-area + .nav-controls + = link_to new_namespace_project_environment_path(@project.namespace, @project), class: 'btn btn-create' do + New environment + + - if @environments.blank? + %ul.content-list.environments + %li.nothing-here-block + No environments to show + - else + .table-holder + %table.table.environments + %tbody + %th Environment + %th Last deployment + %th Date + = render @environments diff --git a/app/views/projects/environments/new.html.haml b/app/views/projects/environments/new.html.haml new file mode 100644 index 00000000000..54465828ba9 --- /dev/null +++ b/app/views/projects/environments/new.html.haml @@ -0,0 +1,9 @@ +- page_title 'New Environment' + +.row.prepend-top-default.append-bottom-default + .col-lg-3 + %h4.prepend-top-0 + New Environment + %p Environments allow you to track deployments of your application + + = render 'form' diff --git a/app/views/projects/environments/show.html.haml b/app/views/projects/environments/show.html.haml new file mode 100644 index 00000000000..069b77b5adf --- /dev/null +++ b/app/views/projects/environments/show.html.haml @@ -0,0 +1,33 @@ +- @no_container = true +- page_title "Environments" += render "projects/pipelines/head" + +%div{ class: (container_class) } + .top-area + .col-md-9 + %h3.page-title= @environment.name.titleize + + .col-md-3 + .nav-controls + - if can?(current_user, :update_environment, @environment) + = link_to 'Destroy', namespace_project_environment_path(@project.namespace, @project, @environment), data: { confirm: 'Are you sure you want to delete this environment?' }, class: 'btn btn-danger', method: :delete + + - if @deployments.blank? + %ul.content-list.environments + %li.nothing-here-block + No deployments for + %strong= @environment.name + - else + .table-holder + %table.table.environments + %thead + %tr + %th ID + %th Commit + %th Build + %th Date + %th + + = render @deployments + + = paginate @deployments, theme: 'gitlab' diff --git a/app/views/projects/pipelines/_head.html.haml b/app/views/projects/pipelines/_head.html.haml index d0ba0d27d7c..9356608f318 100644 --- a/app/views/projects/pipelines/_head.html.haml +++ b/app/views/projects/pipelines/_head.html.haml @@ -11,3 +11,9 @@ = link_to project_builds_path(@project), title: 'Builds', class: 'shortcuts-builds' do %span Builds + + - if project_nav_tab? :environments + = nav_link(controller: %w(environments)) do + = link_to project_environments_path(@project), title: 'Environments', class: 'shortcuts-environments' do + %span + Environments diff --git a/config/routes.rb b/config/routes.rb index fb634901712..d52cbb22428 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -709,6 +709,8 @@ Rails.application.routes.draw do end end + resources :environments, only: [:index, :show, :new, :create, :destroy] + resources :builds, only: [:index, :show], constraints: { id: /\d+/ } do collection do post :cancel_all diff --git a/db/migrate/20160610204157_add_deployments.rb b/db/migrate/20160610204157_add_deployments.rb new file mode 100644 index 00000000000..cb144ea8a6d --- /dev/null +++ b/db/migrate/20160610204157_add_deployments.rb @@ -0,0 +1,27 @@ +# See http://doc.gitlab.com/ce/development/migration_style_guide.html +# for more information on how to write migrations for GitLab. + +class AddDeployments < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + def change + create_table :deployments, force: true do |t| + t.integer :iid, null: false + t.integer :project_id, null: false + t.integer :environment_id, null: false + t.string :ref, null: false + t.boolean :tag, null: false + t.string :sha, null: false + t.integer :user_id + t.integer :deployable_id + t.string :deployable_type + t.datetime :created_at + t.datetime :updated_at + end + + add_index :deployments, :project_id + add_index :deployments, [:project_id, :iid], unique: true + add_index :deployments, [:project_id, :environment_id] + add_index :deployments, [:project_id, :environment_id, :iid] + end +end diff --git a/db/migrate/20160610204158_add_environments.rb b/db/migrate/20160610204158_add_environments.rb new file mode 100644 index 00000000000..e1c71d173c4 --- /dev/null +++ b/db/migrate/20160610204158_add_environments.rb @@ -0,0 +1,17 @@ +# See http://doc.gitlab.com/ce/development/migration_style_guide.html +# for more information on how to write migrations for GitLab. + +class AddEnvironments < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + def change + create_table :environments, force: true do |t| + t.integer :project_id, null: false + t.string :name, null: false + t.datetime :created_at + t.datetime :updated_at + end + + add_index :environments, [:project_id, :name] + end +end diff --git a/db/migrate/20160610211845_add_environment_to_builds.rb b/db/migrate/20160610211845_add_environment_to_builds.rb new file mode 100644 index 00000000000..990e445ac55 --- /dev/null +++ b/db/migrate/20160610211845_add_environment_to_builds.rb @@ -0,0 +1,10 @@ +# See http://doc.gitlab.com/ce/development/migration_style_guide.html +# for more information on how to write migrations for GitLab. + +class AddEnvironmentToBuilds < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + def change + add_column :ci_builds, :environment, :string + end +end diff --git a/db/schema.rb b/db/schema.rb index 5fe39c5e59c..e148a3c975d 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -161,6 +161,7 @@ ActiveRecord::Schema.define(version: 20160610301627) do t.text "artifacts_metadata" t.integer "erased_by_id" t.datetime "erased_at" + t.string "environment" t.datetime "artifacts_expire_at" end @@ -382,6 +383,25 @@ ActiveRecord::Schema.define(version: 20160610301627) do add_index "deploy_keys_projects", ["project_id"], name: "index_deploy_keys_projects_on_project_id", using: :btree + create_table "deployments", force: :cascade do |t| + t.integer "iid", null: false + t.integer "project_id", null: false + t.integer "environment_id", null: false + t.string "ref", null: false + t.boolean "tag", null: false + t.string "sha", null: false + t.integer "user_id" + t.integer "deployable_id" + t.string "deployable_type" + t.datetime "created_at" + t.datetime "updated_at" + end + + add_index "deployments", ["project_id", "environment_id", "iid"], name: "index_deployments_on_project_id_and_environment_id_and_iid", using: :btree + add_index "deployments", ["project_id", "environment_id"], name: "index_deployments_on_project_id_and_environment_id", using: :btree + add_index "deployments", ["project_id", "iid"], name: "index_deployments_on_project_id_and_iid", unique: true, using: :btree + add_index "deployments", ["project_id"], name: "index_deployments_on_project_id", using: :btree + create_table "emails", force: :cascade do |t| t.integer "user_id", null: false t.string "email", null: false @@ -392,6 +412,15 @@ ActiveRecord::Schema.define(version: 20160610301627) do add_index "emails", ["email"], name: "index_emails_on_email", unique: true, using: :btree add_index "emails", ["user_id"], name: "index_emails_on_user_id", using: :btree + create_table "environments", force: :cascade do |t| + t.integer "project_id" + t.string "name", null: false + t.datetime "created_at" + t.datetime "updated_at" + end + + add_index "environments", ["project_id", "name"], name: "index_environments_on_project_id_and_name", using: :btree + create_table "events", force: :cascade do |t| t.string "target_type" t.integer "target_id" @@ -749,37 +778,37 @@ ActiveRecord::Schema.define(version: 20160610301627) do t.datetime "created_at" t.datetime "updated_at" t.integer "creator_id" - t.boolean "issues_enabled", default: true, null: false - t.boolean "merge_requests_enabled", default: true, null: false - t.boolean "wiki_enabled", default: true, null: false + t.boolean "issues_enabled", default: true, null: false + t.boolean "merge_requests_enabled", default: true, null: false + t.boolean "wiki_enabled", default: true, null: false t.integer "namespace_id" - t.boolean "snippets_enabled", default: true, null: false + t.boolean "snippets_enabled", default: true, null: false t.datetime "last_activity_at" t.string "import_url" - t.integer "visibility_level", default: 0, null: false - t.boolean "archived", default: false, null: false + t.integer "visibility_level", default: 0, null: false + t.boolean "archived", default: false, null: false t.string "avatar" t.string "import_status" t.float "repository_size", default: 0.0 - t.integer "star_count", default: 0, null: false + t.integer "star_count", default: 0, null: false t.string "import_type" t.string "import_source" t.integer "commit_count", default: 0 t.text "import_error" t.integer "ci_id" - t.boolean "builds_enabled", default: true, null: false - t.boolean "shared_runners_enabled", default: true, null: false + t.boolean "builds_enabled", default: true, null: false + t.boolean "shared_runners_enabled", default: true, null: false t.string "runners_token" t.string "build_coverage_regex" - t.boolean "build_allow_git_fetch", default: true, null: false - t.integer "build_timeout", default: 3600, null: false + t.boolean "build_allow_git_fetch", default: true, null: false + t.integer "build_timeout", default: 3600, null: false t.boolean "pending_delete", default: false - t.boolean "public_builds", default: true, null: false + t.boolean "public_builds", default: true, null: false t.integer "pushes_since_gc", default: 0 t.boolean "last_repository_check_failed" t.datetime "last_repository_check_at" t.boolean "container_registry_enabled" - t.boolean "only_allow_merge_if_build_succeeds", default: false, null: false + t.boolean "only_allow_merge_if_build_succeeds", default: false, null: false t.boolean "has_external_issue_tracker" end diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index d71ce6d6b13..9c98f9c98c6 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -28,6 +28,7 @@ If you want a quick introduction to GitLab CI, follow our - [only and except](#only-and-except) - [tags](#tags) - [when](#when) + - [environment](#environment) - [artifacts](#artifacts) - [artifacts:name](#artifacts-name) - [artifacts:when](#artifacts-when) @@ -354,6 +355,7 @@ job_name: | cache | no | Define list of files that should be cached between subsequent runs | | before_script | no | Override a set of commands that are executed before build | | after_script | no | Override a set of commands that are executed after build | +| environment | no | Defines a name of environment to which deployment is done by this build | ### script @@ -525,6 +527,31 @@ The above script will: 1. Execute `cleanup_build_job` only when `build_job` fails 2. Always execute `cleanup_job` as the last step in pipeline. +### environment + +>**Note:** +Introduced in GitLab v8.9.0. + +`environment` is used to define that job does deployment to specific environment. +This allows to easily track all deployments to your environments straight from GitLab. + +If `environment` is specified and no environment under that name does exist a new one will be created automatically. + +The `environment` name must contain only letters, digits, '-' and '_'. + +--- + +**Example configurations** + +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: production +``` + +The `deploy to production` job will be marked as doing deployment to `production` environment. + ### artifacts >**Notes:** diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index b76ce31cbad..963b35de3a0 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -28,6 +28,7 @@ documentation](../workflow/add-user/add-user.md). | Manage labels | | ✓ | ✓ | ✓ | ✓ | | See a commit status | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | +| See environments | | ✓ | ✓ | ✓ | ✓ | | Manage merge requests | | | ✓ | ✓ | ✓ | | Create new merge request | | | ✓ | ✓ | ✓ | | Create new branches | | | ✓ | ✓ | ✓ | @@ -40,6 +41,7 @@ documentation](../workflow/add-user/add-user.md). | Create or update commit status | | | ✓ | ✓ | ✓ | | Update a container registry | | | ✓ | ✓ | ✓ | | Remove a container registry image | | | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | | Create new milestones | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | | Push to protected branches | | | | ✓ | ✓ | @@ -52,6 +54,7 @@ documentation](../workflow/add-user/add-user.md). | Manage runners | | | | ✓ | ✓ | | Manage build triggers | | | | ✓ | ✓ | | Manage variables | | | | ✓ | ✓ | +| Delete environments | | | | ✓ | ✓ | | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | diff --git a/lib/api/builds.rb b/lib/api/builds.rb index 645e2dda0b7..979328efe0e 100644 --- a/lib/api/builds.rb +++ b/lib/api/builds.rb @@ -142,7 +142,7 @@ module API return not_found!(build) unless build return forbidden!('Build is not retryable') unless build.retryable? - build = Ci::Build.retry(build) + build = Ci::Build.retry(build, current_user) present build, with: Entities::Build, user_can_download_artifacts: can?(current_user, :read_build, user_project) diff --git a/lib/ci/gitlab_ci_yaml_processor.rb b/lib/ci/gitlab_ci_yaml_processor.rb index e0b89cead06..68246497e90 100644 --- a/lib/ci/gitlab_ci_yaml_processor.rb +++ b/lib/ci/gitlab_ci_yaml_processor.rb @@ -9,7 +9,8 @@ module Ci ALLOWED_YAML_KEYS = [:before_script, :after_script, :image, :services, :types, :stages, :variables, :cache] ALLOWED_JOB_KEYS = [:tags, :script, :only, :except, :type, :image, :services, :allow_failure, :type, :stage, :when, :artifacts, :cache, - :dependencies, :before_script, :after_script, :variables] + :dependencies, :before_script, :after_script, :variables, + :environment] ALLOWED_CACHE_KEYS = [:key, :untracked, :paths] ALLOWED_ARTIFACTS_KEYS = [:name, :untracked, :paths, :when, :expire_in] @@ -90,6 +91,7 @@ module Ci except: job[:except], allow_failure: job[:allow_failure] || false, when: job[:when] || 'on_success', + environment: job[:environment], options: { image: job[:image] || @image, services: job[:services] || @services, @@ -214,6 +216,10 @@ module Ci if job[:when] && !job[:when].in?(%w[on_success on_failure always]) raise ValidationError, "#{name} job: when parameter should be on_success, on_failure or always" end + + if job[:environment] && !validate_environment(job[:environment]) + raise ValidationError, "#{name} job: environment parameter #{Gitlab::Regex.environment_name_regex_message}" + end end def validate_job_script!(name, job) diff --git a/lib/gitlab/ci/config/node/validation_helpers.rb b/lib/gitlab/ci/config/node/validation_helpers.rb index 42ef60244ba..3900fc89391 100644 --- a/lib/gitlab/ci/config/node/validation_helpers.rb +++ b/lib/gitlab/ci/config/node/validation_helpers.rb @@ -24,6 +24,10 @@ module Gitlab value.is_a?(String) || value.is_a?(Symbol) end + def validate_environment(value) + value.is_a?(String) && value =~ Gitlab::Regex.environment_name_regex + end + def validate_boolean(value) value.in?([true, false]) end diff --git a/lib/gitlab/regex.rb b/lib/gitlab/regex.rb index 1cbd6d945a0..c84c68f96f6 100644 --- a/lib/gitlab/regex.rb +++ b/lib/gitlab/regex.rb @@ -100,5 +100,13 @@ module Gitlab def container_registry_reference_regex git_reference_regex end + + def environment_name_regex + @environment_name_regex ||= /\A[a-zA-Z0-9_-]+\z/.freeze + end + + def environment_name_regex_message + "can contain only letters, digits, '-' and '_'." + end end end diff --git a/spec/factories/deployments.rb b/spec/factories/deployments.rb new file mode 100644 index 00000000000..82591604fcb --- /dev/null +++ b/spec/factories/deployments.rb @@ -0,0 +1,13 @@ +FactoryGirl.define do + factory :deployment, class: Deployment do + sha '97de212e80737a608d939f648d959671fb0a0142' + ref 'master' + tag false + + environment factory: :environment + + after(:build) do |deployment, evaluator| + deployment.project = deployment.environment.project + end + end +end diff --git a/spec/factories/environments.rb b/spec/factories/environments.rb new file mode 100644 index 00000000000..07265c26ca3 --- /dev/null +++ b/spec/factories/environments.rb @@ -0,0 +1,7 @@ +FactoryGirl.define do + factory :environment, class: Environment do + sequence(:name) { |n| "environment#{n}" } + + project factory: :empty_project + end +end diff --git a/spec/features/environments_spec.rb b/spec/features/environments_spec.rb new file mode 100644 index 00000000000..40fea5211e9 --- /dev/null +++ b/spec/features/environments_spec.rb @@ -0,0 +1,160 @@ +require 'spec_helper' + +feature 'Environments', feature: true do + given(:project) { create(:empty_project) } + given(:user) { create(:user) } + given(:role) { :developer } + + background do + login_as(user) + project.team << [user, role] + end + + describe 'when showing environments' do + given!(:environment) { } + given!(:deployment) { } + + before do + visit namespace_project_environments_path(project.namespace, project) + end + + context 'without environments' do + scenario 'does show no environments' do + expect(page).to have_content('No environments to show') + end + end + + context 'with environments' do + given(:environment) { create(:environment, project: project) } + + scenario 'does show environment name' do + expect(page).to have_link(environment.name) + end + + context 'without deployments' do + scenario 'does show no deployments' do + expect(page).to have_content('No deployments yet') + end + end + + context 'with deployments' do + given(:deployment) { create(:deployment, environment: environment) } + + scenario 'does show deployment SHA' do + expect(page).to have_link(deployment.short_sha) + end + end + end + + scenario 'does have a New environment button' do + expect(page).to have_link('New environment') + end + end + + describe 'when showing the environment' do + given(:environment) { create(:environment, project: project) } + given!(:deployment) { } + + before do + visit namespace_project_environment_path(project.namespace, project, environment) + end + + context 'without deployments' do + scenario 'does show no deployments' do + expect(page).to have_content('No deployments for') + end + end + + context 'with deployments' do + given(:deployment) { create(:deployment, environment: environment) } + + scenario 'does show deployment SHA' do + expect(page).to have_link(deployment.short_sha) + end + + scenario 'does not show a retry button for deployment without build' do + expect(page).not_to have_link('Retry') + end + + context 'with build' do + given(:build) { create(:ci_build, project: project) } + given(:deployment) { create(:deployment, environment: environment, deployable: build) } + + scenario 'does show build name' do + expect(page).to have_link("#{build.name} (##{build.id})") + end + + scenario 'does show retry button' do + expect(page).to have_link('Retry') + end + end + end + end + + describe 'when creating a new environment' do + before do + visit namespace_project_environments_path(project.namespace, project) + end + + context 'when logged as developer' do + before do + click_link 'New environment' + end + + context 'for valid name' do + before do + fill_in('Name', with: 'production') + click_on 'Create environment' + end + + scenario 'does create a new pipeline' do + expect(page).to have_content('production') + end + end + + context 'for invalid name' do + before do + fill_in('Name', with: 'name with spaces') + click_on 'Create environment' + end + + scenario 'does show errors' do + expect(page).to have_content('Name can contain only letters') + end + end + end + + context 'when logged as reporter' do + given(:role) { :reporter } + + scenario 'does not have a New environment link' do + expect(page).not_to have_link('New environment') + end + end + end + + describe 'when deleting existing environment' do + given(:environment) { create(:environment, project: project) } + + before do + visit namespace_project_environment_path(project.namespace, project, environment) + end + + context 'when logged as master' do + given(:role) { :master } + + scenario 'does delete environment' do + click_link 'Destroy' + expect(page).not_to have_link(environment.name) + end + end + + context 'when logged as developer' do + given(:role) { :developer } + + scenario 'does not have a Destroy link' do + expect(page).not_to have_link('Destroy') + end + end + end +end diff --git a/spec/features/security/project/public_access_spec.rb b/spec/features/security/project/public_access_spec.rb index c5f741709ad..f6c6687e162 100644 --- a/spec/features/security/project/public_access_spec.rb +++ b/spec/features/security/project/public_access_spec.rb @@ -175,6 +175,49 @@ describe "Public Project Access", feature: true do end end + describe "GET /:project_path/environments" do + subject { namespace_project_environments_path(project.namespace, project) } + + it { is_expected.to be_allowed_for :admin } + it { is_expected.to be_allowed_for owner } + it { is_expected.to be_allowed_for master } + it { is_expected.to be_allowed_for developer } + it { is_expected.to be_allowed_for reporter } + it { is_expected.to be_denied_for guest } + it { is_expected.to be_denied_for :user } + it { is_expected.to be_denied_for :external } + it { is_expected.to be_denied_for :visitor } + end + + describe "GET /:project_path/environments/:id" do + let(:environment) { create(:environment, project: project) } + subject { namespace_project_environments_path(project.namespace, project, environment) } + + it { is_expected.to be_allowed_for :admin } + it { is_expected.to be_allowed_for owner } + it { is_expected.to be_allowed_for master } + it { is_expected.to be_allowed_for developer } + it { is_expected.to be_allowed_for reporter } + it { is_expected.to be_denied_for guest } + it { is_expected.to be_denied_for :user } + it { is_expected.to be_denied_for :external } + it { is_expected.to be_denied_for :visitor } + end + + describe "GET /:project_path/environments/new" do + subject { new_namespace_project_environment_path(project.namespace, project) } + + it { is_expected.to be_allowed_for :admin } + it { is_expected.to be_allowed_for owner } + it { is_expected.to be_allowed_for master } + it { is_expected.to be_allowed_for developer } + it { is_expected.to be_denied_for reporter } + it { is_expected.to be_denied_for guest } + it { is_expected.to be_denied_for :user } + it { is_expected.to be_denied_for :external } + it { is_expected.to be_denied_for :visitor } + end + describe "GET /:project_path/blob" do let(:commit) { project.repository.commit } diff --git a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb index 5e1d2b8e4f5..143e2e6d238 100644 --- a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb +++ b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb @@ -26,7 +26,8 @@ module Ci tag_list: [], options: {}, allow_failure: false, - when: "on_success" + when: "on_success", + environment: nil, }) end @@ -387,7 +388,8 @@ module Ci services: ["mysql"] }, allow_failure: false, - when: "on_success" + when: "on_success", + environment: nil, }) end @@ -415,7 +417,8 @@ module Ci services: ["postgresql"] }, allow_failure: false, - when: "on_success" + when: "on_success", + environment: nil, }) end end @@ -605,7 +608,8 @@ module Ci } }, when: "on_success", - allow_failure: false + allow_failure: false, + environment: nil, }) end @@ -627,6 +631,51 @@ module Ci end end + describe '#environment' do + let(:config) do + { + deploy_to_production: { stage: 'deploy', script: 'test', environment: environment } + } + end + + let(:processor) { GitlabCiYamlProcessor.new(YAML.dump(config)) } + let(:builds) { processor.builds_for_stage_and_ref('deploy', 'master') } + + context 'when a production environment is specified' do + let(:environment) { 'production' } + + it 'does return production' do + expect(builds.size).to eq(1) + expect(builds.first[:environment]).to eq(environment) + end + end + + context 'when no environment is specified' do + let(:environment) { nil } + + it 'does return nil environment' do + expect(builds.size).to eq(1) + expect(builds.first[:environment]).to be_nil + end + end + + context 'is not a string' do + let(:environment) { 1 } + + it 'raises error' do + expect { builds }.to raise_error("deploy_to_production job: environment parameter #{Gitlab::Regex.environment_name_regex_message}") + end + end + + context 'is not a valid string' do + let(:environment) { 'production staging' } + + it 'raises error' do + expect { builds }.to raise_error("deploy_to_production job: environment parameter #{Gitlab::Regex.environment_name_regex_message}") + end + end + end + describe "Dependencies" do let(:config) do { @@ -688,7 +737,8 @@ module Ci tag_list: [], options: {}, when: "on_success", - allow_failure: false + allow_failure: false, + environment: nil, }) end end @@ -733,7 +783,8 @@ module Ci tag_list: [], options: {}, when: "on_success", - allow_failure: false + allow_failure: false, + environment: nil, }) expect(subject.second).to eq({ except: nil, @@ -745,7 +796,8 @@ module Ci tag_list: [], options: {}, when: "on_success", - allow_failure: false + allow_failure: false, + environment: nil, }) end end diff --git a/spec/models/deployment_spec.rb b/spec/models/deployment_spec.rb new file mode 100644 index 00000000000..b273018707f --- /dev/null +++ b/spec/models/deployment_spec.rb @@ -0,0 +1,17 @@ +require 'spec_helper' + +describe Deployment, models: true do + subject { build(:deployment) } + + it { is_expected.to belong_to(:project) } + it { is_expected.to belong_to(:environment) } + it { is_expected.to belong_to(:user) } + it { is_expected.to belong_to(:deployable) } + + it { is_expected.to delegate_method(:name).to(:environment).with_prefix } + it { is_expected.to delegate_method(:commit).to(:project) } + it { is_expected.to delegate_method(:commit_title).to(:commit).as(:try) } + + it { is_expected.to validate_presence_of(:ref) } + it { is_expected.to validate_presence_of(:sha) } +end diff --git a/spec/models/environment_spec.rb b/spec/models/environment_spec.rb new file mode 100644 index 00000000000..7629af6a570 --- /dev/null +++ b/spec/models/environment_spec.rb @@ -0,0 +1,14 @@ +require 'spec_helper' + +describe Environment, models: true do + let(:environment) { create(:environment) } + + it { is_expected.to belong_to(:project) } + it { is_expected.to have_many(:deployments) } + + it { is_expected.to delegate_method(:last_deployment).to(:deployments).as(:last) } + + it { is_expected.to validate_presence_of(:name) } + it { is_expected.to validate_uniqueness_of(:name).scoped_to(:project_id) } + it { is_expected.to validate_length_of(:name).is_within(0..255) } +end diff --git a/spec/models/project_spec.rb b/spec/models/project_spec.rb index 30aa2b70c8d..fedab1f913b 100644 --- a/spec/models/project_spec.rb +++ b/spec/models/project_spec.rb @@ -28,6 +28,8 @@ describe Project, models: true do it { is_expected.to have_many(:runners) } it { is_expected.to have_many(:variables) } it { is_expected.to have_many(:triggers) } + it { is_expected.to have_many(:environments).dependent(:destroy) } + it { is_expected.to have_many(:deployments).dependent(:destroy) } it { is_expected.to have_many(:todos).dependent(:destroy) } end diff --git a/spec/services/create_deployment_service_spec.rb b/spec/services/create_deployment_service_spec.rb new file mode 100644 index 00000000000..654e441f3cd --- /dev/null +++ b/spec/services/create_deployment_service_spec.rb @@ -0,0 +1,119 @@ +require 'spec_helper' + +describe CreateDeploymentService, services: true do + let(:project) { create(:empty_project) } + let(:user) { create(:user) } + + let(:service) { described_class.new(project, user, params) } + + describe '#execute' do + let(:params) do + { environment: 'production', + ref: 'master', + tag: false, + sha: '97de212e80737a608d939f648d959671fb0a0142', + } + end + + subject { service.execute } + + context 'when no environments exist' do + it 'does create a new environment' do + expect { subject }.to change { Environment.count }.by(1) + end + + it 'does create a deployment' do + expect(subject).to be_persisted + end + end + + context 'when environment exist' do + before { create(:environment, project: project, name: 'production') } + + it 'does not create a new environment' do + expect { subject }.not_to change { Environment.count } + end + + it 'does create a deployment' do + expect(subject).to be_persisted + end + end + + context 'for environment with invalid name' do + let(:params) do + { environment: 'name with spaces', + ref: 'master', + tag: false, + sha: '97de212e80737a608d939f648d959671fb0a0142', + } + end + + it 'does not create a new environment' do + expect { subject }.not_to change { Environment.count } + end + + it 'does not create a deployment' do + expect(subject).not_to be_persisted + end + end + end + + describe 'processing of builds' do + let(:environment) { nil } + + shared_examples 'does not create environment and deployment' do + it 'does not create a new environment' do + expect { subject }.not_to change { Environment.count } + end + + it 'does not create a new deployment' do + expect { subject }.not_to change { Deployment.count } + end + + it 'does not call a service' do + expect_any_instance_of(described_class).not_to receive(:execute) + subject + end + end + + shared_examples 'does create environment and deployment' do + it 'does create a new environment' do + expect { subject }.to change { Environment.count }.by(1) + end + + it 'does create a new deployment' do + expect { subject }.to change { Deployment.count }.by(1) + end + + it 'does call a service' do + expect_any_instance_of(described_class).to receive(:execute) + subject + end + end + + context 'without environment specified' do + let(:build) { create(:ci_build, project: project) } + + it_behaves_like 'does not create environment and deployment' do + subject { build.success } + end + end + + context 'when environment is specified' do + let(:pipeline) { create(:ci_pipeline, project: project) } + let(:build) { create(:ci_build, pipeline: pipeline, environment: 'production') } + + context 'when build succeeds' do + it_behaves_like 'does create environment and deployment' do + subject { build.success } + end + end + + context 'when build fails' do + it_behaves_like 'does not create environment and deployment' do + subject { build.drop } + end + end + end + end +end |