diff options
34 files changed, 1044 insertions, 278 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index b47b0891686..d6ddc580f7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,11 +1,12 @@ Please view this file on the master branch, on stable branches it's out of date. ## 8.14.0 (2016-11-22) - +- Fix Milestone dropdown not stay selected for `Upcoming` and `No Milestone` option !7117 - Backups do not fail anymore when using tar on annex and custom_hooks only. !5814 - Adds user project membership expired event to clarify why user was removed (Callum Dryden) - Trim leading and trailing whitespace on project_path (Linus Thiel) - Prevent award emoji via notes for issues/MRs authored by user (barthc) +- Adds support for the `token` attribute in project hooks API (Gauvain Pocentek) - Adds an optional path parameter to the Commits API to filter commits by path (Luis HGO) - Fix Markdown styling inside reference links (Jan Zdráhal) - Fix extra space on Build sidebar on Firefox !7060 @@ -17,6 +18,7 @@ Please view this file on the master branch, on stable branches it's out of date. - Add hover to trash icon in notes !7008 (blackst0ne) - Hides project activity tabs when features are disabled - Only show one error message for an invalid email !5905 (lycoperdon) +- Added guide describing how to upgrade PostgreSQL using Slony - Fix sidekiq stats in admin area (blackst0ne) - Added label description as tooltip to issue board list title - Created cycle analytics bundle JavaScript file @@ -38,6 +40,7 @@ Please view this file on the master branch, on stable branches it's out of date. - Optimize Event queries by removing default order - Remove duplicate links from sidebar - API: Fix project deploy keys 400 and 500 errors when adding an existing key. !6784 (Joshua Welsh) +- Add Rake task to create/repair GitLab Shell hooks symlinks !5634 - Add job for removal of unreferenced LFS objects from both the database and the filesystem (Frank Groeneveld) - Replace jquery.cookie plugin with js.cookie !7085 - Use MergeRequestsClosingIssues cache data on Issue#closed_by_merge_requests method @@ -49,6 +52,7 @@ Please view this file on the master branch, on stable branches it's out of date. - Shortened merge request modal to let clipboard button not overlap - In all filterable drop downs, put input field in focus only after load is complete (Ido @leibo) - Improve search query parameter naming in /admin/users !7115 (YarNayar) +- Fix table pagination to be responsive - Allow to search for user by secondary email address in the admin interface(/admin/users) !7115 (YarNayar) ## 8.13.3 @@ -73,51 +77,6 @@ Please view this file on the master branch, on stable branches it's out of date. - Fix and improve `Sortable.highest_label_priority`. !7165 - Fixed sticky merge request tabs when sidebar is pinned. !7167 - Only remove right connector of first build of last stage. !7179 - - Backups do not fail anymore when using tar on annex and custom_hooks only. !5814 - - Adds user project membership expired event to clarify why user was removed (Callum Dryden) - - Trim leading and trailing whitespace on project_path (Linus Thiel) - - Prevent award emoji via notes for issues/MRs authored by user (barthc) - - Adds an optional path parameter to the Commits API to filter commits by path (Luis HGO) - - Fix extra space on Build sidebar on Firefox !7060 - - Fix mobile layout issues in admin user overview page !7087 - - Fix HipChat notifications rendering (airatshigapov, eisnerd) - - Refactor Jira service to use jira-ruby gem - - Add hover to trash icon in notes !7008 (blackst0ne) - - Only show one error message for an invalid email !5905 (lycoperdon) - - Fix sidekiq stats in admin area (blackst0ne) - - Created cycle analytics bundle JavaScript file - - API: Fix booleans not recognized as such when using the `to_boolean` helper - - Removed delete branch tooltip !6954 - - Stop unauthorized users dragging on milestone page (blackst0ne) - - Restore issue boards welcome message when a project is created !6899 - - Escape ref and path for relative links !6050 (winniehell) - - Fixed link typo on /help/ui to Alerts section. !6915 (Sam Rose) - - Fix filtering of milestones with quotes in title (airatshigapov) - - Refactor less readable existance checking code from CoffeeScript !6289 (jlogandavison) - - Update mail_room and enable sentinel support to Reply By Email (!7101) - - Add task completion status in Issues and Merge Requests tabs: "X of Y tasks completed" (!6527, @gmesalazar) - - Added support for executing post deployment migrations - - Simpler arguments passed to named_route on toggle_award_url helper method - - Fix typo in framework css class. !7086 (Daniel Voogsgerd) - - New issue board list dropdown stays open after adding a new list - - Fix: Backup restore doesn't clear cache - - API: Fix project deploy keys 400 and 500 errors when adding an existing key. !6784 (Joshua Welsh) - - Replace jquery.cookie plugin with js.cookie !7085 - - Use MergeRequestsClosingIssues cache data on Issue#closed_by_merge_requests method - - Fix Sign in page 'Forgot your password?' link overlaps on medium-large screens - - Show full status link on MR & commit pipelines - - Fix documents and comments on Build API `scope` - - Refactor email, use setter method instead AR callbacks for email attribute (Semyon Pupkov) - - Shortened merge request modal to let clipboard button not overlap - - In all filterable drop downs, put input field in focus only after load is complete (Ido @leibo) - -## 8.13.2 - - Fix builds dropdown overlapping bug !7124 - - Fix applying labels for GitHub-imported MRs !7139 - - Fix importing MR comments from GitHub !7139 - - Modify GitHub importer to be retryable !7003 - - Fix and improve `Sortable.highest_label_priority` - - Fixed sticky merge request tabs when sidebar is pinned ## 8.13.1 (2016-10-25) @@ -196,7 +196,7 @@ gem 'loofah', '~> 2.0.3' gem 'licensee', '~> 8.0.0' # Protect against bruteforcing -gem 'rack-attack', '~> 4.3.1' +gem 'rack-attack', '~> 4.4.1' # Ace editor gem 'ace-rails-ap', '~> 4.1.0' diff --git a/Gemfile.lock b/Gemfile.lock index 81af1ad2dac..79c828ff0f2 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -520,7 +520,7 @@ GEM rack (1.6.4) rack-accept (0.4.5) rack (>= 0.4) - rack-attack (4.3.1) + rack-attack (4.4.1) rack rack-cors (0.4.0) rack-mount (0.8.3) @@ -929,7 +929,7 @@ DEPENDENCIES poltergeist (~> 1.9.0) premailer-rails (~> 1.9.0) pry-rails (~> 0.3.4) - rack-attack (~> 4.3.1) + rack-attack (~> 4.4.1) rack-cors (~> 0.4.0) rack-oauth2 (~> 1.2.1) rails (= 4.2.7.1) diff --git a/app/assets/stylesheets/framework/pagination.scss b/app/assets/stylesheets/framework/pagination.scss index b6f21fd8c91..cb2c351c368 100644 --- a/app/assets/stylesheets/framework/pagination.scss +++ b/app/assets/stylesheets/framework/pagination.scss @@ -7,8 +7,70 @@ .pagination { padding: 0; } + + .gap, + .gap:hover { + background-color: $gray-light; + padding: $gl-vert-padding; + cursor: default; + } } .panel > .gl-pagination { margin: 0; } + +/** + * Extra-small screen pagination. + */ +@media (max-width: 320px) { + .gl-pagination { + .first, + .last { + display: none; + } + + .page { + display: none; + + &.active { + display: inline; + } + } + } +} + +/** + * Small screen pagination + */ +@media (max-width: $screen-xs) { + .gl-pagination { + .pagination li a { + padding: 6px 10px; + } + + .page { + display: none; + + &.active { + display: inline; + } + } + } +} + +/** + * Medium screen pagination + */ +@media (min-width: $screen-xs) and (max-width: $screen-md-max) { + .gl-pagination { + .page { + display: none; + + &.active, + &.sibling { + display: inline; + } + } + } +} diff --git a/app/assets/stylesheets/pages/builds.scss b/app/assets/stylesheets/pages/builds.scss index d6a55fbd464..6300ac9662f 100644 --- a/app/assets/stylesheets/pages/builds.scss +++ b/app/assets/stylesheets/pages/builds.scss @@ -52,10 +52,25 @@ .build-header { position: relative; - padding-right: 40px; + padding: 0; + display: flex; + min-height: 58px; + align-items: center; - @media (min-width: $screen-sm-min) { - padding-right: 0; + .btn-inverted { + @include btn-outline($white-light, $blue-normal, $blue-normal, $blue-light, $white-light, $blue-light); + } + + @media (max-width: $screen-sm-max) { + padding-right: 40px; + + .btn-inverted { + display: none; + } + } + + .header-content { + flex: 1; } a { @@ -137,10 +152,15 @@ .retry-link { color: $gl-link-color; + display: none; &:hover { text-decoration: underline; } + + @media (max-width: $screen-sm-max) { + display: block; + } } .stage-item { diff --git a/app/policies/project_policy.rb b/app/policies/project_policy.rb index fbb3d4507d6..1ee31023e26 100644 --- a/app/policies/project_policy.rb +++ b/app/policies/project_policy.rb @@ -2,11 +2,11 @@ class ProjectPolicy < BasePolicy def rules team_access!(user) - owner = user.admin? || - project.owner == user || + owner = project.owner == user || (project.group && project.group.has_owner?(user)) - owner_access! if owner + owner_access! if user.admin? || owner + team_member_owner_access! if owner if project.public? || (project.internal? && !user.external?) guest_access! @@ -16,7 +16,7 @@ class ProjectPolicy < BasePolicy can! :read_build if project.public_builds? if project.request_access_enabled && - !(owner || project.team.member?(user) || project_group_member?(user)) + !(owner || user.admin? || project.team.member?(user) || project_group_member?(user)) can! :request_access end end @@ -135,6 +135,10 @@ class ProjectPolicy < BasePolicy can! :destroy_issue end + def team_member_owner_access! + team_member_reporter_access! + end + # Push abilities on the users team role def team_access!(user) access = project.team.max_member_access(user.id) diff --git a/app/views/kaminari/gitlab/_gap.html.haml b/app/views/kaminari/gitlab/_gap.html.haml index 80ca30f36e6..889514c4755 100644 --- a/app/views/kaminari/gitlab/_gap.html.haml +++ b/app/views/kaminari/gitlab/_gap.html.haml @@ -4,6 +4,6 @@ -# total_pages: total number of pages -# per_page: number of items to fetch per page -# remote: data-remote -%li{class: "page"} - %span.page.gap +%li + %span.gap = raw(t 'views.pagination.truncate') diff --git a/app/views/kaminari/gitlab/_page.html.haml b/app/views/kaminari/gitlab/_page.html.haml index 522e4d1d05f..750aed8f329 100644 --- a/app/views/kaminari/gitlab/_page.html.haml +++ b/app/views/kaminari/gitlab/_page.html.haml @@ -6,5 +6,5 @@ -# total_pages: total number of pages -# per_page: number of items to fetch per page -# remote: data-remote -%li{class: "page#{' active' if page.current?}"} +%li{class: "page#{' active' if page.current?}#{' sibling' if page.next? || page.prev?}"} = link_to page, url, {remote: remote, rel: page.next? ? 'next' : page.prev? ? 'prev' : nil} diff --git a/app/views/projects/builds/_header.html.haml b/app/views/projects/builds/_header.html.haml index 51b5bd9db42..3f2ce7377fd 100644 --- a/app/views/projects/builds/_header.html.haml +++ b/app/views/projects/builds/_header.html.haml @@ -1,16 +1,19 @@ .content-block.build-header - = ci_status_with_icon(@build.status) - Build - %strong ##{@build.id} - for commit - = link_to ci_status_path(@build.pipeline) do - %strong= @build.pipeline.short_sha - from - = link_to namespace_project_commits_path(@project.namespace, @project, @build.ref) do - %code - = @build.ref - - if @build.user - = render "user" - = time_ago_with_tooltip(@build.created_at) + .header-content + = ci_status_with_icon(@build.status) + Build + %strong ##{@build.id} + for commit + = link_to ci_status_path(@build.pipeline) do + %strong= @build.pipeline.short_sha + from + = link_to namespace_project_commits_path(@project.namespace, @project, @build.ref) do + %code + = @build.ref + - if @build.user + = render "user" + = time_ago_with_tooltip(@build.created_at) + - if can?(current_user, :update_build, @build) && @build.retryable? + = link_to "Retry build", retry_namespace_project_build_path(@project.namespace, @project, @build), class: 'btn btn-inverted pull-right', method: :post %button.btn.btn-default.pull-right.visible-xs-block.visible-sm-block.build-gutter-toggle.js-sidebar-build-toggle{ role: "button", type: "button" } = icon('angle-double-left') diff --git a/app/views/projects/builds/_sidebar.html.haml b/app/views/projects/builds/_sidebar.html.haml index b1053028279..28f519f11b2 100644 --- a/app/views/projects/builds/_sidebar.html.haml +++ b/app/views/projects/builds/_sidebar.html.haml @@ -44,7 +44,7 @@ .title Build details - if can?(current_user, :update_build, @build) && @build.retryable? - = link_to "Retry", retry_namespace_project_build_path(@project.namespace, @project, @build), class: 'pull-right retry-link', method: :post + = link_to "Retry build", retry_namespace_project_build_path(@project.namespace, @project, @build), class: 'pull-right retry-link', method: :post - if @build.merge_request %p.build-detail-row %span.build-light-text Merge Request: diff --git a/app/views/projects/edit.html.haml b/app/views/projects/edit.html.haml index ff6a899b92a..c40ad06969e 100644 --- a/app/views/projects/edit.html.haml +++ b/app/views/projects/edit.html.haml @@ -181,6 +181,7 @@ %ul %li Build traces and artifacts %li LFS objects + %li Container registry images %hr - if can? current_user, :archive_project, @project .row.prepend-top-default diff --git a/app/views/shared/issuable/_milestone_dropdown.html.haml b/app/views/shared/issuable/_milestone_dropdown.html.haml index f27a9002ec2..40fe53e6a8d 100644 --- a/app/views/shared/issuable/_milestone_dropdown.html.haml +++ b/app/views/shared/issuable/_milestone_dropdown.html.haml @@ -1,10 +1,10 @@ - project = @target_project || @project - extra_class = extra_class || '' - show_menu_above = show_menu_above || false -- selected_text = selected.try(:title) +- selected_text = selected.try(:title) || params[:milestone_title] - dropdown_title = local_assigns.fetch(:dropdown_title, "Filter by milestone") - if selected.present? - = hidden_field_tag(name, name == :milestone_title ? selected.title : selected.id) + = hidden_field_tag(name, name == :milestone_title ? selected_text : selected.id) = dropdown_tag(milestone_dropdown_label(selected_text), options: { title: dropdown_title, toggle_class: "js-milestone-select js-filter-submit #{extra_class}", filter: true, dropdown_class: "dropdown-menu-selectable dropdown-menu-milestone", placeholder: "Search milestones", footer_content: project.present?, data: { show_no: true, show_menu_above: show_menu_above, show_any: show_any, show_upcoming: show_upcoming, field_name: name, selected: selected.try(:title), project_id: project.try(:id), milestones: milestones_filter_dropdown_path, default_label: "Milestone" } }) do - if project diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example index 3451b68cea5..699ab6075b6 100644 --- a/config/gitlab.yml.example +++ b/config/gitlab.yml.example @@ -432,7 +432,9 @@ production: &base ## Repositories settings repositories: # Paths where repositories can be stored. Give the canonicalized absolute pathname. - # NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!! + # IMPORTANT: None of the path components may be symlink, because + # gitlab-shell invokes Dir.pwd inside the repository path and that results + # real path not the symlink. storages: # You must have at least a `default` storage path. default: /home/git/repositories/ diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md new file mode 100644 index 00000000000..f3c2e72341f --- /dev/null +++ b/doc/administration/raketasks/maintenance.md @@ -0,0 +1,220 @@ +# Maintenance Rake Tasks + +## Gather information about GitLab and the system it runs on + +This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:env:info +``` + +**Source Installation** + +``` +bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +Example output: + +``` +System information +System: Debian 7.8 +Current User: git +Using RVM: no +Ruby Version: 2.1.5p273 +Gem Version: 2.4.3 +Bundler Version: 1.7.6 +Rake Version: 10.3.2 +Sidekiq Version: 2.17.8 + +GitLab information +Version: 7.7.1 +Revision: 41ab9e1 +Directory: /home/git/gitlab +DB Adapter: postgresql +URL: https://gitlab.example.com +HTTP Clone URL: https://gitlab.example.com/some-project.git +SSH Clone URL: git@gitlab.example.com:some-project.git +Using LDAP: no +Using Omniauth: no + +GitLab Shell +Version: 2.4.1 +Repositories: /home/git/repositories/ +Hooks: /home/git/gitlab-shell/hooks/ +Git: /usr/bin/git +``` + +## Check GitLab configuration + +Runs the following rake tasks: + +- `gitlab:gitlab_shell:check` +- `gitlab:sidekiq:check` +- `gitlab:app:check` + +It will check that each component was setup according to the installation guide and suggest fixes for issues found. + +You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:check +``` + +**Source Installation** + +``` +bundle exec rake gitlab:check RAILS_ENV=production +``` + +NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output. + +Example output: + +``` +Checking Environment ... + +Git configured for git user? ... yes +Has python2? ... yes +python2 is supported version? ... yes + +Checking Environment ... Finished + +Checking GitLab Shell ... + +GitLab Shell version? ... OK (1.2.0) +Repo base directory exists? ... yes +Repo base directory is a symlink? ... no +Repo base owned by git:git? ... yes +Repo base access is drwxrws---? ... yes +post-receive hook up-to-date? ... yes +post-receive hooks in repos are links: ... yes + +Checking GitLab Shell ... Finished + +Checking Sidekiq ... + +Running? ... yes + +Checking Sidekiq ... Finished + +Checking GitLab ... + +Database config exists? ... yes +Database is SQLite ... no +All migrations up? ... yes +GitLab config exists? ... yes +GitLab config outdated? ... no +Log directory writable? ... yes +Tmp directory writable? ... yes +Init script exists? ... yes +Init script up-to-date? ... yes +Redis version >= 2.0.0? ... yes + +Checking GitLab ... Finished +``` + +## Rebuild authorized_keys file + +In some case it is necessary to rebuild the `authorized_keys` file. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:shell:setup +``` + +**Source Installation** + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production +``` + +``` +This will rebuild an authorized_keys file. +You will lose any data stored in authorized_keys file. +Do you want to continue (yes/no)? yes +``` + +## Clear redis cache + +If for some reason the dashboard shows wrong information you might want to +clear Redis' cache. + +**Omnibus Installation** + +``` +sudo gitlab-rake cache:clear +``` + +**Source Installation** + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +## Precompile the assets + +Sometimes during version upgrades you might end up with some wrong CSS or +missing some icons. In that case, try to precompile the assets again. + +Note that this only applies to source installations and does NOT apply to +Omnibus packages. + +**Source Installation** + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production +``` + +For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at +the release of upstream GitLab. The omnibus version includes optimized versions +of those assets. Unless you are modifying the JavaScript / CSS code on your +production machine after installing the package, there should be no reason to redo +rake assets:precompile on the production machine. If you suspect that assets +have been corrupted, you should reinstall the omnibus package. + +## Tracking Deployments + +GitLab provides a Rake task that lets you track deployments in GitLab +Performance Monitoring. This Rake task simply stores the current GitLab version +in the GitLab Performance Monitoring database. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:track_deployment +``` + +**Source Installation** + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production +``` + +## Create or repair repository hooks symlink + +If the GitLab shell hooks directory location changes or another circumstance +leads to the hooks symlink becoming missing or invalid, run this Rake task +to create or repair the symlinks. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:shell:create_hooks +``` + +**Source Installation** + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:shell:create_hooks RAILS_ENV=production +``` diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index d8dce4388e1..6f1356ddf8f 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -107,7 +107,7 @@ downtime. Otherwise skip to the next section. 1. To see the current threads, run: ``` - apply all thread bt + thread apply all bt ``` 1. Once you're done debugging with `gdb`, be sure to detach from the process and exit: diff --git a/doc/api/projects.md b/doc/api/projects.md index 8ebac57e612..4f4b20a1874 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -1139,6 +1139,7 @@ Parameters: | `pipeline_events` | boolean | no | Trigger hook on pipeline events | | `wiki_events` | boolean | no | Trigger hook on wiki events | | `enable_ssl_verification` | boolean | no | Do SSL verification when triggering the hook | +| `token` | string | no | Secret token to validate received payloads; this will not be returned in the response | ### Edit project hook @@ -1164,6 +1165,7 @@ Parameters: | `pipeline_events` | boolean | no | Trigger hook on pipeline events | | `wiki_events` | boolean | no | Trigger hook on wiki events | | `enable_ssl_verification` | boolean | no | Do SSL verification when triggering the hook | +| `token` | string | no | Secret token to validate received payloads; this will not be returned in the response | ### Delete project hook diff --git a/doc/development/testing.md b/doc/development/testing.md index 8e91ac5e3ba..b0b26ccf57a 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -132,6 +132,42 @@ Adding new Spinach scenarios is acceptable _only if_ the new scenario requires no more than one new `step` definition. If more than that is required, the test should be re-implemented using RSpec instead. +## Testing Rake Tasks + +To make testing Rake tasks a little easier, there is a helper that can be included +in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use +`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures +a few other things to make testing Rake tasks easier. + +At a minimum, requiring the Rake helper will redirect `stdout`, include the +runtime task helpers, and include the `RakeHelpers` Spec support module. + +The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make +executing tasks simple. See `spec/support/rake_helpers.rb` for all available +methods. + +Example: + +```ruby +require 'rake_helper' + +describe 'gitlab:shell rake tasks' do + before do + Rake.application.rake_require 'tasks/gitlab/shell' + + stub_warn_user_is_not_gitlab + end + + describe 'install task' do + it 'invokes create_hooks task' do + expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) + + run_rake_task('gitlab:shell:install') + end + end +end +``` + --- [Return to Development documentation](README.md) diff --git a/doc/integration/jira.md b/doc/integration/jira.md index cf1557ddc44..2e31fd994de 100644 --- a/doc/integration/jira.md +++ b/doc/integration/jira.md @@ -135,7 +135,7 @@ password as they will be needed when configuring GitLab in the next section. JIRA configuration in GitLab is done via a project's **Services**. -#### GitLab 13.0 with JIRA v1000.x +#### GitLab 8.13.0 with JIRA v1000.x To enable JIRA integration in a project, navigate to the project's and open the context menu clicking on the top right gear icon, then go to @@ -160,7 +160,7 @@ with the linked JIRA project. #### GitLab 6.x-7.7 with JIRA v6.x -_**Note:** GitLab versions 13.0 and up contain various integration improvements. +_**Note:** GitLab versions 8.13.0 and up contain various integration improvements. We strongly recommend upgrading._ In `gitlab.yml` enable the JIRA issue tracker section by diff --git a/doc/raketasks/maintenance.md b/doc/raketasks/maintenance.md index 315cb56a089..266aeb7d60e 100644 --- a/doc/raketasks/maintenance.md +++ b/doc/raketasks/maintenance.md @@ -1,188 +1,3 @@ -# Maintenance +# Maintenance Rake Tasks -## Gather information about GitLab and the system it runs on - -This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:env:info - -# installation from source -bundle exec rake gitlab:env:info RAILS_ENV=production -``` - -Example output: - -``` -System information -System: Debian 7.8 -Current User: git -Using RVM: no -Ruby Version: 2.1.5p273 -Gem Version: 2.4.3 -Bundler Version: 1.7.6 -Rake Version: 10.3.2 -Sidekiq Version: 2.17.8 - -GitLab information -Version: 7.7.1 -Revision: 41ab9e1 -Directory: /home/git/gitlab -DB Adapter: postgresql -URL: https://gitlab.example.com -HTTP Clone URL: https://gitlab.example.com/some-project.git -SSH Clone URL: git@gitlab.example.com:some-project.git -Using LDAP: no -Using Omniauth: no - -GitLab Shell -Version: 2.4.1 -Repositories: /home/git/repositories/ -Hooks: /home/git/gitlab-shell/hooks/ -Git: /usr/bin/git -``` - -## Check GitLab configuration - -Runs the following rake tasks: - -- `gitlab:gitlab_shell:check` -- `gitlab:sidekiq:check` -- `gitlab:app:check` - -It will check that each component was setup according to the installation guide and suggest fixes for issues found. - -You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:check - -# installation from source -bundle exec rake gitlab:check RAILS_ENV=production -``` - -NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output. - -Example output: - -``` -Checking Environment ... - -Git configured for git user? ... yes -Has python2? ... yes -python2 is supported version? ... yes - -Checking Environment ... Finished - -Checking GitLab Shell ... - -GitLab Shell version? ... OK (1.2.0) -Repo base directory exists? ... yes -Repo base directory is a symlink? ... no -Repo base owned by git:git? ... yes -Repo base access is drwxrws---? ... yes -post-receive hook up-to-date? ... yes -post-receive hooks in repos are links: ... yes - -Checking GitLab Shell ... Finished - -Checking Sidekiq ... - -Running? ... yes - -Checking Sidekiq ... Finished - -Checking GitLab ... - -Database config exists? ... yes -Database is SQLite ... no -All migrations up? ... yes -GitLab config exists? ... yes -GitLab config outdated? ... no -Log directory writable? ... yes -Tmp directory writable? ... yes -Init script exists? ... yes -Init script up-to-date? ... yes -Redis version >= 2.0.0? ... yes - -Checking GitLab ... Finished -``` - -## Rebuild authorized_keys file - -In some case it is necessary to rebuild the `authorized_keys` file. - -For Omnibus-packages: -``` -sudo gitlab-rake gitlab:shell:setup -``` - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production -``` - -``` -This will rebuild an authorized_keys file. -You will lose any data stored in authorized_keys file. -Do you want to continue (yes/no)? yes -``` - -## Clear redis cache - -If for some reason the dashboard shows wrong information you might want to -clear Redis' cache. - -For Omnibus-packages: -``` -sudo gitlab-rake cache:clear -``` - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production -``` - -## Precompile the assets - -Sometimes during version upgrades you might end up with some wrong CSS or -missing some icons. In that case, try to precompile the assets again. - -Note that this only applies to source installations and does NOT apply to -omnibus packages. - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production -``` - -For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at -the release of upstream GitLab. The omnibus version includes optimized versions -of those assets. Unless you are modifying the JavaScript / CSS code on your -production machine after installing the package, there should be no reason to redo -rake assets:precompile on the production machine. If you suspect that assets -have been corrupted, you should reinstall the omnibus package. - -## Tracking Deployments - -GitLab provides a Rake task that lets you track deployments in GitLab -Performance Monitoring. This Rake task simply stores the current GitLab version -in the GitLab Performance Monitoring database. - -For Omnibus-packages: - -``` -sudo gitlab-rake gitlab:track_deployment -``` - -For installations from source: - -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production -``` +This document was moved to [administration/raketasks/maintenance](../administration/raketasks/maintenance.md). diff --git a/doc/update/README.md b/doc/update/README.md index 975d72164b4..837b31abb97 100644 --- a/doc/update/README.md +++ b/doc/update/README.md @@ -85,6 +85,8 @@ possible. - [MySQL installation guide](../install/database_mysql.md) contains additional information about configuring GitLab to work with a MySQL database. - [Restoring from backup after a failed upgrade](restore_after_failure.md) +- [Upgrading PostgreSQL Using Slony](upgrading_postgresql_using_slony.md), for + upgrading a PostgreSQL database with minimal downtime. [omnidocker]: http://docs.gitlab.com/omnibus/docker/README.html [source-ee]: https://gitlab.com/gitlab-org/gitlab-ee/tree/master/doc/update diff --git a/doc/update/upgrading_postgresql_using_slony.md b/doc/update/upgrading_postgresql_using_slony.md new file mode 100644 index 00000000000..f009906256e --- /dev/null +++ b/doc/update/upgrading_postgresql_using_slony.md @@ -0,0 +1,482 @@ +# Upgrading PostgreSQL Using Slony + +This guide describes the steps one can take to upgrade their PostgreSQL database +to the latest version without the need for hours of downtime. This guide assumes +you have two database servers: one database server running an older version of +PostgreSQL (e.g. 9.2.18) and one server running a newer version (e.g. 9.6.0). + +For this process we'll use a PostgreSQL replication tool called +["Slony"](http://www.slony.info/). Slony allows replication between different +PostgreSQL versions and as such can be used to upgrade a cluster with a minimal +amount of downtime. + +In various places we'll refer to the user `gitlab-psql`. This user should be the +user used to run the various PostgreSQL OS processes. If you're using a +different user (e.g. `postgres`) you should replace `gitlab-psql` with the name +of said user. This guide also assumes your database is called +`gitlabhq_production`. If you happen to use a different database name you should +change this accordingly. + +## Database Dumps + +Slony only replicates data and not any schema changes. As a result we must +ensure that all databases have the same database structure. + +To do so we'll generate a dump of our current database. This dump will only +contain the structure, not any data. To generate this dump run the following +command on your active database server: + +```bash +sudo -u gitlab-psql /opt/gitlab/embedded/bin/pg_dump -h /var/opt/gitlab/postgresql -p 5432 -U gitlab-psql -s -f /tmp/structure.sql gitlabhq_production +``` + +If you're not using GitLab's Omnibus package you may have to adjust the paths to +`pg_dump` and the PostgreSQL installation directory to match the paths of your +configuration. + +Once the structure dump is generated we also need to generate a dump for the +`schema_migrations` table. This table doesn't have any primary keys and as such +can't be replicated easily by Slony. To generate this dump run the following +command on your active database server: + +```bash +sudo -u gitlab-psql /opt/gitlab/embedded/bin/pg_dump -h /var/opt/gitlab/postgresql/ -p 5432 -U gitlab-psql -a -t schema_migrations -f /tmp/migrations.sql gitlabhq_production +``` + +Next we'll need to move these files somewhere accessible by the new database +server. The easiest way is to simply download these files to your local system: + +```bash +scp your-user@production-database-host:/tmp/*.sql /tmp +``` + +This will copy all the SQL files located in `/tmp` to your local system's +`/tmp` directory. Once copied you can safely remove the files from the database +server. + +## Installing Slony + +Slony will be used to upgrade the database without requiring long downtimes. +Slony can be downloaded from http://www.slony.info/. If you have installed +PostgreSQL using your operating system's package manager you may also be able to +install Slony using said package manager. + +When compiling Slony from source you *must* use the following commands to do so: + +```bash +./configure --prefix=/path/to/installation/directory --with-perltools --with-pgconfigdir=/path/to/directory/containing/pg_config/bin +make +make install +``` + +Omnibus users can use the following commands: + +```bash +./configure --prefix=/opt/gitlab/embedded --with-perltools --with-pgconfigdir=/opt/gitlab/embedded/bin +make +make install +``` + +This assumes you have installed GitLab into /opt/gitlab. + +To test if Slony is installed properly, run the following commands: + +```bash +test -f /opt/gitlab/embedded/bin/slonik && echo 'Slony installed' || echo 'Slony not installed' +test -f /opt/gitlab/embedded/bin/slonik_init_cluster && echo 'Slony Perl tools are available' || echo 'Slony Perl tools are not available' +/opt/gitlab/embedded/bin/slonik -v +``` + +This assumes Slony was installed to `/opt/gitlab/embedded`. If Slony was +installed properly the output of these commands will be (the mentioned "slonik" +version may be different): + +``` +Slony installed +Slony Perl tools are available +slonik version 2.2.5 +``` + +## Slony User + +Next we must set up a PostgreSQL user that Slony can use to replicate your +database. To do so, log in to your production database using `psql` using a +super user account. Once done run the following SQL queries: + +```sql +CREATE ROLE slony WITH SUPERUSER LOGIN REPLICATION ENCRYPTED PASSWORD 'password string here'; +ALTER ROLE slony SET statement_timeout TO 0; +``` + +Make sure you replace "password string here" with the actual password for the +user. A password is *required*. This user must be created on _both_ the old and +new database server using the same password. + +Once the user has been created make sure you note down the password as we will +need it later on. + +## Configuring Slony + +Now we can finally start configuring Slony. Slony uses a configuration file for +most of the work so we'll need to set this one up. This configuration file +specifies where to put log files, how Slony should connect to the databases, +etc. + +First we'll need to create some required directories and set the correct +permissions. To do so, run the following commands on both the old and new +database server: + +```bash +sudo mkdir -p /var/log/gitlab/slony /var/run/slony1 /var/opt/gitlab/postgresql/slony +sudo chown gitlab-psql:root /var/log/gitlab/slony /var/run/slony1 /var/opt/gitlab/postgresql/slony +``` + +Here `gitlab-psql` is the user used to run the PostgreSQL database processes. If +you're using a different user you should replace this with the name of said +user. + +Now that the directories are in place we can create the configuration file. For +this we can use the following template: + +```perl +if ($ENV{"SLONYNODES"}) { + require $ENV{"SLONYNODES"}; +} else { + $CLUSTER_NAME = 'slony_replication'; + $LOGDIR = '/var/log/gitlab/slony'; + $MASTERNODE = 1; + $DEBUGLEVEL = 2; + + add_node(host => 'OLD_HOST', dbname => 'gitlabhq_production', port =>5432, + user=>'slony', password=>'SLONY_PASSWORD', node=>1); + + add_node(host => 'NEW_HOST', dbname => 'gitlabhq_production', port =>5432, + user=>'slony', password=>'SLONY_PASSWORD', node=>2, parent=>1 ); +} + +$SLONY_SETS = { + "set1" => { + "set_id" => 1, + "table_id" => 1, + "sequence_id" => 1, + "pkeyedtables" => [ + TABLES + ], + }, +}; + +if ($ENV{"SLONYSET"}) { + require $ENV{"SLONYSET"}; +} + +# Please do not add or change anything below this point. +1; +``` + +In this configuration file you should replace a few placeholders before you can +use it. The following placeholders should be replaced: + +* `OLD_HOST`: the address of the old database server. +* `NEW_HOST`: the address of the new database server. +* `SLONY_PASSWORD`: the password of the Slony user created earlier. +* `TABLES`: the tables to replicate. + +The list of tables to replicate can be generated by running the following +command on your old PostgreSQL database: + +``` +sudo gitlab-psql gitlabhq_production -c "select concat('\"', schemaname, '.', tablename, '\",') from pg_catalog.pg_tables where schemaname = 'public' and tableowner = 'gitlab' and tablename != 'schema_migrations' order by tablename asc;" -t +``` + +If you're not using Omnibus you should replace `gitlab-psql` with the +appropriate path to the `psql` executable. + +The above command outputs a list of tables in a format that can be copy-pasted +directly into the above configuration file. Make sure to _replace_ `TABLES` with +this output, don't just append it below it. Once done you'll end up with +something like this: + +```perl +"pkeyedtables" => [ + "public.abuse_reports", + "public.appearances", + "public.application_settings", + ... more rows here ... +] +``` + +Once you have the configuration file generated you must install it on both the +old and new database. To do so, place it in +`/var/opt/gitlab/postgresql/slony/slon_tools.conf` (for which we created the +directory earlier on). + +Now that the configuration file is in place we can _finally_ start replicating +our database. First we must set up the schema in our new database. To do so make +sure that the SQL files we generated earlier can be found in the `/tmp` +directory of the new server. Once these files are in place start a `psql` +session on this server: + +``` +sudo gitlab-psql gitlabhq_production +``` + +Now run the following commands: + +``` +\i /tmp/structure.sql +\i /tmp/migrations.sql +``` + +To verify if the structure is in place close the session, start it again, then +run `\d`. If all went well you should see output along the lines of the +following: + +``` + List of relations + Schema | Name | Type | Owner +--------+---------------------------------------------+----------+------------- + public | abuse_reports | table | gitlab + public | abuse_reports_id_seq | sequence | gitlab + public | appearances | table | gitlab + public | appearances_id_seq | sequence | gitlab + public | application_settings | table | gitlab + public | application_settings_id_seq | sequence | gitlab + public | approvals | table | gitlab + ... more rows here ... +``` + +Now we can initialize the required tables and what not that Slony will use for +its replication process. To do so, run the following on the old database: + +``` +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slonik_init_cluster --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf | /opt/gitlab/embedded/bin/slonik +``` + +If all went well this will produce something along the lines of: + +``` +<stdin>:10: Set up replication nodes +<stdin>:13: Next: configure paths for each node/origin +<stdin>:16: Replication nodes prepared +<stdin>:17: Please start a slon replication daemon for each node +``` + +Next we need to start a replication node on every server. To do so, run the +following on the old database: + +``` +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slon_start 1 --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf +``` + +If all went well this will produce output such as: + + +``` +Invoke slon for node 1 - /opt/gitlab/embedded/bin/slon -p /var/run/slony1/slony_replication_node1.pid -s 1000 -d2 slony_replication 'host=192.168.0.7 dbname=gitlabhq_production user=slony port=5432 password=hieng8ezohHuCeiqu0leeghai4aeyahp' > /var/log/gitlab/slony/node1/gitlabhq_production-2016-10-06.log 2>&1 & +Slon successfully started for cluster slony_replication, node node1 +PID [26740] +Start the watchdog process as well... +``` + +Next we need to run the following command on the _new_ database server: + +``` +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slon_start 2 --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf +``` + +This will produce similar output if all went well. + +Next we need to tell the new database server what it should replicate. This can +be done by running the following command on the _new_ database server: + +``` +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slonik_create_set 1 --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf | /opt/gitlab/embedded/bin/slonik +``` + +This should produce output along the lines of the following: + +``` +<stdin>:11: Subscription set 1 (set1) created +<stdin>:12: Adding tables to the subscription set +<stdin>:16: Add primary keyed table public.abuse_reports +<stdin>:20: Add primary keyed table public.appearances +<stdin>:24: Add primary keyed table public.application_settings +... more rows here ... +<stdin>:327: Adding sequences to the subscription set +<stdin>:328: All tables added +``` + +Finally we can start the replication process by running the following on the +_new_ database server: + +``` +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slonik_subscribe_set 1 2 --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf | /opt/gitlab/embedded/bin/slonik +``` + +This should produce the following output: + +``` +<stdin>:6: Subscribed nodes to set 1 +``` + +At this point the new database server will start replicating the data of the old +database server. This process can take anywhere from a few minutes to hours, if +not days. Unfortunately Slony itself doesn't really provide a way of knowing +when the two databases are in sync. To get an estimate of the progress you can +use the following shell script: + +``` +#!/usr/bin/env bash + +set -e + +user='slony' +pass='SLONY_PASSWORD' + +function main { + while : + do + local source + local target + + source=$(PGUSER="${user}" PGPASSWORD="${pass}" /opt/gitlab/embedded/bin/psql -h OLD_HOST gitlabhq_production -c "select pg_size_pretty(pg_database_size('gitlabhq_production'));" -t -A) + target=$(PGUSER="${user}" PGPASSWORD="${pass}" /opt/gitlab/embedded/bin/psql -h NEW_HOST gitlabhq_production -c "select pg_size_pretty(pg_database_size('gitlabhq_production'));" -t -A) + + echo "$(date): ${target} of ${source}" >> progress.log + echo "$(date): ${target} of ${source}" + + sleep 60 + done +} + +main +``` + +This script will compare the sizes of the old and new database every minute and +print the result to STDOUT as well as logging it to a file. Make sure to replace +`SLONY_PASSWORD`, `OLD_HOST`, and `NEW_HOST` with the correct values. + +## Stopping Replication + +At some point the two databases are in sync. Once this is the case you'll need +to plan for a few minutes of downtime. This small downtime window is used to +stop the replication process, remove any Slony data from both databases, restart +GitLab so it can use the new database, etc. + +First, let's stop all of GitLab. Omnibus users can do so by running the +following on their GitLab server(s): + +``` +sudo gitlab-ctl stop unicorn +sudo gitlab-ctl stop sidekiq +sudo gitlab-ctl stop mailroom +``` + +If you have any other processes that use PostgreSQL you should also stop those. + +Once everything has been stopped you should update any configuration settings, +DNS records, etc so they all point to the new database. + +Once the settings have been taken care of we need to stop the replication +process. It's crucial that no new data is written to the databases at this point +as this data will be lost. + +To stop replication, run the following on both database servers: + +```bash +sudo -u gitlab-psql /opt/gitlab/embedded/bin/slon_kill --conf /var/opt/gitlab/postgresql/slony/slon_tools.conf +``` + +This will stop all the Slony processes on the host the command was executed on. + +## Resetting Sequences + +The above setup does not replicate database sequences, as such these must be +reset manually in the target database. You can use the following script for +this: + +```bash +#!/usr/bin/env bash +set -e + +function main { + local fix_sequences + local fix_owners + + fix_sequences='/tmp/fix_sequences.sql' + fix_owners='/tmp/fix_owners.sql' + + # The SQL queries were taken from + # https://wiki.postgresql.org/wiki/Fixing_Sequences + sudo gitlab-psql gitlabhq_production -t -c " + SELECT 'ALTER SEQUENCE '|| quote_ident(MIN(schema_name)) ||'.'|| quote_ident(MIN(seq_name)) + ||' OWNED BY '|| quote_ident(MIN(TABLE_NAME)) ||'.'|| quote_ident(MIN(column_name)) ||';' + FROM ( + SELECT + n.nspname AS schema_name, + c.relname AS TABLE_NAME, + a.attname AS column_name, + SUBSTRING(d.adsrc FROM E'^nextval\\(''([^'']*)''(?:::text|::regclass)?\\)') AS seq_name + FROM pg_class c + JOIN pg_attribute a ON (c.oid=a.attrelid) + JOIN pg_attrdef d ON (a.attrelid=d.adrelid AND a.attnum=d.adnum) + JOIN pg_namespace n ON (c.relnamespace=n.oid) + WHERE has_schema_privilege(n.oid,'USAGE') + AND n.nspname NOT LIKE 'pg!_%' escape '!' + AND has_table_privilege(c.oid,'SELECT') + AND (NOT a.attisdropped) + AND d.adsrc ~ '^nextval' + ) seq + GROUP BY seq_name HAVING COUNT(*)=1; + " > "${fix_owners}" + + sudo gitlab-psql gitlabhq_production -t -c " + SELECT 'SELECT SETVAL(' || + quote_literal(quote_ident(PGT.schemaname) || '.' || quote_ident(S.relname)) || + ', COALESCE(MAX(' ||quote_ident(C.attname)|| '), 1) ) FROM ' || + quote_ident(PGT.schemaname)|| '.'||quote_ident(T.relname)|| ';' + FROM pg_class AS S, + pg_depend AS D, + pg_class AS T, + pg_attribute AS C, + pg_tables AS PGT + WHERE S.relkind = 'S' + AND S.oid = D.objid + AND D.refobjid = T.oid + AND D.refobjid = C.attrelid + AND D.refobjsubid = C.attnum + AND T.relname = PGT.tablename + ORDER BY S.relname; + " > "${fix_sequences}" + + sudo gitlab-psql gitlabhq_production -f "${fix_owners}" + sudo gitlab-psql gitlabhq_production -f "${fix_sequences}" + + rm "${fix_owners}" "${fix_sequences}" +} + +main +``` + +Upload this script to the _target_ server and execute it as follows: + +```bash +bash path/to/the/script/above.sh +``` + +This will correct the ownership of sequences and reset the next value for the +`id` column to the next available value. + +## Removing Slony + +Next we need to remove all Slony related data. To do so, run the following +command on the _target_ server: + +```bash +sudo gitlab-psql gitlabhq_production -c "DROP SCHEMA _slony_replication CASCADE;" +``` + +Once done you can safely remove any Slony related files (e.g. the log +directory), and uninstall Slony if desired. At this point you can start your +GitLab instance again and if all went well it should be using your new database +server. diff --git a/features/steps/dashboard/help.rb b/features/steps/dashboard/help.rb index 9c94dc70df0..3c5bf44c538 100644 --- a/features/steps/dashboard/help.rb +++ b/features/steps/dashboard/help.rb @@ -8,7 +8,7 @@ class Spinach::Features::DashboardHelp < Spinach::FeatureSteps end step 'I visit the "Rake Tasks" help page' do - visit help_page_path("raketasks/maintenance") + visit help_page_path("administration/raketasks/maintenance") end step 'I should see "Rake Tasks" page markdown rendered' do diff --git a/lib/api/project_hooks.rb b/lib/api/project_hooks.rb index 14f5be3b5f6..dd93a85dc54 100644 --- a/lib/api/project_hooks.rb +++ b/lib/api/project_hooks.rb @@ -47,7 +47,8 @@ module API :build_events, :pipeline_events, :wiki_page_events, - :enable_ssl_verification + :enable_ssl_verification, + :token ] @hook = user_project.hooks.new(attrs) @@ -82,7 +83,8 @@ module API :build_events, :pipeline_events, :wiki_page_events, - :enable_ssl_verification + :enable_ssl_verification, + :token ] if @hook.update_attributes attrs diff --git a/lib/tasks/gitlab/shell.rake b/lib/tasks/gitlab/shell.rake index 210899882b4..58761a129d4 100644 --- a/lib/tasks/gitlab/shell.rake +++ b/lib/tasks/gitlab/shell.rake @@ -63,11 +63,11 @@ namespace :gitlab do # Launch installation process system(*%W(bin/install) + repository_storage_paths_args) - - # (Re)create hooks - system(*%W(bin/create-hooks) + repository_storage_paths_args) end + # (Re)create hooks + Rake::Task['gitlab:shell:create_hooks'].invoke + # Required for debian packaging with PKGR: Setup .ssh/environment with # the current PATH, so that the correct ruby version gets loaded # Requires to set "PermitUserEnvironment yes" in sshd config (should not @@ -102,6 +102,15 @@ namespace :gitlab do end end end + + desc 'Create or repair repository hooks symlink' + task create_hooks: :environment do + warn_user_is_not_gitlab + + puts 'Creating/Repairing hooks symlinks for all repositories' + system(*%W(#{Gitlab.config.gitlab_shell.path}/bin/create-hooks) + repository_storage_paths_args) + puts 'done'.color(:green) + end end def setup diff --git a/spec/features/issues/filter_by_milestone_spec.rb b/spec/features/issues/filter_by_milestone_spec.rb index 88e1549a22b..9dfa5d1de19 100644 --- a/spec/features/issues/filter_by_milestone_spec.rb +++ b/spec/features/issues/filter_by_milestone_spec.rb @@ -11,6 +11,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(Milestone::None.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: 'No Milestone') expect(page).to have_css('.issue', count: 1) end @@ -22,6 +23,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(Milestone::Upcoming.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: 'Upcoming') expect(page).to have_css('.issue', count: 0) end @@ -33,6 +35,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(Milestone::Upcoming.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: 'Upcoming') expect(page).to have_css('.issue', count: 1) end @@ -44,6 +47,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(Milestone::Upcoming.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: 'Upcoming') expect(page).to have_css('.issue', count: 0) end end @@ -55,6 +59,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(milestone.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: milestone.title) expect(page).to have_css('.issue', count: 1) end @@ -70,6 +75,7 @@ feature 'Issue filtering by Milestone', feature: true do visit_issues(project) filter_by_milestone(milestone.title) + expect(page).to have_css('.milestone-filter .dropdown-toggle-text', text: milestone.title) expect(page).to have_css('.issue', count: 1) end end diff --git a/spec/features/projects/builds_spec.rb b/spec/features/projects/builds_spec.rb index d1685f95503..63a23a14f20 100644 --- a/spec/features/projects/builds_spec.rb +++ b/spec/features/projects/builds_spec.rb @@ -216,7 +216,9 @@ describe "Builds" do @build.run! visit namespace_project_build_path(@project.namespace, @project, @build) click_link 'Cancel' - click_link 'Retry' + page.within('.build-header') do + click_link 'Retry build' + end end it 'shows the right status and buttons' do diff --git a/spec/lib/gitlab/git_access_spec.rb b/spec/lib/gitlab/git_access_spec.rb index a5aa387f4f7..62aa212f1f6 100644 --- a/spec/lib/gitlab/git_access_spec.rb +++ b/spec/lib/gitlab/git_access_spec.rb @@ -122,6 +122,14 @@ describe Gitlab::GitAccess, lib: true do describe 'build authentication_abilities permissions' do let(:authentication_abilities) { build_authentication_abilities } + describe 'owner' do + let(:project) { create(:project, namespace: user.namespace) } + + context 'pull code' do + it { expect(subject).to be_allowed } + end + end + describe 'reporter user' do before { project.team << [user, :reporter] } diff --git a/spec/policies/project_policy_spec.rb b/spec/policies/project_policy_spec.rb index 658e3c13a73..96249a7d8c3 100644 --- a/spec/policies/project_policy_spec.rb +++ b/spec/policies/project_policy_spec.rb @@ -6,6 +6,7 @@ describe ProjectPolicy, models: true do let(:dev) { create(:user) } let(:master) { create(:user) } let(:owner) { create(:user) } + let(:admin) { create(:admin) } let(:project) { create(:empty_project, :public, namespace: owner.namespace) } let(:guest_permissions) do @@ -155,6 +156,19 @@ describe ProjectPolicy, models: true do it do is_expected.to include(*guest_permissions) is_expected.to include(*reporter_permissions) + is_expected.to include(*team_member_reporter_permissions) + is_expected.to include(*developer_permissions) + is_expected.to include(*master_permissions) + is_expected.to include(*owner_permissions) + end + end + + context 'admin' do + let(:current_user) { admin } + + it do + is_expected.to include(*guest_permissions) + is_expected.to include(*reporter_permissions) is_expected.not_to include(*team_member_reporter_permissions) is_expected.to include(*developer_permissions) is_expected.to include(*master_permissions) diff --git a/spec/rake_helper.rb b/spec/rake_helper.rb new file mode 100644 index 00000000000..9b5b4bf9fea --- /dev/null +++ b/spec/rake_helper.rb @@ -0,0 +1,19 @@ +require 'spec_helper' +require 'rake' + +RSpec.configure do |config| + config.include RakeHelpers + + # Redirect stdout so specs don't have so much noise + config.before(:all) do + $stdout = StringIO.new + + Rake.application.rake_require 'tasks/gitlab/task_helpers' + Rake::Task.define_task :environment + end + + # Reset stdout + config.after(:all) do + $stdout = STDOUT + end +end diff --git a/spec/requests/api/project_hooks_spec.rb b/spec/requests/api/project_hooks_spec.rb index cfcdcad74cd..5f39329a1b8 100644 --- a/spec/requests/api/project_hooks_spec.rb +++ b/spec/requests/api/project_hooks_spec.rb @@ -88,6 +88,7 @@ describe API::API, 'ProjectHooks', api: true do expect do post api("/projects/#{project.id}/hooks", user), url: "http://example.com", issues_events: true end.to change {project.hooks.count}.by(1) + expect(response).to have_http_status(201) expect(json_response['url']).to eq('http://example.com') expect(json_response['issues_events']).to eq(true) @@ -99,6 +100,24 @@ describe API::API, 'ProjectHooks', api: true do expect(json_response['pipeline_events']).to eq(false) expect(json_response['wiki_page_events']).to eq(false) expect(json_response['enable_ssl_verification']).to eq(true) + expect(json_response).not_to include('token') + end + + it "adds the token without including it in the response" do + token = "secret token" + + expect do + post api("/projects/#{project.id}/hooks", user), url: "http://example.com", token: token + end.to change {project.hooks.count}.by(1) + + expect(response).to have_http_status(201) + expect(json_response["url"]).to eq("http://example.com") + expect(json_response).not_to include("token") + + hook = project.hooks.find(json_response["id"]) + + expect(hook.url).to eq("http://example.com") + expect(hook.token).to eq(token) end it "returns a 400 error if url not given" do @@ -129,6 +148,19 @@ describe API::API, 'ProjectHooks', api: true do expect(json_response['enable_ssl_verification']).to eq(hook.enable_ssl_verification) end + it "adds the token without including it in the response" do + token = "secret token" + + put api("/projects/#{project.id}/hooks/#{hook.id}", user), url: "http://example.org", token: token + + expect(response).to have_http_status(200) + expect(json_response["url"]).to eq("http://example.org") + expect(json_response).not_to include("token") + + expect(hook.reload.url).to eq("http://example.org") + expect(hook.reload.token).to eq(token) + end + it "returns 404 error if hook id not found" do put api("/projects/#{project.id}/hooks/1234", user), url: 'http://example.org' expect(response).to have_http_status(404) diff --git a/spec/requests/lfs_http_spec.rb b/spec/requests/lfs_http_spec.rb index dbdf83a0dff..9bfc84c7425 100644 --- a/spec/requests/lfs_http_spec.rb +++ b/spec/requests/lfs_http_spec.rb @@ -284,7 +284,17 @@ describe 'Git LFS API and storage' do let(:authorization) { authorize_ci_project } shared_examples 'can download LFS only from own projects' do - context 'for own project' do + context 'for owned project' do + let(:project) { create(:empty_project, namespace: user.namespace) } + + let(:update_permissions) do + project.lfs_objects << lfs_object + end + + it_behaves_like 'responds with a file' + end + + context 'for member of project' do let(:pipeline) { create(:ci_empty_pipeline, project: project) } let(:update_permissions) do diff --git a/spec/services/auth/container_registry_authentication_service_spec.rb b/spec/services/auth/container_registry_authentication_service_spec.rb index c64df4979b0..bb26513103d 100644 --- a/spec/services/auth/container_registry_authentication_service_spec.rb +++ b/spec/services/auth/container_registry_authentication_service_spec.rb @@ -245,6 +245,12 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do it_behaves_like 'a pullable' end + + context 'when you are owner' do + let(:project) { create(:empty_project, namespace: current_user.namespace) } + + it_behaves_like 'a pullable' + end end context 'for private' do @@ -266,6 +272,12 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do it_behaves_like 'a pullable' end + + context 'when you are owner' do + let(:project) { create(:empty_project, namespace: current_user.namespace) } + + it_behaves_like 'a pullable' + end end end end @@ -276,13 +288,21 @@ describe Auth::ContainerRegistryAuthenticationService, services: true do end context 'disallow for all' do - let(:project) { create(:empty_project, :public) } + context 'when you are member' do + let(:project) { create(:empty_project, :public) } - before do - project.team << [current_user, :developer] + before do + project.team << [current_user, :developer] + end + + it_behaves_like 'an inaccessible' end - it_behaves_like 'an inaccessible' + context 'when you are owner' do + let(:project) { create(:empty_project, :public, namespace: current_user.namespace) } + + it_behaves_like 'an inaccessible' + end end end end diff --git a/spec/support/rake_helpers.rb b/spec/support/rake_helpers.rb new file mode 100644 index 00000000000..52d80c69835 --- /dev/null +++ b/spec/support/rake_helpers.rb @@ -0,0 +1,10 @@ +module RakeHelpers + def run_rake_task(task_name) + Rake::Task[task_name].reenable + Rake.application.invoke_task task_name + end + + def stub_warn_user_is_not_gitlab + allow_any_instance_of(Object).to receive(:warn_user_is_not_gitlab) + end +end diff --git a/spec/tasks/gitlab/shell_rake_spec.rb b/spec/tasks/gitlab/shell_rake_spec.rb new file mode 100644 index 00000000000..226d34fe2c9 --- /dev/null +++ b/spec/tasks/gitlab/shell_rake_spec.rb @@ -0,0 +1,26 @@ +require 'rake_helper' + +describe 'gitlab:shell rake tasks' do + before do + Rake.application.rake_require 'tasks/gitlab/shell' + + stub_warn_user_is_not_gitlab + end + + describe 'install task' do + it 'invokes create_hooks task' do + expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) + + run_rake_task('gitlab:shell:install') + end + end + + describe 'create_hooks task' do + it 'calls gitlab-shell bin/create_hooks' do + expect_any_instance_of(Object).to receive(:system) + .with("#{Gitlab.config.gitlab_shell.path}/bin/create-hooks", *repository_storage_paths_args) + + run_rake_task('gitlab:shell:create_hooks') + end + end +end |