diff options
| author | Tomasz Maczukin <tomasz@maczukin.pl> | 2015-12-30 17:09:26 +0100 |
|---|---|---|
| committer | Tomasz Maczukin <tomasz@maczukin.pl> | 2015-12-30 17:09:26 +0100 |
| commit | 44dd4782f0f8956670eb76714747252189233ad3 (patch) | |
| tree | 80a70fc83529b77eed8cef6ba60c82dbebf0661c | |
| parent | a17bf380cb4c90696349f268ca4a8c2fedc1f545 (diff) | |
| parent | 333d3d9e5d1fceea97eac58a9e822ab55046a7b4 (diff) | |
| download | gitlab-ce-44dd4782f0f8956670eb76714747252189233ad3.tar.gz | |
Merge branch 'master' into ci/api-builds
* master: (108 commits)
Fix project transfer e-mail sending incorrect paths in e-mail notification
Update CHANGELOG
Use Gitlab::CurrentSettings for InfluxDB
Write to InfluxDB directly via UDP
Strip newlines from obfuscated SQL
Add hotfix that allows to access build artifacts created before 8.3
note votes methids implementation
When reCAPTCHA is disabled, allow registrations to go through without a code
Downcased user or email search for avatar_icon.
Handle missing settings table for metrics
Fix broken link in permissions page [ci skip]
reCAPTCHA is configurable through Admin Settings, no reload needed.
Fixed syntax in gitlab.yml.example
Move InfluxDB settings to ApplicationSetting
Fix spelling mistake, thanks Connor.
Restart settings are moved too.
Hotfix for builds trace data integrity
add issue weight to contributing
Added host option for InfluxDB
Fixed styling of MetricsWorker specs
...
123 files changed, 3221 insertions, 275 deletions
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a8da3de83f8..c23a7a3bf0e 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -12,6 +12,7 @@ before_script: spec:feature: script: + - RAILS_ENV=test bundle exec rake assets:precompile 2>/dev/null - RAILS_ENV=test SIMPLECOV=true bundle exec rake spec:feature tags: - ruby diff --git a/CHANGELOG b/CHANGELOG index 27e6564b49f..425beb17743 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,21 +1,35 @@ Please view this file on the master branch, on stable branches it's out of date. v 8.4.0 (unreleased) - - Fix Error 500 when doing a search in dashboard before visiting any project (Stan Hu) - Implement new UI for group page - Implement search inside emoji picker + - Add API support for looking up a user by username (Stan Hu) - Add project permissions to all project API endpoints (Stan Hu) + - Only allow group/project members to mention `@all` + - Expose Git's version in the admin area - Add "Frequently used" category to emoji picker + - Add CAS support (tduehr) + - Add link to merge request on build detail page + - Revert back upvote and downvote button to the issue and MR pages + - Enable "Add key" button when user fills in a proper key (Stan Hu) + +v 8.3.3 (unreleased) + - Fix project transfer e-mail sending incorrect paths in e-mail notification (Stan Hu) +v 8.3.2 + - Disable --follow in `git log` to avoid loading duplicate commit data in infinite scroll (Stan Hu) + - Add support for Google reCAPTCHA in user registration -v 8.3.1 (unreleased) +v 8.3.1 - Fix Error 500 when global milestones have slashes (Stan Hu) + - Fix Error 500 when doing a search in dashboard before visiting any project (Stan Hu) + - Fix LDAP identity and user retrieval when special characters are used + - Move Sidekiq-cron configuration to gitlab.yml + - Enable forcing Two-Factor authentication sitewide, with optional grace period v 8.3.0 - - Add CAS support (tduehr) - Bump rack-attack to 4.3.1 for security fix (Stan Hu) - API support for starred projects for authorized user (Zeger-Jan van de Weg) - - Add link to merge request on build detail page. - Add open_issues_count to project API (Stan Hu) - Expand character set of usernames created by Omniauth (Corey Hinshaw) - Add button to automatically merge a merge request when the build succeeds (Zeger-Jan van de Weg) @@ -75,7 +89,6 @@ v 8.3.0 - Do not show build status unless builds are enabled and `.gitlab-ci.yml` is present - Persist runners registration token in database - Fix online editor should not remove newlines at the end of the file - - Expose Git's version in the admin area v 8.2.3 - Fix application settings cache not expiring after changes (Stan Hu) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 950824e35ab..b9c2b3d2f8e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -155,6 +155,28 @@ sudo -u git -H bundle exec rake gitlab:env:info) ``` +### Issue weight + +Issue weight allows us to get an idea of the amount of work required to solve +one or multiple issues. This makes it possible to schedule work more accurately. + +You are encouraged to set the weight of any issue. Following the guidelines +below will make it easy to manage this, without unnecessary overhead. + +1. Set weight for any issue at the earliest possible convenience +1. If you don't agree with a set weight, discuss with other developers until +consensus is reached about the weight +1. Issue weights are an abstract measurement of complexity of the issue. Do not +relate issue weight directly to time. This is called [anchoring](https://en.wikipedia.org/wiki/Anchoring) +and something you want to avoid. +1. Something that has a weight of 1 (or no weight) is really small and simple. +Something that is 9 is rewriting a large fundamental part of GitLab, +which might lead to many hard problems to solve. Changing some text in GitLab +is probably 1, adding a new Git Hook maybe 4 or 5, big features 7-9. +1. If something is very large, it should probably be split up in multiple +issues or chunks. You can simply not set the weight of a parent issue and set +weights to children issues. + ## Merge requests We welcome merge requests with fixes and improvements to GitLab code, tests, @@ -35,6 +35,9 @@ gem 'omniauth-twitter', '~> 1.2.0' gem 'omniauth_crowd' gem 'rack-oauth2', '~> 1.2.1' +# reCAPTCHA protection +gem 'recaptcha', require: 'recaptcha/rails' + # Two-factor authentication gem 'devise-two-factor', '~> 2.0.0' gem 'rqrcode-rails3', '~> 0.1.7' @@ -212,9 +215,17 @@ gem 'select2-rails', '~> 3.5.9' gem 'virtus', '~> 1.0.1' gem 'net-ssh', '~> 3.0.1' +# Metrics +group :metrics do + gem 'allocations', '~> 1.0', require: false, platform: :mri + gem 'method_source', '~> 0.8', require: false + gem 'influxdb', '~> 0.2', require: false + gem 'connection_pool', '~> 2.0', require: false +end + group :development do gem "foreman" - gem 'brakeman', '3.0.1', require: false + gem 'brakeman', '~> 3.1.0', require: false gem "annotate", "~> 2.6.0" gem "letter_opener", '~> 1.1.2' diff --git a/Gemfile.lock b/Gemfile.lock index 4f4b10c0fb7..c4cadbafa26 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -49,6 +49,7 @@ GEM addressable (2.3.8) after_commit_queue (1.3.0) activerecord (>= 3.0) + allocations (1.0.1) annotate (2.6.10) activerecord (>= 3.2, <= 4.3) rake (~> 10.4) @@ -65,7 +66,7 @@ GEM attr_encrypted (1.3.4) encryptor (>= 1.3.0) attr_required (1.0.0) - autoprefixer-rails (6.1.1) + autoprefixer-rails (6.1.2) execjs json awesome_print (1.2.0) @@ -84,15 +85,17 @@ GEM bootstrap-sass (3.3.5) autoprefixer-rails (>= 5.0.0.1) sass (>= 3.2.19) - brakeman (3.0.1) + brakeman (3.1.4) erubis (~> 2.6) fastercsv (~> 1.5) haml (>= 3.0, < 5.0) - highline (~> 1.6.20) + highline (>= 1.6.20, < 2.0) multi_json (~> 1.2) - ruby2ruby (~> 2.1.1) - ruby_parser (~> 3.5.0) + ruby2ruby (>= 2.1.1, < 2.3.0) + ruby_parser (~> 3.7.0) + safe_yaml (>= 1.0) sass (~> 3.0) + slim (>= 1.3.6, < 4.0) terminal-table (~> 1.4) browser (1.0.1) builder (3.2.2) @@ -102,7 +105,7 @@ GEM bundler-audit (0.4.0) bundler (~> 1.2) thor (~> 0.18) - byebug (8.2.0) + byebug (8.2.1) cal-heatmap-rails (0.0.1) capybara (2.4.4) mime-types (>= 1.16) @@ -117,6 +120,7 @@ GEM activemodel (>= 3.2.0) activesupport (>= 3.2.0) json (>= 1.7) + cause (0.1) charlock_holmes (0.7.3) chunky_png (1.3.5) cliver (0.3.2) @@ -140,10 +144,10 @@ GEM term-ansicolor (~> 1.3) thor (~> 0.19.1) tins (~> 1.6.0) - crack (0.4.2) + crack (0.4.3) safe_yaml (~> 1.0.0) creole (0.5.0) - d3_rails (3.5.6) + d3_rails (3.5.11) railties (>= 3.1.0) daemons (1.2.3) database_cleaner (1.4.1) @@ -230,7 +234,7 @@ GEM ipaddress (~> 0.5) nokogiri (~> 1.5, >= 1.5.11) opennebula - fog-brightbox (0.9.0) + fog-brightbox (0.10.1) fog-core (~> 1.22) fog-json inflecto (~> 0.0.2) @@ -249,7 +253,7 @@ GEM fog-core (>= 1.21.0) fog-json fog-xml (>= 0.0.1) - fog-sakuracloud (1.4.0) + fog-sakuracloud (1.5.0) fog-core fog-json fog-softlayer (1.0.2) @@ -277,11 +281,11 @@ GEM ruby-progressbar (~> 1.4) gemnasium-gitlab-service (0.2.6) rugged (~> 0.21) - gemojione (2.1.0) + gemojione (2.1.1) json get_process_mem (0.2.0) gherkin-ruby (0.3.2) - github-linguist (4.7.2) + github-linguist (4.7.3) charlock_holmes (~> 0.7.3) escape_utils (~> 1.1.0) mime-types (>= 1.19) @@ -298,7 +302,7 @@ GEM posix-spawn (~> 0.3) gitlab_emoji (0.2.0) gemojione (~> 2.1) - gitlab_git (7.2.21) + gitlab_git (7.2.22) activesupport (~> 4.0) charlock_holmes (~> 0.7.3) github-linguist (~> 4.7.0) @@ -347,7 +351,7 @@ GEM html2haml (>= 1.0.1) railties (>= 4.0.1) hashie (3.4.3) - highline (1.6.21) + highline (1.7.8) hike (1.2.3) hipchat (1.5.2) httparty @@ -370,6 +374,9 @@ GEM i18n (0.7.0) ice_nine (0.11.1) inflecto (0.0.2) + influxdb (0.2.3) + cause + json ipaddress (0.8.0) jquery-atwho-rails (1.3.2) jquery-rails (4.0.5) @@ -417,7 +424,7 @@ GEM net-ldap (0.12.1) net-ssh (3.0.1) netrc (0.11.0) - newrelic-grape (2.0.0) + newrelic-grape (2.1.0) grape newrelic_rpm newrelic_rpm (3.9.4.245) @@ -566,6 +573,8 @@ GEM trollop rdoc (3.12.2) json (~> 1.4) + recaptcha (1.0.2) + json redcarpet (3.3.3) redis (3.2.2) redis-actionpack (4.0.1) @@ -636,10 +645,10 @@ GEM ruby-saml (1.0.0) nokogiri (>= 1.5.10) uuid (~> 2.3) - ruby2ruby (2.1.4) + ruby2ruby (2.2.0) ruby_parser (~> 3.1) sexp_processor (~> 4.0) - ruby_parser (3.5.0) + ruby_parser (3.7.2) sexp_processor (~> 4.1) rubyntlm (0.5.2) rubypants (0.2.0) @@ -693,6 +702,9 @@ GEM tilt (>= 1.3, < 3) six (0.2.0) slack-notifier (1.2.1) + slim (3.0.6) + temple (~> 0.7.3) + tilt (>= 1.3.3, < 2.1) slop (3.6.0) spinach (0.8.10) colorize @@ -734,6 +746,7 @@ GEM railties (>= 3.2.5, < 5) teaspoon-jasmine (2.2.0) teaspoon (>= 1.0.0) + temple (0.7.6) term-ansicolor (1.3.2) tins (~> 1.0) terminal-table (1.5.2) @@ -789,7 +802,7 @@ GEM coercible (~> 1.0) descendants_tracker (~> 0.0, >= 0.0.3) equalizer (~> 0.0, >= 0.0.9) - warden (1.2.3) + warden (1.2.4) rack (>= 1.0) web-console (2.2.1) activemodel (>= 4.0) @@ -820,6 +833,7 @@ DEPENDENCIES acts-as-taggable-on (~> 3.4) addressable (~> 2.3.8) after_commit_queue + allocations (~> 1.0) annotate (~> 2.6.0) asana (~> 0.4.0) asciidoctor (~> 1.5.2) @@ -830,7 +844,7 @@ DEPENDENCIES better_errors (~> 1.0.1) binding_of_caller (~> 0.7.2) bootstrap-sass (~> 3.0) - brakeman (= 3.0.1) + brakeman (~> 3.1.0) browser (~> 1.0.0) bullet bundler-audit @@ -842,6 +856,7 @@ DEPENDENCIES charlock_holmes (~> 0.7.3) coffee-rails (~> 4.1.0) colorize (~> 0.7.0) + connection_pool (~> 2.0) coveralls (~> 0.8.2) creole (~> 0.5.0) d3_rails (~> 3.5.5) @@ -879,6 +894,7 @@ DEPENDENCIES hipchat (~> 1.5.0) html-pipeline (~> 1.11.0) httparty (~> 0.13.3) + influxdb (~> 0.2) jquery-atwho-rails (~> 1.3.2) jquery-rails (~> 4.0.0) jquery-scrollto-rails (~> 1.4.3) @@ -887,6 +903,7 @@ DEPENDENCIES kaminari (~> 0.16.3) letter_opener (~> 1.1.2) mail_room (~> 0.6.1) + method_source (~> 0.8) minitest (~> 5.7.0) mousetrap-rails (~> 1.4.6) mysql2 (~> 0.3.16) @@ -924,6 +941,7 @@ DEPENDENCIES raphael-rails (~> 2.1.2) rblineprof rdoc (~> 3.6) + recaptcha redcarpet (~> 3.3.3) redis-namespace redis-rails (~> 4.0.0) diff --git a/app/assets/images/brand_logo.png b/app/assets/images/brand_logo.png Binary files differdeleted file mode 100644 index 9c564bb6141..00000000000 --- a/app/assets/images/brand_logo.png +++ /dev/null diff --git a/app/assets/images/gitlab_logo.png b/app/assets/images/gitlab_logo.png Binary files differnew file mode 100644 index 00000000000..0c157546b9c --- /dev/null +++ b/app/assets/images/gitlab_logo.png diff --git a/app/assets/javascripts/awards_handler.coffee b/app/assets/javascripts/awards_handler.coffee index 04bf5cc7bb5..619abb1fb07 100644 --- a/app/assets/javascripts/awards_handler.coffee +++ b/app/assets/javascripts/awards_handler.coffee @@ -43,15 +43,19 @@ class @AwardsHandler decrementCounter: (emoji) -> counter = @findEmojiIcon(emoji).siblings(".counter") + emojiIcon = counter.parent() if parseInt(counter.text()) > 1 counter.text(parseInt(counter.text()) - 1) - counter.parent().removeClass("active") + emojiIcon.removeClass("active") @removeMeFromAuthorList(emoji) + else if emoji =="thumbsup" || emoji == "thumbsdown" + emojiIcon.tooltip("destroy") + counter.text(0) + emojiIcon.removeClass("active") else - award = counter.parent() - award.tooltip("destroy") - award.remove() + emojiIcon.tooltip("destroy") + emojiIcon.remove() removeMeFromAuthorList: (emoji) -> award_block = @findEmojiIcon(emoji).parent() @@ -127,21 +131,19 @@ class @AwardsHandler getFrequentlyUsedEmojis: -> frequently_used_emojis = ($.cookie('frequently_used_emojis') || "").split(",") - - frequently_used_emojis = ["thumbsup", "thumbsdown"].concat(frequently_used_emojis) - _.compact(_.uniq(frequently_used_emojis)) renderFrequentlyUsedBlock: -> - frequently_used_emojis = @getFrequentlyUsedEmojis() + if $.cookie('frequently_used_emojis') + frequently_used_emojis = @getFrequentlyUsedEmojis() - ul = $("<ul>") + ul = $("<ul>") - for emoji in frequently_used_emojis - do (emoji) -> - $(".emoji-menu-content [data-emoji='#{emoji}']").closest("li").clone().appendTo(ul) + for emoji in frequently_used_emojis + do (emoji) -> + $(".emoji-menu-content [data-emoji='#{emoji}']").closest("li").clone().appendTo(ul) - $("input.emoji-search").after(ul).after($("<h5>").text("Frequently used")) + $("input.emoji-search").after(ul).after($("<h5>").text("Frequently used")) setupSearch: -> $("input.emoji-search").keyup (ev) => diff --git a/app/assets/javascripts/dispatcher.js.coffee b/app/assets/javascripts/dispatcher.js.coffee index 599b4c49540..69e061ce6e9 100644 --- a/app/assets/javascripts/dispatcher.js.coffee +++ b/app/assets/javascripts/dispatcher.js.coffee @@ -49,7 +49,7 @@ class Dispatcher new DropzoneInput($('.release-form')) when 'projects:merge_requests:show' new Diff() - shortcut_handler = new ShortcutsIssuable() + shortcut_handler = new ShortcutsIssuable(true) new ZenMode() when "projects:merge_requests:diffs" new Diff() diff --git a/app/assets/javascripts/issue.js.coffee b/app/assets/javascripts/issue.js.coffee index eff80bf63bb..c256ec8f41b 100644 --- a/app/assets/javascripts/issue.js.coffee +++ b/app/assets/javascripts/issue.js.coffee @@ -1,3 +1,4 @@ +#= require flash #= require jquery.waitforimages #= require task_list @@ -6,13 +7,44 @@ class @Issue # Prevent duplicate event bindings @disableTaskList() - if $("a.btn-close").length + if $('a.btn-close').length @initTaskList() + @initIssueBtnEventListeners() initTaskList: -> $('.detail-page-description .js-task-list-container').taskList('enable') $(document).on 'tasklist:changed', '.detail-page-description .js-task-list-container', @updateTaskList + initIssueBtnEventListeners: -> + issueFailMessage = 'Unable to update this issue at this time.' + $('a.btn-close, a.btn-reopen').on 'click', (e) -> + e.preventDefault() + e.stopImmediatePropagation() + $this = $(this) + isClose = $this.hasClass('btn-close') + $this.prop('disabled', true) + url = $this.attr('href') + $.ajax + type: 'PUT' + url: url, + error: (jqXHR, textStatus, errorThrown) -> + issueStatus = if isClose then 'close' else 'open' + new Flash(issueFailMessage, 'alert') + success: (data, textStatus, jqXHR) -> + if data.saved + $this.addClass('hidden') + if isClose + $('a.btn-reopen').removeClass('hidden') + $('div.status-box-closed').removeClass('hidden') + $('div.status-box-open').addClass('hidden') + else + $('a.btn-close').removeClass('hidden') + $('div.status-box-closed').addClass('hidden') + $('div.status-box-open').removeClass('hidden') + else + new Flash(issueFailMessage, 'alert') + $this.prop('disabled', false) + disableTaskList: -> $('.detail-page-description .js-task-list-container').taskList('disable') $(document).off 'tasklist:changed', '.detail-page-description .js-task-list-container' diff --git a/app/assets/javascripts/shortcuts.js.coffee b/app/assets/javascripts/shortcuts.js.coffee index e9aeb1e9525..4d915bfc8c5 100644 --- a/app/assets/javascripts/shortcuts.js.coffee +++ b/app/assets/javascripts/shortcuts.js.coffee @@ -7,7 +7,7 @@ class @Shortcuts selectiveHelp: (e) => Shortcuts.showHelp(e, @enabledHelp) - + @showHelp: (e, location) -> if $('#modal-shortcuts').length > 0 $('#modal-shortcuts').modal('show') @@ -17,8 +17,7 @@ class @Shortcuts dataType: 'script', success: (e) -> if location and location.length > 0 - for l in location - $(l).show() + $(l).show() for l in location else $('.hidden-shortcut').show() $('.js-more-help-button').remove() @@ -28,3 +27,8 @@ class @Shortcuts @focusSearch: (e) -> $('#search').focus() e.preventDefault() + +$(document).on 'click.more_help', '.js-more-help-button', (e) -> + $(@).remove() + $('.hidden-shortcut').show() + e.preventDefault() diff --git a/app/assets/javascripts/users_select.js.coffee b/app/assets/javascripts/users_select.js.coffee index 12abf806bfa..9467011799f 100644 --- a/app/assets/javascripts/users_select.js.coffee +++ b/app/assets/javascripts/users_select.js.coffee @@ -117,5 +117,5 @@ class @UsersSelect callback(users) buildUrl: (url) -> - url = gon.relative_url_root + url if gon.relative_url_root? + url = gon.relative_url_root.replace(/\/$/, '') + url if gon.relative_url_root? return url diff --git a/app/controllers/admin/application_settings_controller.rb b/app/controllers/admin/application_settings_controller.rb index 9dd16f8c735..10e736fd362 100644 --- a/app/controllers/admin/application_settings_controller.rb +++ b/app/controllers/admin/application_settings_controller.rb @@ -49,6 +49,8 @@ class Admin::ApplicationSettingsController < Admin::ApplicationController :default_branch_protection, :signup_enabled, :signin_enabled, + :require_two_factor_authentication, + :two_factor_grace_period, :gravatar_enabled, :twitter_sharing_enabled, :sign_in_text, @@ -65,6 +67,17 @@ class Admin::ApplicationSettingsController < Admin::ApplicationController :user_oauth_applications, :shared_runners_enabled, :max_artifacts_size, + :metrics_enabled, + :metrics_host, + :metrics_port, + :metrics_username, + :metrics_password, + :metrics_pool_size, + :metrics_timeout, + :metrics_method_call_threshold, + :recaptcha_enabled, + :recaptcha_site_key, + :recaptcha_private_key, restricted_visibility_levels: [], import_sources: [] ) diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index 01e2e7b2f98..d9a37a4d45f 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -13,6 +13,7 @@ class ApplicationController < ActionController::Base before_action :validate_user_service_ticket! before_action :reject_blocked! before_action :check_password_expiration + before_action :check_2fa_requirement before_action :ldap_security_check before_action :default_headers before_action :add_gon_variables @@ -223,6 +224,12 @@ class ApplicationController < ActionController::Base end end + def check_2fa_requirement + if two_factor_authentication_required? && current_user && !current_user.two_factor_enabled && !skip_two_factor? + redirect_to new_profile_two_factor_auth_path + end + end + def ldap_security_check if current_user && current_user.requires_ldap_check? unless Gitlab::LDAP::Access.allowed?(current_user) @@ -357,6 +364,23 @@ class ApplicationController < ActionController::Base current_application_settings.import_sources.include?('git') end + def two_factor_authentication_required? + current_application_settings.require_two_factor_authentication + end + + def two_factor_grace_period + current_application_settings.two_factor_grace_period + end + + def two_factor_grace_period_expired? + date = current_user.otp_grace_period_started_at + date && (date + two_factor_grace_period.hours) < Time.current + end + + def skip_two_factor? + session[:skip_tfa] && session[:skip_tfa] > Time.current + end + def redirect_to_home_page_url? # If user is not signed-in and tries to access root_path - redirect him to landing page # Don't redirect to the default URL to prevent endless redirections diff --git a/app/controllers/profiles/two_factor_auths_controller.rb b/app/controllers/profiles/two_factor_auths_controller.rb index e6b99be37fb..6e91d9b4ad9 100644 --- a/app/controllers/profiles/two_factor_auths_controller.rb +++ b/app/controllers/profiles/two_factor_auths_controller.rb @@ -1,8 +1,22 @@ class Profiles::TwoFactorAuthsController < Profiles::ApplicationController + skip_before_action :check_2fa_requirement + def new unless current_user.otp_secret current_user.otp_secret = User.generate_otp_secret(32) - current_user.save! + end + + unless current_user.otp_grace_period_started_at && two_factor_grace_period + current_user.otp_grace_period_started_at = Time.current + end + + current_user.save! if current_user.changed? + + if two_factor_grace_period_expired? + flash.now[:alert] = 'You must configure Two-Factor Authentication in your account.' + else + grace_period_deadline = current_user.otp_grace_period_started_at + two_factor_grace_period.hours + flash.now[:alert] = "You must configure Two-Factor Authentication in your account until #{l(grace_period_deadline)}." end @qr_code = build_qr_code @@ -34,6 +48,15 @@ class Profiles::TwoFactorAuthsController < Profiles::ApplicationController redirect_to profile_account_path end + def skip + if two_factor_grace_period_expired? + redirect_to new_profile_two_factor_auth_path, alert: 'Cannot skip two factor authentication setup' + else + session[:skip_tfa] = current_user.otp_grace_period_started_at + two_factor_grace_period.hours + redirect_to root_path + end + end + private def build_qr_code diff --git a/app/controllers/projects/commits_controller.rb b/app/controllers/projects/commits_controller.rb index 58fb946dbc2..04a88990bf4 100644 --- a/app/controllers/projects/commits_controller.rb +++ b/app/controllers/projects/commits_controller.rb @@ -9,7 +9,7 @@ class Projects::CommitsController < Projects::ApplicationController def show @repo = @project.repository - @limit, @offset = (params[:limit] || 40), (params[:offset] || 0) + @limit, @offset = (params[:limit] || 40).to_i, (params[:offset] || 0).to_i @commits = @repo.commits(@ref, @path, @limit, @offset) @note_counts = project.notes.where(commit_id: @commits.map(&:id)). diff --git a/app/controllers/projects_controller.rb b/app/controllers/projects_controller.rb index 2dab04f2a7c..3004722bce0 100644 --- a/app/controllers/projects_controller.rb +++ b/app/controllers/projects_controller.rb @@ -178,7 +178,7 @@ class ProjectsController < ApplicationController def markdown_preview text = params[:text] - ext = Gitlab::ReferenceExtractor.new(@project, current_user) + ext = Gitlab::ReferenceExtractor.new(@project, current_user, current_user) ext.analyze(text) render json: { diff --git a/app/controllers/registrations_controller.rb b/app/controllers/registrations_controller.rb index 3b3dc86cb68..c48175a4c5a 100644 --- a/app/controllers/registrations_controller.rb +++ b/app/controllers/registrations_controller.rb @@ -1,10 +1,21 @@ class RegistrationsController < Devise::RegistrationsController before_action :signup_enabled? + include Recaptcha::Verify def new redirect_to(new_user_session_path) end + def create + if !Gitlab::Recaptcha.load_configurations! || verify_recaptcha + super + else + flash[:alert] = "There was an error with the reCAPTCHA code below. Please re-enter the code." + flash.delete :recaptcha_error + render action: 'new' + end + end + def destroy DeleteUserService.new(current_user).execute(current_user) @@ -38,4 +49,16 @@ class RegistrationsController < Devise::RegistrationsController def sign_up_params params.require(:user).permit(:username, :email, :name, :password, :password_confirmation) end + + def resource_name + :user + end + + def resource + @resource ||= User.new(sign_up_params) + end + + def devise_mapping + @devise_mapping ||= Devise.mappings[:user] + end end diff --git a/app/controllers/sessions_controller.rb b/app/controllers/sessions_controller.rb index 1b60d3e27d0..825f85199be 100644 --- a/app/controllers/sessions_controller.rb +++ b/app/controllers/sessions_controller.rb @@ -1,9 +1,11 @@ class SessionsController < Devise::SessionsController include AuthenticatesWithTwoFactor + include Recaptcha::ClientHelper prepend_before_action :authenticate_with_two_factor, only: [:create] prepend_before_action :store_redirect_path, only: [:new] before_action :auto_sign_in_with_provider, only: [:new] + before_action :load_recaptcha def new if Gitlab.config.ldap.enabled @@ -40,7 +42,7 @@ class SessionsController < Devise::SessionsController User.find(session[:otp_user_id]) end end - + def store_redirect_path redirect_path = if request.referer.present? && (params['redirect_to_referer'] == 'yes') @@ -87,14 +89,14 @@ class SessionsController < Devise::SessionsController provider = Gitlab.config.omniauth.auto_sign_in_with_provider return unless provider.present? - # Auto sign in with an Omniauth provider only if the standard "you need to sign-in" alert is - # registered or no alert at all. In case of another alert (such as a blocked user), it is safer + # Auto sign in with an Omniauth provider only if the standard "you need to sign-in" alert is + # registered or no alert at all. In case of another alert (such as a blocked user), it is safer # to do nothing to prevent redirection loops with certain Omniauth providers. return unless flash[:alert].blank? || flash[:alert] == I18n.t('devise.failure.unauthenticated') - + # Prevent alert from popping up on the first page shown after authentication. - flash[:alert] = nil - + flash[:alert] = nil + redirect_to user_omniauth_authorize_path(provider.to_sym) end @@ -107,4 +109,8 @@ class SessionsController < Devise::SessionsController AuditEventService.new(user, user, options). for_authentication.security_event end + + def load_recaptcha + Gitlab::Recaptcha.load_configurations! + end end diff --git a/app/helpers/application_helper.rb b/app/helpers/application_helper.rb index 0b00b9a0702..f7f7a1a02d3 100644 --- a/app/helpers/application_helper.rb +++ b/app/helpers/application_helper.rb @@ -72,7 +72,7 @@ module ApplicationHelper if user_or_email.is_a?(User) user = user_or_email else - user = User.find_by(email: user_or_email) + user = User.find_by(email: user_or_email.downcase) end if user diff --git a/app/helpers/auth_helper.rb b/app/helpers/auth_helper.rb index 2c81ea1623c..0cfc0565e84 100644 --- a/app/helpers/auth_helper.rb +++ b/app/helpers/auth_helper.rb @@ -50,5 +50,17 @@ module AuthHelper current_user.identities.exists?(provider: provider.to_s) end + def two_factor_skippable? + current_application_settings.require_two_factor_authentication && + !current_user.two_factor_enabled && + current_application_settings.two_factor_grace_period && + !two_factor_grace_period_expired? + end + + def two_factor_grace_period_expired? + current_user.otp_grace_period_started_at && + (current_user.otp_grace_period_started_at + current_application_settings.two_factor_grace_period.hours) < Time.current + end + extend self end diff --git a/app/helpers/issues_helper.rb b/app/helpers/issues_helper.rb index 4fe84322199..80e2741b09a 100644 --- a/app/helpers/issues_helper.rb +++ b/app/helpers/issues_helper.rb @@ -69,6 +69,10 @@ module IssuesHelper end end + def issue_button_visibility(issue, closed) + return 'hidden' if issue.closed? == closed + end + def issue_to_atom(xml, issue) xml.entry do xml.id namespace_project_issue_url(issue.project.namespace, @@ -120,6 +124,18 @@ module IssuesHelper end end + def awards_sort(awards) + awards.sort_by do |award, notes| + if award == "thumbsup" + 0 + elsif award == "thumbsdown" + 1 + else + 2 + end + end.to_h + end + # Required for Banzai::Filter::IssueReferenceFilter module_function :url_for_issue end diff --git a/app/helpers/page_layout_helper.rb b/app/helpers/page_layout_helper.rb index 9bf750124b2..791cb9e50bd 100644 --- a/app/helpers/page_layout_helper.rb +++ b/app/helpers/page_layout_helper.rb @@ -8,6 +8,80 @@ module PageLayoutHelper @page_title.join(" \u00b7 ") end + # Define or get a description for the current page + # + # description - String (default: nil) + # + # If this helper is called multiple times with an argument, only the last + # description will be returned when called without an argument. Descriptions + # have newlines replaced with spaces and all HTML tags are sanitized. + # + # Examples: + # + # page_description # => "GitLab Community Edition" + # page_description("Foo") + # page_description # => "Foo" + # + # page_description("<b>Bar</b>\nBaz") + # page_description # => "Bar Baz" + # + # Returns an HTML-safe String. + def page_description(description = nil) + @page_description ||= page_description_default + + if description.present? + @page_description = description.squish + else + sanitize(@page_description, tags: []).truncate_words(30) + end + end + + # Default value for page_description when one hasn't been defined manually by + # a view + def page_description_default + if @project + @project.description || brand_title + else + brand_title + end + end + + def page_image + default = image_url('gitlab_logo.png') + + if @project + @project.avatar_url || default + elsif @user + avatar_icon(@user) + else + default + end + end + + # Define or get attributes to be used as Twitter card metadata + # + # map - Hash of label => data pairs. Keys become labels, values become data + # + # Raises ArgumentError if given more than two attributes + def page_card_attributes(map = {}) + raise ArgumentError, 'cannot provide more than two attributes' if map.length > 2 + + @page_card_attributes ||= {} + @page_card_attributes = map.reject { |_,v| v.blank? } if map.present? + @page_card_attributes + end + + def page_card_meta_tags + tags = '' + + page_card_attributes.each_with_index do |pair, i| + tags << tag(:meta, property: "twitter:label#{i + 1}", content: pair[0]) + tags << tag(:meta, property: "twitter:data#{i + 1}", content: pair[1]) + end + + tags.html_safe + end + def header_title(title = nil, title_url = nil) if title @header_title = title diff --git a/app/models/ability.rb b/app/models/ability.rb index cd5ae0fb0fd..1b3ee757040 100644 --- a/app/models/ability.rb +++ b/app/models/ability.rb @@ -132,14 +132,14 @@ class Ability end def public_project_rules - project_guest_rules + [ + @public_project_rules ||= project_guest_rules + [ :download_code, :fork_project ] end def project_guest_rules - [ + @project_guest_rules ||= [ :read_project, :read_wiki, :read_issue, @@ -157,7 +157,7 @@ class Ability end def project_report_rules - project_guest_rules + [ + @project_report_rules ||= project_guest_rules + [ :create_commit_status, :read_commit_statuses, :download_code, @@ -170,7 +170,7 @@ class Ability end def project_dev_rules - project_report_rules + [ + @project_dev_rules ||= project_report_rules + [ :admin_merge_request, :create_merge_request, :create_wiki, @@ -181,7 +181,7 @@ class Ability end def project_archived_rules - [ + @project_archived_rules ||= [ :create_merge_request, :push_code, :push_code_to_protected_branches, @@ -191,7 +191,7 @@ class Ability end def project_master_rules - project_dev_rules + [ + @project_master_rules ||= project_dev_rules + [ :push_code_to_protected_branches, :update_project_snippet, :update_merge_request, @@ -206,7 +206,7 @@ class Ability end def project_admin_rules - project_master_rules + [ + @project_admin_rules ||= project_master_rules + [ :change_namespace, :change_visibility_level, :rename_project, @@ -332,7 +332,7 @@ class Ability end if snippet.public? || snippet.internal? - rules << :read_personal_snippet + rules << :read_personal_snippet end rules diff --git a/app/models/application_setting.rb b/app/models/application_setting.rb index 724429e7558..be69d317d73 100644 --- a/app/models/application_setting.rb +++ b/app/models/application_setting.rb @@ -2,32 +2,34 @@ # # Table name: application_settings # -# id :integer not null, primary key -# default_projects_limit :integer -# signup_enabled :boolean -# signin_enabled :boolean -# gravatar_enabled :boolean -# sign_in_text :text -# created_at :datetime -# updated_at :datetime -# home_page_url :string(255) -# default_branch_protection :integer default(2) -# twitter_sharing_enabled :boolean default(TRUE) -# restricted_visibility_levels :text -# version_check_enabled :boolean default(TRUE) -# max_attachment_size :integer default(10), not null -# default_project_visibility :integer -# default_snippet_visibility :integer -# restricted_signup_domains :text -# user_oauth_applications :boolean default(TRUE) -# after_sign_out_path :string(255) -# session_expire_delay :integer default(10080), not null -# import_sources :text -# help_page_text :text -# admin_notification_email :string(255) -# shared_runners_enabled :boolean default(TRUE), not null -# max_artifacts_size :integer default(100), not null -# runners_registration_token :string(255) +# id :integer not null, primary key +# default_projects_limit :integer +# signup_enabled :boolean +# signin_enabled :boolean +# gravatar_enabled :boolean +# sign_in_text :text +# created_at :datetime +# updated_at :datetime +# home_page_url :string(255) +# default_branch_protection :integer default(2) +# twitter_sharing_enabled :boolean default(TRUE) +# restricted_visibility_levels :text +# version_check_enabled :boolean default(TRUE) +# max_attachment_size :integer default(10), not null +# default_project_visibility :integer +# default_snippet_visibility :integer +# restricted_signup_domains :text +# user_oauth_applications :boolean default(TRUE) +# after_sign_out_path :string(255) +# session_expire_delay :integer default(10080), not null +# import_sources :text +# help_page_text :text +# admin_notification_email :string(255) +# shared_runners_enabled :boolean default(TRUE), not null +# max_artifacts_size :integer default(100), not null +# runners_registration_token :string(255) +# require_two_factor_authentication :boolean default(TRUE) +# two_factor_grace_period :integer default(48) # class ApplicationSetting < ActiveRecord::Base @@ -42,21 +44,32 @@ class ApplicationSetting < ActiveRecord::Base attr_accessor :restricted_signup_domains_raw validates :session_expire_delay, - presence: true, - numericality: { only_integer: true, greater_than_or_equal_to: 0 } + presence: true, + numericality: { only_integer: true, greater_than_or_equal_to: 0 } validates :home_page_url, - allow_blank: true, - url: true, - if: :home_page_url_column_exist + allow_blank: true, + url: true, + if: :home_page_url_column_exist validates :after_sign_out_path, - allow_blank: true, - url: true + allow_blank: true, + url: true validates :admin_notification_email, - allow_blank: true, - email: true + allow_blank: true, + email: true + + validates :two_factor_grace_period, + numericality: { greater_than_or_equal_to: 0 } + + validates :recaptcha_site_key, + presence: true, + if: :recaptcha_enabled + + validates :recaptcha_private_key, + presence: true, + if: :recaptcha_enabled validates_each :restricted_visibility_levels do |record, attr, value| unless value.nil? @@ -112,6 +125,8 @@ class ApplicationSetting < ActiveRecord::Base import_sources: ['github','bitbucket','gitlab','gitorious','google_code','fogbugz','git'], shared_runners_enabled: Settings.gitlab_ci['shared_runners_enabled'], max_artifacts_size: Settings.artifacts['max_size'], + require_two_factor_authentication: false, + two_factor_grace_period: 48 ) end diff --git a/app/models/ci/build.rb b/app/models/ci/build.rb index 7b89fe069ea..3e67b2771c1 100644 --- a/app/models/ci/build.rb +++ b/app/models/ci/build.rb @@ -194,8 +194,11 @@ module Ci end def raw_trace - if File.exist?(path_to_trace) + if File.file?(path_to_trace) File.read(path_to_trace) + elsif project.ci_id && File.file?(old_path_to_trace) + # Temporary fix for build trace data integrity + File.read(old_path_to_trace) else # backward compatibility read_attribute :trace @@ -212,8 +215,8 @@ module Ci end def trace=(trace) - unless Dir.exists? dir_to_trace - FileUtils.mkdir_p dir_to_trace + unless Dir.exists?(dir_to_trace) + FileUtils.mkdir_p(dir_to_trace) end File.write(path_to_trace, trace) @@ -231,6 +234,55 @@ module Ci "#{dir_to_trace}/#{id}.log" end + ## + # Deprecated + # + # This is a hotfix for CI build data integrity, see #4246 + # Should be removed in 8.4, after CI files migration has been done. + # + def old_dir_to_trace + File.join( + Settings.gitlab_ci.builds_path, + created_at.utc.strftime("%Y_%m"), + project.ci_id.to_s + ) + end + + ## + # Deprecated + # + # This is a hotfix for CI build data integrity, see #4246 + # Should be removed in 8.4, after CI files migration has been done. + # + def old_path_to_trace + "#{old_dir_to_trace}/#{id}.log" + end + + ## + # Deprecated + # + # This contains a hotfix for CI build data integrity, see #4246 + # + # This method is used by `ArtifactUploader` to create a store_dir. + # Warning: Uploader uses it after AND before file has been stored. + # + # This method returns old path to artifacts only if it already exists. + # + def artifacts_path + old = File.join(created_at.utc.strftime('%Y_%m'), + project.ci_id.to_s, + id.to_s) + + old_store = File.join(ArtifactUploader.artifacts_path, old) + return old if project.ci_id && File.directory?(old_store) + + File.join( + created_at.utc.strftime('%Y_%m'), + project.id.to_s, + id.to_s + ) + end + def token project.runners_token end diff --git a/app/models/concerns/issuable.rb b/app/models/concerns/issuable.rb index f56fd3e02d4..18a00f95b48 100644 --- a/app/models/concerns/issuable.rb +++ b/app/models/concerns/issuable.rb @@ -95,14 +95,12 @@ module Issuable opened? || reopened? end - # Deprecated. Still exists to preserve API compatibility. def downvotes - 0 + notes.awards.where(note: "thumbsdown").count end - # Deprecated. Still exists to preserve API compatibility. def upvotes - 0 + notes.awards.where(note: "thumbsup").count end def subscribed?(user) @@ -161,6 +159,14 @@ module Issuable self.class.to_s.underscore end + # Returns a Hash of attributes to be used for Twitter card metadata + def card_attributes + { + 'Author' => author.try(:name), + 'Assignee' => assignee.try(:name) + } + end + def notes_with_associations notes.includes(:author, :project) end diff --git a/app/models/concerns/mentionable.rb b/app/models/concerns/mentionable.rb index 1fdcda97520..6316ee208b5 100644 --- a/app/models/concerns/mentionable.rb +++ b/app/models/concerns/mentionable.rb @@ -44,7 +44,7 @@ module Mentionable end def all_references(current_user = self.author, text = nil) - ext = Gitlab::ReferenceExtractor.new(self.project, current_user) + ext = Gitlab::ReferenceExtractor.new(self.project, current_user, self.author) if text ext.analyze(text) diff --git a/app/models/note.rb b/app/models/note.rb index 8c5b5836f9a..3d5b663c99f 100644 --- a/app/models/note.rb +++ b/app/models/note.rb @@ -107,9 +107,16 @@ class Note < ActiveRecord::Base end def grouped_awards + notes = {} + awards.select(:note).distinct.map do |note| - [ note.note, where(note: note.note) ] + notes[note.note] = where(note: note.note) end + + notes["thumbsup"] ||= Note.none + notes["thumbsdown"] ||= Note.none + + notes end end @@ -339,14 +346,12 @@ class Note < ActiveRecord::Base read_attribute(:system) end - # Deprecated. Still exists to preserve API compatibility. def downvote? - false + is_award && note == "thumbsdown" end - # Deprecated. Still exists to preserve API compatibility. def upvote? - false + is_award && note == "thumbsup" end def editable? diff --git a/app/models/project.rb b/app/models/project.rb index 75f85310d5f..017471995ec 100644 --- a/app/models/project.rb +++ b/app/models/project.rb @@ -555,7 +555,9 @@ class Project < ActiveRecord::Base end def send_move_instructions(old_path_with_namespace) - NotificationService.new.project_was_moved(self, old_path_with_namespace) + # New project path needs to be committed to the DB or notification will + # retrieve stale information + run_after_commit { NotificationService.new.project_was_moved(self, old_path_with_namespace) } end def owner diff --git a/app/models/repository.rb b/app/models/repository.rb index 9f688e3b45b..a9bf4eb4033 100644 --- a/app/models/repository.rb +++ b/app/models/repository.rb @@ -76,7 +76,9 @@ class Repository path: path, limit: limit, offset: offset, - follow: path.present? + # --follow doesn't play well with --skip. See: + # https://gitlab.com/gitlab-org/gitlab-ce/issues/3574#note_3040520 + follow: false } commits = Gitlab::Git::Commit.where(options) diff --git a/app/uploaders/artifact_uploader.rb b/app/uploaders/artifact_uploader.rb index 1dccc39e7e2..1b0ae6c0056 100644 --- a/app/uploaders/artifact_uploader.rb +++ b/app/uploaders/artifact_uploader.rb @@ -20,16 +20,12 @@ class ArtifactUploader < CarrierWave::Uploader::Base @build, @field = build, field end - def artifacts_path - File.join(build.created_at.utc.strftime('%Y_%m'), build.project.id.to_s, build.id.to_s) - end - def store_dir - File.join(ArtifactUploader.artifacts_path, artifacts_path) + File.join(self.class.artifacts_path, @build.artifacts_path) end def cache_dir - File.join(ArtifactUploader.artifacts_cache_path, artifacts_path) + File.join(self.class.artifacts_cache_path, @build.artifacts_path) end def file_storage? diff --git a/app/views/admin/application_settings/_form.html.haml b/app/views/admin/application_settings/_form.html.haml index 6c355366948..214e0209bb7 100644 --- a/app/views/admin/application_settings/_form.html.haml +++ b/app/views/admin/application_settings/_form.html.haml @@ -105,6 +105,18 @@ = f.check_box :signin_enabled Sign-in enabled .form-group + = f.label :two_factor_authentication, 'Two-Factor authentication', class: 'control-label col-sm-2' + .col-sm-10 + .checkbox + = f.label :require_two_factor_authentication do + = f.check_box :require_two_factor_authentication + Require all users to setup Two-Factor authentication + .form-group + = f.label :two_factor_authentication, 'Two-Factor grace period (hours)', class: 'control-label col-sm-2' + .col-sm-10 + = f.number_field :two_factor_grace_period, min: 0, class: 'form-control', placeholder: '0' + .help-block Amount of time (in hours) that users are allowed to skip forced configuration of two-factor authentication + .form-group = f.label :restricted_signup_domains, 'Restricted domains for sign-ups', class: 'control-label col-sm-2' .col-sm-10 = f.text_area :restricted_signup_domains_raw, placeholder: 'domain.com', class: 'form-control' @@ -144,5 +156,82 @@ .col-sm-10 = f.number_field :max_artifacts_size, class: 'form-control' + %fieldset + %legend Metrics + %p + These settings require a restart to take effect. + .form-group + .col-sm-offset-2.col-sm-10 + .checkbox + = f.label :metrics_enabled do + = f.check_box :metrics_enabled + Enable InfluxDB Metrics + .form-group + = f.label :metrics_host, 'InfluxDB host', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :metrics_host, class: 'form-control', placeholder: 'influxdb.example.com' + .form-group + = f.label :metrics_port, 'InfluxDB port', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :metrics_port, class: 'form-control', placeholder: '8089' + .help-block + The UDP port to use for connecting to InfluxDB. InfluxDB requires that + your server configuration specifies a database to store data in when + sending messages to this port, without it metrics data will not be + saved. + .form-group + = f.label :metrics_username, 'InfluxDB username', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :metrics_username, class: 'form-control' + .form-group + = f.label :metrics_password, 'InfluxDB password', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :metrics_password, class: 'form-control' + .form-group + = f.label :metrics_pool_size, 'Connection pool size', class: 'control-label col-sm-2' + .col-sm-10 + = f.number_field :metrics_pool_size, class: 'form-control' + .help-block + The amount of InfluxDB connections to open. Connections are opened + lazily. Users using multi-threaded application servers should ensure + enough connections are available (at minimum the amount of application + server threads). + .form-group + = f.label :metrics_timeout, 'Connection timeout', class: 'control-label col-sm-2' + .col-sm-10 + = f.number_field :metrics_timeout, class: 'form-control' + .help-block + The amount of seconds after which an InfluxDB connection will time + out. + .form-group + = f.label :metrics_method_call_threshold, 'Method Call Threshold (ms)', class: 'control-label col-sm-2' + .col-sm-10 + = f.number_field :metrics_method_call_threshold, class: 'form-control' + .help-block + A method call is only tracked when it takes longer to complete than + the given amount of milliseconds. + + %fieldset + %legend Spam and Anti-bot Protection + .form-group + .col-sm-offset-2.col-sm-10 + .checkbox + = f.label :recaptcha_enabled do + = f.check_box :recaptcha_enabled + Enable reCAPTCHA + %span.help-block#recaptcha_help_block Helps preventing bots from creating accounts + + .form-group + = f.label :recaptcha_site_key, 'reCAPTCHA Site Key', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :recaptcha_site_key, class: 'form-control' + .help-block + Generate site and private keys here: + %a{ href: 'http://www.google.com/recaptcha', target: 'blank'} http://www.google.com/recaptcha + .form-group + = f.label :recaptcha_private_key, 'reCAPTCHA Private Key', class: 'control-label col-sm-2' + .col-sm-10 + = f.text_field :recaptcha_private_key, class: 'form-control' + .form-actions = f.submit 'Save', class: 'btn btn-primary' diff --git a/app/views/devise/shared/_signup_box.html.haml b/app/views/devise/shared/_signup_box.html.haml index 9dc6aeffd59..cb93ff2465e 100644 --- a/app/views/devise/shared/_signup_box.html.haml +++ b/app/views/devise/shared/_signup_box.html.haml @@ -6,17 +6,21 @@ .login-heading %h3 Create an account .login-body + - user = params[:user].present? ? params[:user] : {} = form_for(resource, as: resource_name, url: registration_path(resource_name)) do |f| .devise-errors = devise_error_messages! %div - = f.text_field :name, class: "form-control top", placeholder: "Name", required: true + = f.text_field :name, class: "form-control top", value: user[:name], placeholder: "Name", required: true %div - = f.text_field :username, class: "form-control middle", placeholder: "Username", required: true + = f.text_field :username, class: "form-control middle", value: user[:username], placeholder: "Username", required: true %div - = f.email_field :email, class: "form-control middle", placeholder: "Email", required: true + = f.email_field :email, class: "form-control middle", value: user[:email], placeholder: "Email", required: true .form-group.append-bottom-20#password-strength - = f.password_field :password, class: "form-control bottom", id: "user_password_sign_up", placeholder: "Password", required: true + = f.password_field :password, class: "form-control bottom", value: user[:password], id: "user_password_sign_up", placeholder: "Password", required: true + %div + - if current_application_settings.recaptcha_enabled + = recaptcha_tags %div = f.submit "Sign up", class: "btn-create btn" diff --git a/app/views/groups/edit.html.haml b/app/views/groups/edit.html.haml index 8daac585960..1dea77c2e96 100644 --- a/app/views/groups/edit.html.haml +++ b/app/views/groups/edit.html.haml @@ -31,7 +31,7 @@ .col-sm-10 .checkbox = f.check_box :public - %span.descr Make this group public (even if there is no any public project inside this group) + %span.descr Make this group public (even if there are no public projects inside this group) .form-actions = f.submit 'Save group', class: "btn btn-save" diff --git a/app/views/help/_shortcuts.html.haml b/app/views/help/_shortcuts.html.haml index 7e801b5332d..e8e331dd109 100644 --- a/app/views/help/_shortcuts.html.haml +++ b/app/views/help/_shortcuts.html.haml @@ -219,11 +219,3 @@ %td.shortcut .key r %td Reply (quoting selected text) - - -:javascript - $('.js-more-help-button').click(function (e) { - $(this).remove()l - $('.hidden-shortcut').show(); - e.preventDefault(); - }); diff --git a/app/views/layouts/_head.html.haml b/app/views/layouts/_head.html.haml index 74174a72f5a..2e0bd2007a3 100644 --- a/app/views/layouts/_head.html.haml +++ b/app/views/layouts/_head.html.haml @@ -1,10 +1,26 @@ -- page_title "GitLab" -%head +%head{prefix: "og: http://ogp.me/ns#"} %meta{charset: "utf-8"} %meta{'http-equiv' => 'X-UA-Compatible', content: 'IE=edge'} - %meta{content: "GitLab Community Edition", name: "description"} %meta{name: 'referrer', content: 'origin-when-cross-origin'} + %meta{name: "description", content: page_description} + + -# Open Graph - http://ogp.me/ + %meta{property: 'og:type', content: "object"} + %meta{property: 'og:site_name', content: "GitLab"} + %meta{property: 'og:title', content: page_title} + %meta{property: 'og:description', content: page_description} + %meta{property: 'og:image', content: page_image} + %meta{property: 'og:url', content: request.base_url + request.fullpath} + + -# Twitter Card - https://dev.twitter.com/cards/types/summary + %meta{property: 'twitter:card', content: "summary"} + %meta{property: 'twitter:title', content: page_title} + %meta{property: 'twitter:description', content: page_description} + %meta{property: 'twitter:image', content: page_image} + = page_card_meta_tags + + - page_title "GitLab" %title= page_title = favicon_link_tag 'favicon.ico' diff --git a/app/views/profiles/keys/new.html.haml b/app/views/profiles/keys/new.html.haml index 11166dc6d99..13a18269d11 100644 --- a/app/views/profiles/keys/new.html.haml +++ b/app/views/profiles/keys/new.html.haml @@ -12,6 +12,6 @@ comment = val.match(/^\S+ \S+ (.+)\n?$/); if( comment && comment.length > 1 && title.val() == '' ){ - $('#key_title').val( comment[1] ); + $('#key_title').val( comment[1] ).change(); } }); diff --git a/app/views/profiles/two_factor_auths/new.html.haml b/app/views/profiles/two_factor_auths/new.html.haml index 92dc58c10d7..1a5b6efce35 100644 --- a/app/views/profiles/two_factor_auths/new.html.haml +++ b/app/views/profiles/two_factor_auths/new.html.haml @@ -38,3 +38,4 @@ = text_field_tag :pin_code, nil, class: "form-control", required: true, autofocus: true .form-actions = submit_tag 'Submit', class: 'btn btn-success' + = link_to 'Configure it later', skip_profile_two_factor_auth_path, :method => :patch, class: 'btn btn-cancel' if two_factor_skippable? diff --git a/app/views/projects/issues/show.html.haml b/app/views/projects/issues/show.html.haml index b6efa05a1ae..f931a0d3b92 100644 --- a/app/views/projects/issues/show.html.haml +++ b/app/views/projects/issues/show.html.haml @@ -1,13 +1,13 @@ -- page_title "#{@issue.title} (##{@issue.iid})", "Issues" +- page_title "#{@issue.title} (##{@issue.iid})", "Issues" +- page_description @issue.description +- page_card_attributes @issue.card_attributes + = render "header_title" .issue .detail-page-header - .status-box{ class: status_box_class(@issue) } - - if @issue.closed? - Closed - - else - Open + .status-box{ class: "status-box-closed #{issue_button_visibility(@issue, false)}"} Closed + .status-box{ class: "status-box-open #{issue_button_visibility(@issue, true)}"} Open %span.identifier Issue ##{@issue.iid} %span.creator @@ -27,10 +27,8 @@ = icon('plus') New Issue - if can?(current_user, :update_issue, @issue) - - if @issue.closed? - = link_to 'Reopen', issue_path(@issue, issue: {state_event: :reopen}, status_only: true), method: :put, class: 'btn btn-nr btn-grouped btn-reopen' - - else - = link_to 'Close', issue_path(@issue, issue: {state_event: :close}, status_only: true), method: :put, class: 'btn btn-nr btn-grouped btn-close', title: 'Close Issue' + = link_to 'Reopen', issue_path(@issue, issue: {state_event: :reopen}, status_only: true, format: 'json'), data: {no_turbolink: true}, class: "btn btn-nr btn-grouped btn-reopen #{issue_button_visibility(@issue, false)}", title: 'Reopen Issue' + = link_to 'Close', issue_path(@issue, issue: {state_event: :close}, status_only: true, format: 'json'), data: {no_turbolink: true}, class: "btn btn-nr btn-grouped btn-close #{issue_button_visibility(@issue, true)}", title: 'Close Issue' = link_to edit_namespace_project_issue_path(@project.namespace, @project, @issue), class: 'btn btn-nr btn-grouped issuable-edit' do = icon('pencil-square-o') diff --git a/app/views/projects/merge_requests/_show.html.haml b/app/views/projects/merge_requests/_show.html.haml index e9ffbd06be2..ba7c2c01e93 100644 --- a/app/views/projects/merge_requests/_show.html.haml +++ b/app/views/projects/merge_requests/_show.html.haml @@ -1,4 +1,7 @@ -- page_title "#{@merge_request.title} (##{@merge_request.iid})", "Merge Requests" +- page_title "#{@merge_request.title} (##{@merge_request.iid})", "Merge Requests" +- page_description @merge_request.description +- page_card_attributes @merge_request.card_attributes + = render "header_title" - if params[:view] == 'parallel' diff --git a/app/views/projects/milestones/show.html.haml b/app/views/projects/milestones/show.html.haml index 7e73ae274e9..1670ea8741a 100644 --- a/app/views/projects/milestones/show.html.haml +++ b/app/views/projects/milestones/show.html.haml @@ -1,4 +1,6 @@ -- page_title @milestone.title, "Milestones" +- page_title @milestone.title, "Milestones" +- page_description @milestone.description + = render "header_title" .detail-page-header diff --git a/app/views/users/show.html.haml b/app/views/users/show.html.haml index b7a7eb4e6f7..0bca8177e14 100644 --- a/app/views/users/show.html.haml +++ b/app/views/users/show.html.haml @@ -1,5 +1,6 @@ -- page_title @user.name -- header_title @user.name, user_path(@user) +- page_title @user.name +- page_description @user.bio +- header_title @user.name, user_path(@user) = content_for :meta_tags do = auto_discovery_link_tag(:atom, user_url(@user, format: :atom), title: "#{@user.name} activity") diff --git a/app/views/votes/_votes_block.html.haml b/app/views/votes/_votes_block.html.haml index e16187bb42f..ce0a0113403 100644 --- a/app/views/votes/_votes_block.html.haml +++ b/app/views/votes/_votes_block.html.haml @@ -1,5 +1,5 @@ .awards.votes-block - - votable.notes.awards.grouped_awards.each do |emoji, notes| + - awards_sort(votable.notes.awards.grouped_awards).each do |emoji, notes| .award{class: (note_active_class(notes, current_user)), title: emoji_author_list(notes, current_user)} = emoji_icon(emoji) .counter diff --git a/config/database.yml.env b/config/database.yml.env new file mode 100644 index 00000000000..b2ff23cb5ab --- /dev/null +++ b/config/database.yml.env @@ -0,0 +1,9 @@ +<%= ENV['RAILS_ENV'] %>: + adapter: <%= ENV['GITLAB_DATABASE_ADAPTER'] || 'postgresql' %> + encoding: <%= ENV['GITLAB_DATABASE_ENCODING'] || 'unicode' %> + database: <%= ENV['GITLAB_DATABASE_DATABASE'] || "gitlab_#{ENV['RAILS_ENV']}" %> + pool: <%= ENV['GITLAB_DATABASE_POOL'] || '10' %> + username: <%= ENV['GITLAB_DATABASE_USERNAME'] || 'root' %> + password: <%= ENV['GITLAB_DATABASE_PASSWORD'] || '' %> + host: <%= ENV['GITLAB_DATABASE_HOST'] || 'localhost' %> + port: <%= ENV['GITLAB_DATABASE_PORT'] || '5432' %> diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example index db68b5512b8..2d9f730c183 100644 --- a/config/gitlab.yml.example +++ b/config/gitlab.yml.example @@ -4,8 +4,8 @@ # ########################### NOTE ##################################### # This file should not receive new settings. All configuration options # -# that do not require an application restart are being moved to # -# ApplicationSetting model! # +# * are being moved to ApplicationSetting model! # +# If a setting requires an application restart say so in that screen. # # If you change this file in a Merge Request, please also create # # a MR on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests # ######################################################################## diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb index 816cb0c02a9..dea59f4fec8 100644 --- a/config/initializers/1_settings.rb +++ b/config/initializers/1_settings.rb @@ -131,6 +131,7 @@ Settings.omniauth.cas3['session_duration'] ||= 8.hours Settings.omniauth['session_tickets'] ||= Settingslogic.new({}) Settings.omniauth.session_tickets['cas3'] = 'ticket' + Settings['shared'] ||= Settingslogic.new({}) Settings.shared['path'] = File.expand_path(Settings.shared['path'] || "shared", Rails.root) @@ -144,7 +145,7 @@ Settings.gitlab['default_projects_limit'] ||= 10 Settings.gitlab['default_branch_protection'] ||= 2 Settings.gitlab['default_can_create_group'] = true if Settings.gitlab['default_can_create_group'].nil? Settings.gitlab['default_theme'] = Gitlab::Themes::APPLICATION_DEFAULT if Settings.gitlab['default_theme'].nil? -Settings.gitlab['host'] ||= 'localhost' +Settings.gitlab['host'] ||= ENV['GITLAB_HOST'] || 'localhost' Settings.gitlab['ssh_host'] ||= Settings.gitlab.host Settings.gitlab['https'] = false if Settings.gitlab['https'].nil? Settings.gitlab['port'] ||= Settings.gitlab.https ? 443 : 80 diff --git a/config/initializers/metrics.rb b/config/initializers/metrics.rb new file mode 100644 index 00000000000..2e4908192a1 --- /dev/null +++ b/config/initializers/metrics.rb @@ -0,0 +1,64 @@ +if Gitlab::Metrics.enabled? + require 'influxdb' + require 'socket' + require 'connection_pool' + require 'method_source' + + # These are manually require'd so the classes are registered properly with + # ActiveSupport. + require 'gitlab/metrics/subscribers/action_view' + require 'gitlab/metrics/subscribers/active_record' + + Gitlab::Application.configure do |config| + config.middleware.use(Gitlab::Metrics::RackMiddleware) + end + + Sidekiq.configure_server do |config| + config.server_middleware do |chain| + chain.add Gitlab::Metrics::SidekiqMiddleware + end + end + + # This instruments all methods residing in app/models that (appear to) use any + # of the ActiveRecord methods. This has to take place _after_ initializing as + # for some unknown reason calling eager_load! earlier breaks Devise. + Gitlab::Application.config.after_initialize do + Rails.application.eager_load! + + models = Rails.root.join('app', 'models').to_s + + regex = Regexp.union( + ActiveRecord::Querying.public_instance_methods(false).map(&:to_s) + ) + + Gitlab::Metrics::Instrumentation. + instrument_class_hierarchy(ActiveRecord::Base) do |klass, method| + # Instrumenting the ApplicationSetting class can lead to an infinite + # loop. Since the data is cached any way we don't really need to + # instrument it. + if klass == ApplicationSetting + false + else + loc = method.source_location + + loc && loc[0].start_with?(models) && method.source =~ regex + end + end + end + + Gitlab::Metrics::Instrumentation.configure do |config| + config.instrument_instance_methods(Gitlab::Shell) + + config.instrument_methods(Gitlab::Git) + + Gitlab::Git.constants.each do |name| + const = Gitlab::Git.const_get(name) + + config.instrument_methods(const) if const.is_a?(Module) + end + end + + GC::Profiler.enable + + Gitlab::Metrics::Sampler.new.start +end diff --git a/config/routes.rb b/config/routes.rb index b9242327de1..3e7d9f78710 100644 --- a/config/routes.rb +++ b/config/routes.rb @@ -297,6 +297,7 @@ Rails.application.routes.draw do resource :two_factor_auth, only: [:new, :create, :destroy] do member do post :codes + patch :skip end end end diff --git a/db/migrate/20151218154042_add_tfa_to_application_settings.rb b/db/migrate/20151218154042_add_tfa_to_application_settings.rb new file mode 100644 index 00000000000..dd95db775c5 --- /dev/null +++ b/db/migrate/20151218154042_add_tfa_to_application_settings.rb @@ -0,0 +1,8 @@ +class AddTfaToApplicationSettings < ActiveRecord::Migration + def change + change_table :application_settings do |t| + t.boolean :require_two_factor_authentication, default: false + t.integer :two_factor_grace_period, default: 48 + end + end +end diff --git a/db/migrate/20151221234414_add_tfa_additional_fields.rb b/db/migrate/20151221234414_add_tfa_additional_fields.rb new file mode 100644 index 00000000000..c16df47932f --- /dev/null +++ b/db/migrate/20151221234414_add_tfa_additional_fields.rb @@ -0,0 +1,7 @@ +class AddTfaAdditionalFields < ActiveRecord::Migration + def change + change_table :users do |t| + t.datetime :otp_grace_period_started_at, null: true + end + end +end diff --git a/db/migrate/20151228150906_influxdb_settings.rb b/db/migrate/20151228150906_influxdb_settings.rb new file mode 100644 index 00000000000..3012bd52cfd --- /dev/null +++ b/db/migrate/20151228150906_influxdb_settings.rb @@ -0,0 +1,18 @@ +class InfluxdbSettings < ActiveRecord::Migration + def change + add_column :application_settings, :metrics_enabled, :boolean, default: false + + add_column :application_settings, :metrics_host, :string, + default: 'localhost' + + add_column :application_settings, :metrics_database, :string, + default: 'gitlab' + + add_column :application_settings, :metrics_username, :string + add_column :application_settings, :metrics_password, :string + add_column :application_settings, :metrics_pool_size, :integer, default: 16 + add_column :application_settings, :metrics_timeout, :integer, default: 10 + add_column :application_settings, :metrics_method_call_threshold, + :integer, default: 10 + end +end diff --git a/db/migrate/20151228175719_add_recaptcha_to_application_settings.rb b/db/migrate/20151228175719_add_recaptcha_to_application_settings.rb new file mode 100644 index 00000000000..259fd0248d2 --- /dev/null +++ b/db/migrate/20151228175719_add_recaptcha_to_application_settings.rb @@ -0,0 +1,9 @@ +class AddRecaptchaToApplicationSettings < ActiveRecord::Migration + def change + change_table :application_settings do |t| + t.boolean :recaptcha_enabled, default: false + t.string :recaptcha_site_key + t.string :recaptcha_private_key + end + end +end diff --git a/db/migrate/20151229102248_influxdb_udp_port_setting.rb b/db/migrate/20151229102248_influxdb_udp_port_setting.rb new file mode 100644 index 00000000000..ae0499f936d --- /dev/null +++ b/db/migrate/20151229102248_influxdb_udp_port_setting.rb @@ -0,0 +1,5 @@ +class InfluxdbUdpPortSetting < ActiveRecord::Migration + def change + add_column :application_settings, :metrics_port, :integer, default: 8089 + end +end diff --git a/db/migrate/20151229112614_influxdb_remote_database_setting.rb b/db/migrate/20151229112614_influxdb_remote_database_setting.rb new file mode 100644 index 00000000000..f0e1ee1e7a7 --- /dev/null +++ b/db/migrate/20151229112614_influxdb_remote_database_setting.rb @@ -0,0 +1,5 @@ +class InfluxdbRemoteDatabaseSetting < ActiveRecord::Migration + def change + remove_column :application_settings, :metrics_database + end +end diff --git a/db/schema.rb b/db/schema.rb index 0d53105b057..df7f72d5ad4 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -11,7 +11,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema.define(version: 20151224123230) do +ActiveRecord::Schema.define(version: 20151229112614) do # These are extensions that must be enabled in order to support this database enable_extension "plpgsql" @@ -33,23 +33,36 @@ ActiveRecord::Schema.define(version: 20151224123230) do t.datetime "created_at" t.datetime "updated_at" t.string "home_page_url" - t.integer "default_branch_protection", default: 2 - t.boolean "twitter_sharing_enabled", default: true + t.integer "default_branch_protection", default: 2 + t.boolean "twitter_sharing_enabled", default: true t.text "restricted_visibility_levels" - t.boolean "version_check_enabled", default: true - t.integer "max_attachment_size", default: 10, null: false + t.boolean "version_check_enabled", default: true + t.integer "max_attachment_size", default: 10, null: false t.integer "default_project_visibility" t.integer "default_snippet_visibility" t.text "restricted_signup_domains" - t.boolean "user_oauth_applications", default: true + t.boolean "user_oauth_applications", default: true t.string "after_sign_out_path" - t.integer "session_expire_delay", default: 10080, null: false + t.integer "session_expire_delay", default: 10080, null: false t.text "import_sources" t.text "help_page_text" t.string "admin_notification_email" - t.boolean "shared_runners_enabled", default: true, null: false - t.integer "max_artifacts_size", default: 100, null: false + t.boolean "shared_runners_enabled", default: true, null: false + t.integer "max_artifacts_size", default: 100, null: false t.string "runners_registration_token" + t.boolean "require_two_factor_authentication", default: false + t.integer "two_factor_grace_period", default: 48 + t.boolean "metrics_enabled", default: false + t.string "metrics_host", default: "localhost" + t.string "metrics_username" + t.string "metrics_password" + t.integer "metrics_pool_size", default: 16 + t.integer "metrics_timeout", default: 10 + t.integer "metrics_method_call_threshold", default: 10 + t.boolean "recaptcha_enabled", default: false + t.string "recaptcha_site_key" + t.string "recaptcha_private_key" + t.integer "metrics_port", default: 8089 end create_table "audit_events", force: :cascade do |t| @@ -783,12 +796,12 @@ ActiveRecord::Schema.define(version: 20151224123230) do add_index "tags", ["name"], name: "index_tags_on_name", unique: true, using: :btree create_table "users", force: :cascade do |t| - t.string "email", default: "", null: false - t.string "encrypted_password", default: "", null: false + t.string "email", default: "", null: false + t.string "encrypted_password", default: "", null: false t.string "reset_password_token" t.datetime "reset_password_sent_at" t.datetime "remember_created_at" - t.integer "sign_in_count", default: 0 + t.integer "sign_in_count", default: 0 t.datetime "current_sign_in_at" t.datetime "last_sign_in_at" t.string "current_sign_in_ip" @@ -796,22 +809,22 @@ ActiveRecord::Schema.define(version: 20151224123230) do t.datetime "created_at" t.datetime "updated_at" t.string "name" - t.boolean "admin", default: false, null: false - t.integer "projects_limit", default: 10 - t.string "skype", default: "", null: false - t.string "linkedin", default: "", null: false - t.string "twitter", default: "", null: false + t.boolean "admin", default: false, null: false + t.integer "projects_limit", default: 10 + t.string "skype", default: "", null: false + t.string "linkedin", default: "", null: false + t.string "twitter", default: "", null: false t.string "authentication_token" - t.integer "theme_id", default: 1, null: false + t.integer "theme_id", default: 1, null: false t.string "bio" - t.integer "failed_attempts", default: 0 + t.integer "failed_attempts", default: 0 t.datetime "locked_at" t.string "username" - t.boolean "can_create_group", default: true, null: false - t.boolean "can_create_team", default: true, null: false + t.boolean "can_create_group", default: true, null: false + t.boolean "can_create_team", default: true, null: false t.string "state" - t.integer "color_scheme_id", default: 1, null: false - t.integer "notification_level", default: 1, null: false + t.integer "color_scheme_id", default: 1, null: false + t.integer "notification_level", default: 1, null: false t.datetime "password_expires_at" t.integer "created_by_id" t.datetime "last_credential_check_at" @@ -820,24 +833,25 @@ ActiveRecord::Schema.define(version: 20151224123230) do t.datetime "confirmed_at" t.datetime "confirmation_sent_at" t.string "unconfirmed_email" - t.boolean "hide_no_ssh_key", default: false - t.string "website_url", default: "", null: false + t.boolean "hide_no_ssh_key", default: false + t.string "website_url", default: "", null: false t.string "notification_email" - t.boolean "hide_no_password", default: false - t.boolean "password_automatically_set", default: false + t.boolean "hide_no_password", default: false + t.boolean "password_automatically_set", default: false t.string "location" t.string "encrypted_otp_secret" t.string "encrypted_otp_secret_iv" t.string "encrypted_otp_secret_salt" - t.boolean "otp_required_for_login", default: false, null: false + t.boolean "otp_required_for_login", default: false, null: false t.text "otp_backup_codes" - t.string "public_email", default: "", null: false - t.integer "dashboard", default: 0 - t.integer "project_view", default: 0 + t.string "public_email", default: "", null: false + t.integer "dashboard", default: 0 + t.integer "project_view", default: 0 t.integer "consumed_timestep" - t.integer "layout", default: 0 - t.boolean "hide_project_limit", default: false + t.integer "layout", default: 0 + t.boolean "hide_project_limit", default: false t.string "unlock_token" + t.datetime "otp_grace_period_started_at" end add_index "users", ["admin"], name: "index_users_on_admin", using: :btree diff --git a/doc/README.md b/doc/README.md index 8bac00f2f23..d82ff8b908b 100644 --- a/doc/README.md +++ b/doc/README.md @@ -28,17 +28,18 @@ - [Using SSH keys](ci/ssh_keys/README.md) - [User permissions](ci/permissions/README.md) - [API](ci/api/README.md) +- [Triggering builds through the API](ci/triggers/README.md) ### CI Languages -+ [Testing PHP](ci/languages/php.md) +- [Testing PHP](ci/languages/php.md) ### CI Services -+ [Using MySQL](ci/services/mysql.md) -+ [Using PostgreSQL](ci/services/postgres.md) -+ [Using Redis](ci/services/redis.md) -+ [Using Other Services](ci/docker/using_docker_images.md#how-to-use-other-images-as-services) +- [Using MySQL](ci/services/mysql.md) +- [Using PostgreSQL](ci/services/postgres.md) +- [Using Redis](ci/services/redis.md) +- [Using Other Services](ci/docker/using_docker_images.md#how-to-use-other-images-as-services) ### CI Examples @@ -55,6 +56,7 @@ - [Issue closing](customization/issue_closing.md) Customize how to close an issue from commit messages. - [Libravatar](customization/libravatar.md) Use Libravatar for user avatars. - [Log system](logs/logs.md) Log system. +- [Environmental Variables](administration/environmental_variables.md) to configure GitLab. - [Operations](operations/README.md) Keeping GitLab up and running - [Raketasks](raketasks/README.md) Backups, maintenance, automatic web hook setup and the importing of projects. - [Security](security/README.md) Learn what you can do to further secure your GitLab instance. diff --git a/doc/administration/enviroment_variables.md b/doc/administration/enviroment_variables.md new file mode 100644 index 00000000000..d7f5cb7c21f --- /dev/null +++ b/doc/administration/enviroment_variables.md @@ -0,0 +1,45 @@ +# Environment Variables + +## Introduction + +Commonly people configure GitLab via the gitlab.rb configuration file in the Omnibus package. + +But if you prefer to use environment variables we allow that too. + +## Supported environment variables + +Variable | Type | Explanation +--- | --- | --- +GITLAB_ROOT_PASSWORD | string | sets the password for the `root` user on installation +GITLAB_HOST | url | hostname of the GitLab server includes http or https +RAILS_ENV | production/development/staging/test | Rails environment +DATABASE_URL | url | For example: postgresql://localhost/blog_development?pool=5 + +## Complete database variables + +As explained in the [Heroku documentation](https://devcenter.heroku.com/articles/rails-database-connection-behavior) the DATABASE_URL doesn't let you set: + +- adapter +- database +- username +- password +- host +- port + +To do so please `cp config/database.yml.env config/database.yml` and use the following variables: + +Variable | Default +--- | --- +GITLAB_DATABASE_ADAPTER | postgresql +GITLAB_DATABASE_ENCODING | unicode +GITLAB_DATABASE_DATABASE | gitlab_#{ENV['RAILS_ENV'] +GITLAB_DATABASE_POOL | 10 +GITLAB_DATABASE_USERNAME | root +GITLAB_DATABASE_PASSWORD | +GITLAB_DATABASE_HOST | localhost +GITLAB_DATABASE_PORT | 5432 + +## Other variables + +We welcome merge requests to make more settings configurable via variables. +Please stick to the naming scheme "GITLAB_#{name 1_settings.rb in upper case}". diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 366a1f8abec..8bc0a67067a 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -4,8 +4,7 @@ Get all merge requests for this project. The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). -The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. With GitLab 8.2 the return fields `upvotes` and -`downvotes` are deprecated and always return `0`. +The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. ``` GET /projects/:id/merge_requests @@ -58,7 +57,7 @@ Parameters: ## Get single MR -Shows information about a single merge request. With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and always return `0`. +Shows information about a single merge request. ``` GET /projects/:id/merge_request/:merge_request_id @@ -141,8 +140,6 @@ Parameters: ## Get single MR changes Shows information about the merge request including its files and changes. -With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and -always return `0`. ``` GET /projects/:id/merge_request/:merge_request_id/changes @@ -213,9 +210,7 @@ Parameters: ## Create MR -Creates a new merge request. With GitLab 8.2 the return fields `upvotes` and ` -downvotes` are deprecated and always return `0`. - +Creates a new merge request. ``` POST /projects/:id/merge_requests ``` @@ -266,8 +261,7 @@ If an error occurs, an error number and a message explaining the reason is retur ## Update MR -Updates an existing merge request. You can change the target branch, title, or even close the MR. With GitLab 8.2 the return fields `upvotes` and `downvotes` -are deprecated and always return `0`. +Updates an existing merge request. You can change the target branch, title, or even close the MR. ``` PUT /projects/:id/merge_request/:merge_request_id @@ -318,8 +312,7 @@ If an error occurs, an error number and a message explaining the reason is retur ## Accept MR -Merge changes submitted with MR using this API. With GitLab 8.2 the return -fields `upvotes` and `downvotes` are deprecated and always return `0`. +Merge changes submitted with MR using this API. If merge success you get `200 OK`. diff --git a/doc/api/notes.md b/doc/api/notes.md index 4d7ef288df8..d4d63e825ab 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -6,8 +6,7 @@ Notes are comments on snippets, issues or merge requests. ### List project issue notes -Gets a list of all notes for a single issue. With GitLab 8.2 the return fields -`upvote` and `downvote` are deprecated and always return `false`. +Gets a list of all notes for a single issue. ``` GET /projects/:id/issues/:issue_id/notes diff --git a/doc/api/users.md b/doc/api/users.md index 7ba2db248ff..66d2fd52526 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -90,7 +90,17 @@ GET /users You can search for users by email or username with: `/users?search=John` -Also see `def search query` in `app/models/user.rb`. +In addition, you can lookup users by username: + +``` +GET /users?username=:username +``` + +For example: + +``` +GET /users?username=jack_smith +``` ## Single user diff --git a/doc/ci/README.md b/doc/ci/README.md index 965b007bedb..a1f5513d88e 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -2,25 +2,26 @@ ### User documentation -+ [Quick Start](quick_start/README.md) -+ [Configuring project (.gitlab-ci.yml)](yaml/README.md) -+ [Configuring runner](runners/README.md) -+ [Configuring deployment](deployment/README.md) -+ [Using Docker Images](docker/using_docker_images.md) -+ [Using Docker Build](docker/using_docker_build.md) -+ [Using Variables](variables/README.md) -+ [Using SSH keys](ssh_keys/README.md) +* [Quick Start](quick_start/README.md) +* [Configuring project (.gitlab-ci.yml)](yaml/README.md) +* [Configuring runner](runners/README.md) +* [Configuring deployment](deployment/README.md) +* [Using Docker Images](docker/using_docker_images.md) +* [Using Docker Build](docker/using_docker_build.md) +* [Using Variables](variables/README.md) +* [Using SSH keys](ssh_keys/README.md) +* [Triggering builds through the API](triggers/README.md) ### Languages -+ [Testing PHP](languages/php.md) +* [Testing PHP](languages/php.md) ### Services -+ [Using MySQL](services/mysql.md) -+ [Using PostgreSQL](services/postgres.md) -+ [Using Redis](services/redis.md) -+ [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services) +* [Using MySQL](services/mysql.md) +* [Using PostgreSQL](services/postgres.md) +* [Using Redis](services/redis.md) +* [Using Other Services](docker/using_docker_images.md#how-to-use-other-images-as-services) ### Examples @@ -32,5 +33,5 @@ ### Administrator documentation -+ [User permissions](permissions/README.md) -+ [API](api/README.md) +* [User permissions](permissions/README.md) +* [API](api/README.md) diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md new file mode 100644 index 00000000000..9f7c1bfe6a0 --- /dev/null +++ b/doc/ci/triggers/README.md @@ -0,0 +1,172 @@ +# Triggering Builds through the API + +_**Note:** This feature was [introduced][ci-229] in GitLab CE 7.14_ + +Triggers can be used to force a rebuild of a specific branch, tag or commit, +with an API call. + +## Add a trigger + +You can add a new trigger by going to your project's **Settings > Triggers**. +The **Add trigger** button will create a new token which you can then use to +trigger a rebuild of this particular project. + +Every new trigger you create, gets assigned a different token which you can +then use inside your scripts or `.gitlab-ci.yml`. You also have a nice +overview of the time the triggers were last used. + + + +## Revoke a trigger + +You can revoke a trigger any time by going at your project's +**Settings > Triggers** and hitting the **Revoke** button. The action is +irreversible. + +## Trigger a build + +To trigger a build you need to send a `POST` request to GitLab's API endpoint: + +``` +POST /projects/:id/trigger/builds +``` + +The required parameters are the trigger's `token` and the Git `ref` on which +the trigger will be performed. Valid refs are the branch, the tag or the commit +SHA. The `:id` of a project can be found by [querying the API](../api/projects.md) +or by visiting the **Triggers** page which provides self-explanatory examples. + +When a rebuild is triggered, the information is exposed in GitLab's UI under +the **Builds** page and the builds are marked as `triggered`. + + + +--- + +You can see which trigger caused the rebuild by visiting the single build page. +The token of the trigger is exposed in the UI as you can see from the image +below. + + + +--- + +See the [Examples](#examples) section for more details on how to actually +trigger a rebuild. + +## Pass build variables to a trigger + +You can pass any number of arbitrary variables in the trigger API call and they +will be available in GitLab CI so that they can be used in your `.gitlab-ci.yml` +file. The parameter is of the form: + +``` +variables[key]=value +``` + +This information is also exposed in the UI. + + + +--- + +See the [Examples](#examples) section below for more details. + +## Examples + +Using cURL you can trigger a rebuild with minimal effort, for example: + +```bash +curl -X POST \ + -F token=TOKEN \ + -F ref=master \ + https://gitlab.example.com/api/v3/projects/9/trigger/builds +``` + +In this case, the project with ID `9` will get rebuilt on `master` branch. + + +### Triggering a build within `.gitlab-ci.yml` + +You can also benefit by using triggers in your `.gitlab-ci.yml`. Let's say that +you have two projects, A and B, and you want to trigger a rebuild on the `master` +branch of project B whenever a tag on project A is created. This is the job you +need to add in project's A `.gitlab-ci.yml`: + +```yaml +build_docs: + stage: deploy + script: + - "curl -X POST -F token=TOKEN -F ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds" + only: + - tags +``` + +Now, whenever a new tag is pushed on project A, the build will run and the +`build_docs` job will be executed, triggering a rebuild of project B. The +`stage: deploy` ensures that this job will run only after all jobs with +`stage: test` complete successfully. + +_**Note:** If your project is public, passing the token in plain text is +probably not the wisest idea, so you might want to use a +[secure variable](../variables/README.md#user-defined-variables-secure-variables) +for that purpose._ + +### Making use of trigger variables + +Using trigger variables can be proven useful for a variety of reasons. + +* Identifiable jobs. Since the variable is exposed in the UI you can know + why the rebuild was triggered if you pass a variable that explains the + purpose. +* Conditional job processing. You can have conditional jobs that run whenever + a certain variable is present. + +Consider the following `.gitlab-ci.yml` where we set three +[stages](../yaml/README.md#stages) and the `upload_package` job is run only +when all jobs from the test and build stages pass. When the `UPLOAD_TO_S3` +variable is non-zero, `make upload` is run. + +```yaml +stages: +- test +- build +- package + +run_tests: + script: + - make test + +build_package: + stage: build + script: + - make build + +upload_package: + stage: package + script: + - if [ -n "${UPLOAD_TO_S3}" ]; then make upload; fi +``` + +You can then trigger a rebuild while you pass the `UPLOAD_TO_S3` variable +and the script of the `upload_package` job will run: + +```bash +curl -X POST \ + -F token=TOKEN \ + -F ref=master \ + -F "variables[UPLOAD_TO_S3]=true" \ + https://gitlab.example.com/api/v3/projects/9/trigger/builds +``` + +### Using cron to trigger nightly builds + +Whether you craft a script or just run cURL directly, you can trigger builds +in conjunction with cron. The example below triggers a build on the `master` +branch of project with ID `9` every night at `00:30`: + +```bash +30 0 * * * curl -X POST -F token=TOKEN -F ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds +``` + +[ci-229]: https://gitlab.com/gitlab-org/gitlab-ci/merge_requests/229 diff --git a/doc/ci/triggers/img/builds_page.png b/doc/ci/triggers/img/builds_page.png Binary files differnew file mode 100644 index 00000000000..e78794fbee7 --- /dev/null +++ b/doc/ci/triggers/img/builds_page.png diff --git a/doc/ci/triggers/img/trigger_single_build.png b/doc/ci/triggers/img/trigger_single_build.png Binary files differnew file mode 100644 index 00000000000..c25f27409d6 --- /dev/null +++ b/doc/ci/triggers/img/trigger_single_build.png diff --git a/doc/ci/triggers/img/trigger_variables.png b/doc/ci/triggers/img/trigger_variables.png Binary files differnew file mode 100644 index 00000000000..2207e8b34cb --- /dev/null +++ b/doc/ci/triggers/img/trigger_variables.png diff --git a/doc/ci/triggers/img/triggers_page.png b/doc/ci/triggers/img/triggers_page.png Binary files differnew file mode 100644 index 00000000000..268368dc3c5 --- /dev/null +++ b/doc/ci/triggers/img/triggers_page.png diff --git a/doc/integration/README.md b/doc/integration/README.md index 6263353851f..2a9f76533b7 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -13,6 +13,7 @@ See the documentation below for details on how to configure these services. - [Slack](slack.md) Integrate with the Slack chat service - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages +- [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users GitLab Enterprise Edition contains [advanced JIRA support](http://doc.gitlab.com/ee/integration/jira.html) and [advanced Jenkins support](http://doc.gitlab.com/ee/integration/jenkins.html). diff --git a/doc/integration/recaptcha.md b/doc/integration/recaptcha.md new file mode 100644 index 00000000000..a301d1a613c --- /dev/null +++ b/doc/integration/recaptcha.md @@ -0,0 +1,23 @@ +# reCAPTCHA + +GitLab leverages [Google's reCAPTCHA](https://www.google.com/recaptcha/intro/index.html) +to protect against spam and abuse. GitLab displays the CAPTCHA form on the sign-up page +to confirm that a real user, not a bot, is attempting to create an account. + +## Configuration + +To use reCAPTCHA, first you must create a site and private key. + +1. Go to the URL: https://www.google.com/recaptcha/admin + +2. Fill out the form necessary to obtain reCAPTCHA keys. + +3. Login to your GitLab server, with administrator credentials. + +4. Go to Applications Settings on Admin Area (`admin/application_settings`) + +5. Fill all recaptcha fields with keys from previous steps + +6. Check the `Enable reCAPTCHA` checkbox + +7. Save the configuration. diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index bcd00cfc6bf..1be78ac1823 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -6,11 +6,11 @@ If a user is both in a project group and in the project itself, the highest perm If a user is a GitLab administrator they receive all permissions. -On public projects the Guest role is not enforced. -All users will be able to create issues, leave comments, and pull or download the project code. +On public projects the Guest role is not enforced. +All users will be able to create issues, leave comments, and pull or download the project code. To add or import a user, you can follow the [project users and members -documentation](doc/workflow/add-user/add-user.md). +documentation](../workflow/add-user/add-user.md). ## Project diff --git a/doc/security/README.md b/doc/security/README.md index fba6013d9c1..f34c792d005 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -6,3 +6,5 @@ - [Information exclusivity](information_exclusivity.md) - [Reset your root password](reset_root_password.md) - [User File Uploads](user_file_uploads.md) +- [How we manage the CRIME vulnerability](crime_vulnerability.md) +- [Enforce Two-Factor authentication](two_factor_authentication.md) diff --git a/doc/security/crime_vulnerability.md b/doc/security/crime_vulnerability.md new file mode 100644 index 00000000000..94ba5d1375d --- /dev/null +++ b/doc/security/crime_vulnerability.md @@ -0,0 +1,63 @@ +# How we manage the TLS protocol CRIME vulnerability + +> CRIME ("Compression Ratio Info-leak Made Easy") is a security exploit against +secret web cookies over connections using the HTTPS and SPDY protocols that also +use data compression. When used to recover the content of secret +authentication cookies, it allows an attacker to perform session hijacking on an +authenticated web session, allowing the launching of further attacks. +([CRIME](https://en.wikipedia.org/w/index.php?title=CRIME&oldid=692423806)) + +### Description + +The TLS Protocol CRIME Vulnerability affects compression over HTTPS, therefore +it warns against using SSL Compression (for example gzip) or SPDY which +optionally uses compression as well. + +GitLab supports both gzip and [SPDY][ngx-spdy] and mitigates the CRIME +vulnerability by deactivating gzip when HTTPS is enabled. You can see the +sources of the files in question: + +* [Source installation NGINX file][source-nginx] +* [Omnibus installation NGINX file][omnibus-nginx] + +Although SPDY is enabled in Omnibus installations, CRIME relies on compression +(the 'C') and the default compression level in NGINX's SPDY module is 0 +(no compression). + +### Nessus + +The Nessus scanner, [reports a possible CRIME vulnerability][nessus] in GitLab +similar to the following format: + +``` +Description + +This remote service has one of two configurations that are known to be required for the CRIME attack: +SSL/TLS compression is enabled. +TLS advertises the SPDY protocol earlier than version 4. + +... + +Output + +The following configuration indicates that the remote service may be vulnerable to the CRIME attack: +SPDY support earlier than version 4 is advertised. +``` + +From the report above it is important to note that Nessus is only checking if +TLS advertises the SPDY protocol earlier than version 4, it does not perform an +attack nor does it check if compression is enabled. With just this approach, it +cannot tell that SPDY's compression is disabled and not subject to the CRIME +vulnerability. + +### References + +* Nginx ["Module ngx_http_spdy_module"][ngx-spdy] +* Tenable Network Security, Inc. ["Transport Layer Security (TLS) Protocol CRIME Vulnerability"][nessus] +* Wikipedia contributors, ["CRIME"][wiki-crime] Wikipedia, The Free Encyclopedia + +[source-nginx]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab-ssl +[omnibus-nginx]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb +[ngx-spdy]: http://nginx.org/en/docs/http/ngx_http_spdy_module.html +[nessus]: https://www.tenable.com/plugins/index.php?view=single&id=62565 +[wiki-crime]: https://en.wikipedia.org/wiki/CRIME diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md new file mode 100644 index 00000000000..4e25a1fdc3f --- /dev/null +++ b/doc/security/two_factor_authentication.md @@ -0,0 +1,38 @@ +# Enforce Two-factor Authentication (2FA) + +Two-factor Authentication (2FA) provides an additional level of security to your +users' GitLab account. Once enabled, in addition to supplying their username and +password to login, they'll be prompted for a code generated by an application on +their phone. + +You can read more about it here: +[Two-factor Authentication (2FA)](doc/profile/two_factor_authentication.md) + +## Enabling 2FA + +Users on GitLab, can enable it without any admin's intervention. If you want to +enforce everyone to setup 2FA, you can choose from two different ways: + + 1. Enforce on next login + 2. Suggest on next login, but allow a grace period before enforcing. + +In the Admin area under **Settings** (`/admin/application_settings`), look for +the "Sign-in Restrictions" area, where you can configure both. + +If you want 2FA enforcement to take effect on next login, change the grace +period to `0` + +## Disabling 2FA for everyone + +There may be some special situations where you want to disable 2FA for everyone +even when forced 2FA is disabled. There is a rake task for that: + +``` +# use this command if you've installed GitLab with the Omnibus package +sudo gitlab-rake gitlab:two_factor:disable_for_all_users + +# if you've installed GitLab from source +sudo -u git -H bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production +``` + +**IMPORTANT: this is a permanent and irreversible action. Users will have to reactivate 2FA from scratch if they want to use it again.** diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index c4661dc16af..3748941b781 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -99,8 +99,6 @@ sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production # Clean up assets and cache sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production -# Update init.d script -sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` ### 7. Update configuration files diff --git a/features/steps/project/issues/award_emoji.rb b/features/steps/project/issues/award_emoji.rb index a7e15398819..1404f34cfe0 100644 --- a/features/steps/project/issues/award_emoji.rb +++ b/features/steps/project/issues/award_emoji.rb @@ -15,15 +15,17 @@ class Spinach::Features::AwardEmoji < Spinach::FeatureSteps end step 'I click to emoji in the picker' do - page.within '.emoji-menu' do + page.within '.emoji-menu-content' do page.first('.emoji-icon').click end end step 'I can remove it by clicking to icon' do page.within '.awards' do - page.first('.award').click - expect(page).to_not have_selector '.award' + expect do + page.find('.award.active').click + sleep 0.1 + end.to change{ page.all(".award").size }.from(3).to(2) end end @@ -37,7 +39,7 @@ class Spinach::Features::AwardEmoji < Spinach::FeatureSteps step 'I have award added' do page.within '.awards' do expect(page).to have_selector '.award' - expect(page.find('.award .counter')).to have_content '1' + expect(page.find('.award.active .counter')).to have_content '1' end end diff --git a/lib/api/entities.rb b/lib/api/entities.rb index 76b5d14f202..f21da54b8fc 100644 --- a/lib/api/entities.rb +++ b/lib/api/entities.rb @@ -166,7 +166,6 @@ module API class MergeRequest < ProjectEntity expose :target_branch, :source_branch - # deprecated, always returns 0 expose :upvotes, :downvotes expose :author, :assignee, using: Entities::UserBasic expose :source_project_id, :target_project_id diff --git a/lib/api/users.rb b/lib/api/users.rb index a98d668e02d..3400f0713ef 100644 --- a/lib/api/users.rb +++ b/lib/api/users.rb @@ -8,11 +8,17 @@ module API # # Example Request: # GET /users + # GET /users?search=Admin + # GET /users?username=root get do - @users = User.all - @users = @users.active if params[:active].present? - @users = @users.search(params[:search]) if params[:search].present? - @users = paginate @users + if params[:username].present? + @users = User.where(username: params[:username]) + else + @users = User.all + @users = @users.active if params[:active].present? + @users = @users.search(params[:search]) if params[:search].present? + @users = paginate @users + end if current_user.is_admin? present @users, with: Entities::UserFull diff --git a/lib/banzai/filter/abstract_reference_filter.rb b/lib/banzai/filter/abstract_reference_filter.rb index bdaa4721b4b..63ad8910c0f 100644 --- a/lib/banzai/filter/abstract_reference_filter.rb +++ b/lib/banzai/filter/abstract_reference_filter.rb @@ -98,7 +98,7 @@ module Banzai project = project_from_ref(project_ref) if project && object = find_object(project, id) - title = escape_once(object_link_title(object)) + title = object_link_title(object) klass = reference_class(object_sym) data = data_attribute( @@ -110,17 +110,11 @@ module Banzai url = matches[:url] if matches.names.include?("url") url ||= url_for_object(object, project) - text = link_text - unless text - text = object.reference_link_text(context[:project]) - - extras = object_link_text_extras(object, matches) - text += " (#{extras.join(", ")})" if extras.any? - end + text = link_text || object_link_text(object, matches) %(<a href="#{url}" #{data} - title="#{title}" - class="#{klass}">#{text}</a>) + title="#{escape_once(title)}" + class="#{klass}">#{escape_once(text)}</a>) else match end @@ -140,6 +134,15 @@ module Banzai def object_link_title(object) "#{object_class.name.titleize}: #{object.title}" end + + def object_link_text(object, matches) + text = object.reference_link_text(context[:project]) + + extras = object_link_text_extras(object, matches) + text += " (#{extras.join(", ")})" if extras.any? + + text + end end end end diff --git a/lib/banzai/filter/external_issue_reference_filter.rb b/lib/banzai/filter/external_issue_reference_filter.rb index f5942740cd6..6136e73c096 100644 --- a/lib/banzai/filter/external_issue_reference_filter.rb +++ b/lib/banzai/filter/external_issue_reference_filter.rb @@ -63,15 +63,15 @@ module Banzai url = url_for_issue(id, project, only_path: context[:only_path]) - title = escape_once("Issue in #{project.external_issue_tracker.title}") + title = "Issue in #{project.external_issue_tracker.title}" klass = reference_class(:issue) data = data_attribute(project: project.id, external_issue: id) text = link_text || match %(<a href="#{url}" #{data} - title="#{title}" - class="#{klass}">#{text}</a>) + title="#{escape_once(title)}" + class="#{klass}">#{escape_once(text)}</a>) end end diff --git a/lib/banzai/filter/label_reference_filter.rb b/lib/banzai/filter/label_reference_filter.rb index 07bac2dd7fd..a3a7a23c1e6 100644 --- a/lib/banzai/filter/label_reference_filter.rb +++ b/lib/banzai/filter/label_reference_filter.rb @@ -60,7 +60,7 @@ module Banzai text = link_text || render_colored_label(label) %(<a href="#{url}" #{data} - class="#{klass}">#{text}</a>) + class="#{klass}">#{escape_once(text)}</a>) else match end diff --git a/lib/banzai/filter/redactor_filter.rb b/lib/banzai/filter/redactor_filter.rb index 89e7a79789a..f01a32b5ae5 100644 --- a/lib/banzai/filter/redactor_filter.rb +++ b/lib/banzai/filter/redactor_filter.rb @@ -11,7 +11,7 @@ module Banzai class RedactorFilter < HTML::Pipeline::Filter def call doc.css('a.gfm').each do |node| - unless user_can_reference?(node) + unless user_can_see_reference?(node) # The reference should be replaced by the original text, # which is not always the same as the rendered text. text = node.attr('data-original') || node.text @@ -24,12 +24,12 @@ module Banzai private - def user_can_reference?(node) + def user_can_see_reference?(node) if node.has_attribute?('data-reference-filter') reference_type = node.attr('data-reference-filter') reference_filter = Banzai::Filter.const_get(reference_type) - reference_filter.user_can_reference?(current_user, node, context) + reference_filter.user_can_see_reference?(current_user, node, context) else true end diff --git a/lib/banzai/filter/reference_filter.rb b/lib/banzai/filter/reference_filter.rb index 33457a3f361..8ca05ace88c 100644 --- a/lib/banzai/filter/reference_filter.rb +++ b/lib/banzai/filter/reference_filter.rb @@ -12,7 +12,7 @@ module Banzai # :project (required) - Current project, ignored if reference is cross-project. # :only_path - Generate path-only links. class ReferenceFilter < HTML::Pipeline::Filter - def self.user_can_reference?(user, node, context) + def self.user_can_see_reference?(user, node, context) if node.has_attribute?('data-project') project_id = node.attr('data-project').to_i return true if project_id == context[:project].try(:id) @@ -24,6 +24,10 @@ module Banzai end end + def self.user_can_reference?(user, node, context) + true + end + def self.referenced_by(node) raise NotImplementedError, "#{self} does not implement #{__method__}" end @@ -44,11 +48,11 @@ module Banzai # Returns a String def data_attribute(attributes = {}) attributes[:reference_filter] = self.class.name.demodulize - attributes.map { |key, value| %Q(data-#{key.to_s.dasherize}="#{value}") }.join(" ") + attributes.map { |key, value| %Q(data-#{key.to_s.dasherize}="#{escape_once(value)}") }.join(" ") end def escape_once(html) - ERB::Util.html_escape_once(html) + html.html_safe? ? html : ERB::Util.html_escape_once(html) end def ignore_parents diff --git a/lib/banzai/filter/reference_gatherer_filter.rb b/lib/banzai/filter/reference_gatherer_filter.rb index 855f238ac1e..12412ff7ea9 100644 --- a/lib/banzai/filter/reference_gatherer_filter.rb +++ b/lib/banzai/filter/reference_gatherer_filter.rb @@ -35,7 +35,9 @@ module Banzai return if context[:reference_filter] && reference_filter != context[:reference_filter] - return unless reference_filter.user_can_reference?(current_user, node, context) + return if author && !reference_filter.user_can_reference?(author, node, context) + + return unless reference_filter.user_can_see_reference?(current_user, node, context) references = reference_filter.referenced_by(node) return unless references @@ -57,6 +59,10 @@ module Banzai def current_user context[:current_user] end + + def author + context[:author] + end end end end diff --git a/lib/banzai/filter/user_reference_filter.rb b/lib/banzai/filter/user_reference_filter.rb index 67c24faf991..964ab60f614 100644 --- a/lib/banzai/filter/user_reference_filter.rb +++ b/lib/banzai/filter/user_reference_filter.rb @@ -39,7 +39,7 @@ module Banzai end end - def self.user_can_reference?(user, node, context) + def self.user_can_see_reference?(user, node, context) if node.has_attribute?('data-group') group = Group.find(node.attr('data-group')) rescue nil Ability.abilities.allowed?(user, :read_group, group) @@ -48,6 +48,18 @@ module Banzai end end + def self.user_can_reference?(user, node, context) + # Only team members can reference `@all` + if node.has_attribute?('data-project') + project = Project.find(node.attr('data-project')) rescue nil + return false unless project + + user && project.team.member?(user) + else + super + end + end + def call replace_text_nodes_matching(User.reference_pattern) do |content| user_link_filter(content) @@ -122,7 +134,7 @@ module Banzai end def link_tag(url, data, text) - %(<a href="#{url}" #{data} class="#{link_class}">#{text}</a>) + %(<a href="#{url}" #{data} class="#{link_class}">#{escape_once(text)}</a>) end end end diff --git a/lib/gitlab/metrics.rb b/lib/gitlab/metrics.rb new file mode 100644 index 00000000000..2d266ccfe9e --- /dev/null +++ b/lib/gitlab/metrics.rb @@ -0,0 +1,104 @@ +module Gitlab + module Metrics + extend Gitlab::CurrentSettings + + RAILS_ROOT = Rails.root.to_s + METRICS_ROOT = Rails.root.join('lib', 'gitlab', 'metrics').to_s + PATH_REGEX = /^#{RAILS_ROOT}\/?/ + + def self.pool_size + current_application_settings[:metrics_pool_size] || 16 + end + + def self.timeout + current_application_settings[:metrics_timeout] || 10 + end + + def self.enabled? + current_application_settings[:metrics_enabled] || false + end + + def self.mri? + RUBY_ENGINE == 'ruby' + end + + def self.method_call_threshold + # This is memoized since this method is called for every instrumented + # method. Loading data from an external cache on every method call slows + # things down too much. + @method_call_threshold ||= + (current_application_settings[:metrics_method_call_threshold] || 10) + end + + def self.pool + @pool + end + + def self.hostname + @hostname + end + + # Returns a relative path and line number based on the last application call + # frame. + def self.last_relative_application_frame + frame = caller_locations.find do |l| + l.path.start_with?(RAILS_ROOT) && !l.path.start_with?(METRICS_ROOT) + end + + if frame + return frame.path.sub(PATH_REGEX, ''), frame.lineno + else + return nil, nil + end + end + + def self.submit_metrics(metrics) + prepared = prepare_metrics(metrics) + + pool.with do |connection| + prepared.each do |metric| + begin + connection.write_points([metric]) + rescue StandardError + end + end + end + end + + def self.prepare_metrics(metrics) + metrics.map do |hash| + new_hash = hash.symbolize_keys + + new_hash[:tags].each do |key, value| + if value.blank? + new_hash[:tags].delete(key) + else + new_hash[:tags][key] = escape_value(value) + end + end + + new_hash + end + end + + def self.escape_value(value) + value.to_s.gsub('=', '\\=') + end + + @hostname = Socket.gethostname + + # When enabled this should be set before being used as the usual pattern + # "@foo ||= bar" is _not_ thread-safe. + if enabled? + @pool = ConnectionPool.new(size: pool_size, timeout: timeout) do + host = current_application_settings[:metrics_host] + user = current_application_settings[:metrics_username] + pw = current_application_settings[:metrics_password] + port = current_application_settings[:metrics_port] + + InfluxDB::Client. + new(udp: { host: host, port: port }, username: user, password: pw) + end + end + end +end diff --git a/lib/gitlab/metrics/delta.rb b/lib/gitlab/metrics/delta.rb new file mode 100644 index 00000000000..bcf28eed84d --- /dev/null +++ b/lib/gitlab/metrics/delta.rb @@ -0,0 +1,32 @@ +module Gitlab + module Metrics + # Class for calculating the difference between two numeric values. + # + # Every call to `compared_with` updates the internal value. This makes it + # possible to use a single Delta instance to calculate the delta over time + # of an ever increasing number. + # + # Example usage: + # + # delta = Delta.new(0) + # + # delta.compared_with(10) # => 10 + # delta.compared_with(15) # => 5 + # delta.compared_with(20) # => 5 + class Delta + def initialize(value = 0) + @value = value + end + + # new_value - The value to compare with as a Numeric. + # + # Returns a new Numeric (depending on the type of `new_value`). + def compared_with(new_value) + delta = new_value - @value + @value = new_value + + delta + end + end + end +end diff --git a/lib/gitlab/metrics/instrumentation.rb b/lib/gitlab/metrics/instrumentation.rb new file mode 100644 index 00000000000..06fc2f25948 --- /dev/null +++ b/lib/gitlab/metrics/instrumentation.rb @@ -0,0 +1,146 @@ +module Gitlab + module Metrics + # Module for instrumenting methods. + # + # This module allows instrumenting of methods without having to actually + # alter the target code (e.g. by including modules). + # + # Example usage: + # + # Gitlab::Metrics::Instrumentation.instrument_method(User, :by_login) + module Instrumentation + SERIES = 'method_calls' + + def self.configure + yield self + end + + # Instruments a class method. + # + # mod - The module to instrument as a Module/Class. + # name - The name of the method to instrument. + def self.instrument_method(mod, name) + instrument(:class, mod, name) + end + + # Instruments an instance method. + # + # mod - The module to instrument as a Module/Class. + # name - The name of the method to instrument. + def self.instrument_instance_method(mod, name) + instrument(:instance, mod, name) + end + + # Recursively instruments all subclasses of the given root module. + # + # This can be used to for example instrument all ActiveRecord models (as + # these all inherit from ActiveRecord::Base). + # + # This method can optionally take a block to pass to `instrument_methods` + # and `instrument_instance_methods`. + # + # root - The root module for which to instrument subclasses. The root + # module itself is not instrumented. + def self.instrument_class_hierarchy(root, &block) + visit = root.subclasses + + until visit.empty? + klass = visit.pop + + instrument_methods(klass, &block) + instrument_instance_methods(klass, &block) + + klass.subclasses.each { |c| visit << c } + end + end + + # Instruments all public methods of a module. + # + # This method optionally takes a block that can be used to determine if a + # method should be instrumented or not. The block is passed the receiving + # module and an UnboundMethod. If the block returns a non truthy value the + # method is not instrumented. + # + # mod - The module to instrument. + def self.instrument_methods(mod) + mod.public_methods(false).each do |name| + method = mod.method(name) + + if method.owner == mod.singleton_class + if !block_given? || block_given? && yield(mod, method) + instrument_method(mod, name) + end + end + end + end + + # Instruments all public instance methods of a module. + # + # See `instrument_methods` for more information. + # + # mod - The module to instrument. + def self.instrument_instance_methods(mod) + mod.public_instance_methods(false).each do |name| + method = mod.instance_method(name) + + if method.owner == mod + if !block_given? || block_given? && yield(mod, method) + instrument_instance_method(mod, name) + end + end + end + end + + # Instruments a method. + # + # type - The type (:class or :instance) of method to instrument. + # mod - The module containing the method. + # name - The name of the method to instrument. + def self.instrument(type, mod, name) + return unless Metrics.enabled? + + name = name.to_sym + alias_name = :"_original_#{name}" + target = type == :instance ? mod : mod.singleton_class + + if type == :instance + target = mod + label = "#{mod.name}##{name}" + else + target = mod.singleton_class + label = "#{mod.name}.#{name}" + end + + target.class_eval <<-EOF, __FILE__, __LINE__ + 1 + alias_method #{alias_name.inspect}, #{name.inspect} + + def #{name}(*args, &block) + trans = Gitlab::Metrics::Instrumentation.transaction + + if trans + start = Time.now + retval = __send__(#{alias_name.inspect}, *args, &block) + duration = (Time.now - start) * 1000.0 + + if duration >= Gitlab::Metrics.method_call_threshold + trans.add_metric(Gitlab::Metrics::Instrumentation::SERIES, + { duration: duration }, + method: #{label.inspect}) + end + + retval + else + __send__(#{alias_name.inspect}, *args, &block) + end + end + EOF + end + + # Small layer of indirection to make it easier to stub out the current + # transaction. + def self.transaction + Transaction.current + end + end + end +end diff --git a/lib/gitlab/metrics/metric.rb b/lib/gitlab/metrics/metric.rb new file mode 100644 index 00000000000..79241f56874 --- /dev/null +++ b/lib/gitlab/metrics/metric.rb @@ -0,0 +1,34 @@ +module Gitlab + module Metrics + # Class for storing details of a single metric (label, value, etc). + class Metric + attr_reader :series, :values, :tags, :created_at + + # series - The name of the series (as a String) to store the metric in. + # values - A Hash containing the values to store. + # tags - A Hash containing extra tags to add to the metrics. + def initialize(series, values, tags = {}) + @values = values + @series = series + @tags = tags + @created_at = Time.now.utc + end + + # Returns a Hash in a format that can be directly written to InfluxDB. + def to_hash + { + series: @series, + tags: @tags.merge( + hostname: Metrics.hostname, + ruby_engine: RUBY_ENGINE, + ruby_version: RUBY_VERSION, + gitlab_version: Gitlab::VERSION, + process_type: Sidekiq.server? ? 'sidekiq' : 'rails' + ), + values: @values, + timestamp: @created_at.to_i * 1_000_000_000 + } + end + end + end +end diff --git a/lib/gitlab/metrics/obfuscated_sql.rb b/lib/gitlab/metrics/obfuscated_sql.rb new file mode 100644 index 00000000000..fe97d7a0534 --- /dev/null +++ b/lib/gitlab/metrics/obfuscated_sql.rb @@ -0,0 +1,47 @@ +module Gitlab + module Metrics + # Class for producing SQL queries with sensitive data stripped out. + class ObfuscatedSQL + REPLACEMENT = / + \d+(\.\d+)? # integers, floats + | '.+?' # single quoted strings + | \/.+?(?<!\\)\/ # regexps (including escaped slashes) + /x + + MYSQL_REPLACEMENTS = / + ".+?" # double quoted strings + /x + + # Regex to replace consecutive placeholders with a single one indicating + # the length. This can be useful when a "IN" statement uses thousands of + # IDs (storing this would just be a waste of space). + CONSECUTIVE = /(\?(\s*,\s*)?){2,}/ + + # sql - The raw SQL query as a String. + def initialize(sql) + @sql = sql + end + + # Returns a new, obfuscated SQL query. + def to_s + regex = REPLACEMENT + + if Gitlab::Database.mysql? + regex = Regexp.union(regex, MYSQL_REPLACEMENTS) + end + + sql = @sql.gsub(regex, '?').gsub(CONSECUTIVE) do |match| + "#{match.count(',') + 1} values" + end + + # InfluxDB escapes double quotes upon output, so lets get rid of them + # whenever we can. + if Gitlab::Database.postgresql? + sql = sql.delete('"') + end + + sql.tr("\n", ' ') + end + end + end +end diff --git a/lib/gitlab/metrics/rack_middleware.rb b/lib/gitlab/metrics/rack_middleware.rb new file mode 100644 index 00000000000..5c0587c4c51 --- /dev/null +++ b/lib/gitlab/metrics/rack_middleware.rb @@ -0,0 +1,49 @@ +module Gitlab + module Metrics + # Rack middleware for tracking Rails requests. + class RackMiddleware + CONTROLLER_KEY = 'action_controller.instance' + + def initialize(app) + @app = app + end + + # env - A Hash containing Rack environment details. + def call(env) + trans = transaction_from_env(env) + retval = nil + + begin + retval = trans.run { @app.call(env) } + + # Even in the event of an error we want to submit any metrics we + # might've gathered up to this point. + ensure + if env[CONTROLLER_KEY] + tag_controller(trans, env) + end + + trans.finish + end + + retval + end + + def transaction_from_env(env) + trans = Transaction.new + + trans.add_tag(:request_method, env['REQUEST_METHOD']) + trans.add_tag(:request_uri, env['REQUEST_URI']) + + trans + end + + def tag_controller(trans, env) + controller = env[CONTROLLER_KEY] + label = "#{controller.class.name}##{controller.action_name}" + + trans.add_tag(:action, label) + end + end + end +end diff --git a/lib/gitlab/metrics/sampler.rb b/lib/gitlab/metrics/sampler.rb new file mode 100644 index 00000000000..998578e1c0a --- /dev/null +++ b/lib/gitlab/metrics/sampler.rb @@ -0,0 +1,98 @@ +module Gitlab + module Metrics + # Class that sends certain metrics to InfluxDB at a specific interval. + # + # This class is used to gather statistics that can't be directly associated + # with a transaction such as system memory usage, garbage collection + # statistics, etc. + class Sampler + # interval - The sampling interval in seconds. + def initialize(interval = 15) + @interval = interval + @metrics = [] + + @last_minor_gc = Delta.new(GC.stat[:minor_gc_count]) + @last_major_gc = Delta.new(GC.stat[:major_gc_count]) + + if Gitlab::Metrics.mri? + require 'allocations' + + Allocations.start + end + end + + def start + Thread.new do + Thread.current.abort_on_exception = true + + loop do + sleep(@interval) + + sample + end + end + end + + def sample + sample_memory_usage + sample_file_descriptors + sample_objects + sample_gc + + flush + ensure + GC::Profiler.clear + @metrics.clear + end + + def flush + Metrics.submit_metrics(@metrics.map(&:to_hash)) + end + + def sample_memory_usage + @metrics << Metric.new('memory_usage', value: System.memory_usage) + end + + def sample_file_descriptors + @metrics << Metric. + new('file_descriptors', value: System.file_descriptor_count) + end + + if Metrics.mri? + def sample_objects + sample = Allocations.to_hash + counts = sample.each_with_object({}) do |(klass, count), hash| + hash[klass.name] = count + end + + # Symbols aren't allocated so we'll need to add those manually. + counts['Symbol'] = Symbol.all_symbols.length + + counts.each do |name, count| + @metrics << Metric.new('object_counts', { count: count }, type: name) + end + end + else + def sample_objects + end + end + + def sample_gc + time = GC::Profiler.total_time * 1000.0 + stats = GC.stat.merge(total_time: time) + + # We want the difference of GC runs compared to the last sample, not the + # total amount since the process started. + stats[:minor_gc_count] = + @last_minor_gc.compared_with(stats[:minor_gc_count]) + + stats[:major_gc_count] = + @last_major_gc.compared_with(stats[:major_gc_count]) + + stats[:count] = stats[:minor_gc_count] + stats[:major_gc_count] + + @metrics << Metric.new('gc_statistics', stats) + end + end + end +end diff --git a/lib/gitlab/metrics/sidekiq_middleware.rb b/lib/gitlab/metrics/sidekiq_middleware.rb new file mode 100644 index 00000000000..ad441decfa2 --- /dev/null +++ b/lib/gitlab/metrics/sidekiq_middleware.rb @@ -0,0 +1,23 @@ +module Gitlab + module Metrics + # Sidekiq middleware for tracking jobs. + # + # This middleware is intended to be used as a server-side middleware. + class SidekiqMiddleware + def call(worker, message, queue) + trans = Transaction.new + + begin + trans.run { yield } + ensure + tag_worker(trans, worker) + trans.finish + end + end + + def tag_worker(trans, worker) + trans.add_tag(:action, "#{worker.class.name}#perform") + end + end + end +end diff --git a/lib/gitlab/metrics/subscribers/action_view.rb b/lib/gitlab/metrics/subscribers/action_view.rb new file mode 100644 index 00000000000..7e0dcf99d92 --- /dev/null +++ b/lib/gitlab/metrics/subscribers/action_view.rb @@ -0,0 +1,53 @@ +module Gitlab + module Metrics + module Subscribers + # Class for tracking the rendering timings of views. + class ActionView < ActiveSupport::Subscriber + attach_to :action_view + + SERIES = 'views' + + def render_template(event) + track(event) if current_transaction + end + + alias_method :render_view, :render_template + + private + + def track(event) + values = values_for(event) + tags = tags_for(event) + + current_transaction.add_metric(SERIES, values, tags) + end + + def relative_path(path) + path.gsub(/^#{Rails.root.to_s}\/?/, '') + end + + def values_for(event) + { duration: event.duration } + end + + def tags_for(event) + path = relative_path(event.payload[:identifier]) + tags = { view: path } + + file, line = Metrics.last_relative_application_frame + + if file and line + tags[:file] = file + tags[:line] = line + end + + tags + end + + def current_transaction + Transaction.current + end + end + end + end +end diff --git a/lib/gitlab/metrics/subscribers/active_record.rb b/lib/gitlab/metrics/subscribers/active_record.rb new file mode 100644 index 00000000000..d947c128ce2 --- /dev/null +++ b/lib/gitlab/metrics/subscribers/active_record.rb @@ -0,0 +1,48 @@ +module Gitlab + module Metrics + module Subscribers + # Class for tracking raw SQL queries. + # + # Queries are obfuscated before being logged to ensure no private data is + # exposed via InfluxDB/Grafana. + class ActiveRecord < ActiveSupport::Subscriber + attach_to :active_record + + SERIES = 'sql_queries' + + def sql(event) + return unless current_transaction + + values = values_for(event) + tags = tags_for(event) + + current_transaction.add_metric(SERIES, values, tags) + end + + private + + def values_for(event) + { duration: event.duration } + end + + def tags_for(event) + sql = ObfuscatedSQL.new(event.payload[:sql]).to_s + tags = { sql: sql } + + file, line = Metrics.last_relative_application_frame + + if file and line + tags[:file] = file + tags[:line] = line + end + + tags + end + + def current_transaction + Transaction.current + end + end + end + end +end diff --git a/lib/gitlab/metrics/system.rb b/lib/gitlab/metrics/system.rb new file mode 100644 index 00000000000..83371265278 --- /dev/null +++ b/lib/gitlab/metrics/system.rb @@ -0,0 +1,35 @@ +module Gitlab + module Metrics + # Module for gathering system/process statistics such as the memory usage. + # + # This module relies on the /proc filesystem being available. If /proc is + # not available the methods of this module will be stubbed. + module System + if File.exist?('/proc') + # Returns the current process' memory usage in bytes. + def self.memory_usage + mem = 0 + match = File.read('/proc/self/status').match(/VmRSS:\s+(\d+)/) + + if match and match[1] + mem = match[1].to_f * 1024 + end + + mem + end + + def self.file_descriptor_count + Dir.glob('/proc/self/fd/*').length + end + else + def self.memory_usage + 0.0 + end + + def self.file_descriptor_count + 0 + end + end + end + end +end diff --git a/lib/gitlab/metrics/transaction.rb b/lib/gitlab/metrics/transaction.rb new file mode 100644 index 00000000000..a61dbd989e7 --- /dev/null +++ b/lib/gitlab/metrics/transaction.rb @@ -0,0 +1,66 @@ +module Gitlab + module Metrics + # Class for storing metrics information of a single transaction. + class Transaction + THREAD_KEY = :_gitlab_metrics_transaction + + SERIES = 'transactions' + + attr_reader :uuid, :tags + + def self.current + Thread.current[THREAD_KEY] + end + + # name - The name of this transaction as a String. + def initialize + @metrics = [] + @uuid = SecureRandom.uuid + + @started_at = nil + @finished_at = nil + + @tags = {} + end + + def duration + @finished_at ? (@finished_at - @started_at) * 1000.0 : 0.0 + end + + def run + Thread.current[THREAD_KEY] = self + + @started_at = Time.now + + yield + ensure + @finished_at = Time.now + + Thread.current[THREAD_KEY] = nil + end + + def add_metric(series, values, tags = {}) + tags = tags.merge(transaction_id: @uuid) + + @metrics << Metric.new(series, values, tags) + end + + def add_tag(key, value) + @tags[key] = value + end + + def finish + track_self + submit + end + + def track_self + add_metric(SERIES, { duration: duration }, @tags) + end + + def submit + Metrics.submit_metrics(@metrics.map(&:to_hash)) + end + end + end +end diff --git a/lib/gitlab/recaptcha.rb b/lib/gitlab/recaptcha.rb new file mode 100644 index 00000000000..70e7f25d518 --- /dev/null +++ b/lib/gitlab/recaptcha.rb @@ -0,0 +1,14 @@ +module Gitlab + module Recaptcha + def self.load_configurations! + if current_application_settings.recaptcha_enabled + ::Recaptcha.configure do |config| + config.public_key = current_application_settings.recaptcha_site_key + config.private_key = current_application_settings.recaptcha_private_key + end + + true + end + end + end +end diff --git a/lib/gitlab/reference_extractor.rb b/lib/gitlab/reference_extractor.rb index 0a70d21b1ce..be795649e59 100644 --- a/lib/gitlab/reference_extractor.rb +++ b/lib/gitlab/reference_extractor.rb @@ -3,11 +3,12 @@ require 'banzai' module Gitlab # Extract possible GFM references from an arbitrary String for further processing. class ReferenceExtractor < Banzai::ReferenceExtractor - attr_accessor :project, :current_user + attr_accessor :project, :current_user, :author - def initialize(project, current_user = nil) + def initialize(project, current_user = nil, author = nil) @project = project @current_user = current_user + @author = author @references = {} @@ -20,18 +21,22 @@ module Gitlab %i(user label merge_request snippet commit commit_range).each do |type| define_method("#{type}s") do - @references[type] ||= references(type, project: project, current_user: current_user) + @references[type] ||= references(type, reference_context) end end def issues - options = { project: project, current_user: current_user } - if project && project.jira_tracker? - @references[:external_issue] ||= references(:external_issue, options) + @references[:external_issue] ||= references(:external_issue, reference_context) else - @references[:issue] ||= references(:issue, options) + @references[:issue] ||= references(:issue, reference_context) end end + + private + + def reference_context + { project: project, current_user: current_user, author: author } + end end end diff --git a/spec/features/login_spec.rb b/spec/features/login_spec.rb index 922c76285d1..2451e56fe7c 100644 --- a/spec/features/login_spec.rb +++ b/spec/features/login_spec.rb @@ -98,4 +98,56 @@ feature 'Login', feature: true do expect(page).to have_content('Invalid login or password.') end end + + describe 'with required two-factor authentication enabled' do + let(:user) { create(:user) } + before(:each) { stub_application_setting(require_two_factor_authentication: true) } + + context 'with grace period defined' do + before(:each) do + stub_application_setting(two_factor_grace_period: 48) + login_with(user) + end + + context 'within the grace period' do + it 'redirects to two-factor configuration page' do + expect(current_path).to eq new_profile_two_factor_auth_path + expect(page).to have_content('You must configure Two-Factor Authentication in your account until') + end + + it 'two-factor configuration is skippable' do + expect(current_path).to eq new_profile_two_factor_auth_path + + click_link 'Configure it later' + expect(current_path).to eq root_path + end + end + + context 'after the grace period' do + let(:user) { create(:user, otp_grace_period_started_at: 9999.hours.ago) } + + it 'redirects to two-factor configuration page' do + expect(current_path).to eq new_profile_two_factor_auth_path + expect(page).to have_content('You must configure Two-Factor Authentication in your account.') + end + + it 'two-factor configuration is not skippable' do + expect(current_path).to eq new_profile_two_factor_auth_path + expect(page).not_to have_link('Configure it later') + end + end + end + + context 'without grace pariod defined' do + before(:each) do + stub_application_setting(two_factor_grace_period: 0) + login_with(user) + end + + it 'redirects to two-factor configuration page' do + expect(current_path).to eq new_profile_two_factor_auth_path + expect(page).to have_content('You must configure Two-Factor Authentication in your account.') + end + end + end end diff --git a/spec/helpers/issues_helper_spec.rb b/spec/helpers/issues_helper_spec.rb index 04e795025d2..ffd8ebae029 100644 --- a/spec/helpers/issues_helper_spec.rb +++ b/spec/helpers/issues_helper_spec.rb @@ -141,4 +141,11 @@ describe IssuesHelper do expect(note_active_class(Note.all, @note.author)).to eq("active") end end + + describe "#awards_sort" do + it "sorts a hash so thumbsup and thumbsdown are always on top" do + data = { "thumbsdown" => "some value", "lifter" => "some value", "thumbsup" => "some value" } + expect(awards_sort(data).keys).to eq(["thumbsup", "thumbsdown", "lifter"]) + end + end end diff --git a/spec/helpers/page_layout_helper_spec.rb b/spec/helpers/page_layout_helper_spec.rb new file mode 100644 index 00000000000..fd7107779f6 --- /dev/null +++ b/spec/helpers/page_layout_helper_spec.rb @@ -0,0 +1,129 @@ +require 'rails_helper' + +describe PageLayoutHelper do + describe 'page_description' do + it 'defaults to value returned by page_description_default helper' do + allow(helper).to receive(:page_description_default).and_return('Foo') + + expect(helper.page_description).to eq 'Foo' + end + + it 'returns the last-pushed description' do + helper.page_description('Foo') + helper.page_description('Bar') + helper.page_description('Baz') + + expect(helper.page_description).to eq 'Baz' + end + + it 'squishes multiple newlines' do + helper.page_description("Foo\nBar\nBaz") + + expect(helper.page_description).to eq 'Foo Bar Baz' + end + + it 'truncates' do + helper.page_description <<-LOREM.strip_heredoc + Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo + ligula eget dolor. Aenean massa. Cum sociis natoque penatibus et magnis + dis parturient montes, nascetur ridiculus mus. Donec quam felis, + ultricies nec, pellentesque eu, pretium quis, sem. Nulla consequat massa + quis enim. Donec pede justo, fringilla vel, aliquet nec, vulputate eget, + arcu. + LOREM + + expect(helper.page_description).to end_with 'quam felis,...' + end + + it 'sanitizes all HTML' do + helper.page_description("<b>Bold</b> <h1>Header</h1>") + + expect(helper.page_description).to eq 'Bold Header' + end + end + + describe 'page_description_default' do + it 'uses Project description when available' do + project = double(description: 'Project Description') + helper.instance_variable_set(:@project, project) + + expect(helper.page_description_default).to eq 'Project Description' + end + + it 'uses brand_title when Project description is nil' do + project = double(description: nil) + helper.instance_variable_set(:@project, project) + + expect(helper).to receive(:brand_title).and_return('Brand Title') + expect(helper.page_description_default).to eq 'Brand Title' + end + + it 'falls back to brand_title' do + allow(helper).to receive(:brand_title).and_return('Brand Title') + + expect(helper.page_description_default).to eq 'Brand Title' + end + end + + describe 'page_image' do + it 'defaults to the GitLab logo' do + expect(helper.page_image).to end_with 'assets/gitlab_logo.png' + end + + context 'with @project' do + it 'uses Project avatar if available' do + project = double(avatar_url: 'http://example.com/uploads/avatar.png') + helper.instance_variable_set(:@project, project) + + expect(helper.page_image).to eq project.avatar_url + end + + it 'falls back to the default' do + project = double(avatar_url: nil) + helper.instance_variable_set(:@project, project) + + expect(helper.page_image).to end_with 'assets/gitlab_logo.png' + end + end + + context 'with @user' do + it 'delegates to avatar_icon helper' do + user = double('User') + helper.instance_variable_set(:@user, user) + + expect(helper).to receive(:avatar_icon).with(user) + + helper.page_image + end + end + end + + describe 'page_card_attributes' do + it 'raises ArgumentError when given more than two attributes' do + map = { foo: 'foo', bar: 'bar', baz: 'baz' } + + expect { helper.page_card_attributes(map) }. + to raise_error(ArgumentError, /more than two attributes/) + end + + it 'rejects blank values' do + map = { foo: 'foo', bar: '' } + helper.page_card_attributes(map) + + expect(helper.page_card_attributes).to eq({ foo: 'foo' }) + end + end + + describe 'page_card_meta_tags' do + it 'returns the twitter:label and twitter:data tags' do + allow(helper).to receive(:page_card_attributes).and_return(foo: 'bar') + + tags = helper.page_card_meta_tags + + aggregate_failures do + expect(tags).to include %q(<meta property="twitter:label1" content="foo" />) + expect(tags).to include %q(<meta property="twitter:data1" content="bar" />) + end + end + end +end diff --git a/spec/javascripts/fixtures/issues_show.html.haml b/spec/javascripts/fixtures/issues_show.html.haml index 8447dfdda32..470cabeafbb 100644 --- a/spec/javascripts/fixtures/issues_show.html.haml +++ b/spec/javascripts/fixtures/issues_show.html.haml @@ -1,4 +1,14 @@ -%a.btn-close +:css + .hidden { display: none !important; } + +.flash-container + .flash-alert + .flash-notice + +.status-box.status-box-open Open +.status-box.status-box-closed.hidden Closed +%a.btn-close{"href" => "http://gitlab.com/issues/6/close"} Close +%a.btn-reopen.hidden{"href" => "http://gitlab.com/issues/6/reopen"} Reopen .detail-page-description .description.js-task-list-container diff --git a/spec/javascripts/issue_spec.js.coffee b/spec/javascripts/issue_spec.js.coffee index 268e4c68c31..7e67c778861 100644 --- a/spec/javascripts/issue_spec.js.coffee +++ b/spec/javascripts/issue_spec.js.coffee @@ -20,3 +20,89 @@ describe 'Issue', -> expect(req.data.issue.description).not.toBe(null) $('.js-task-list-field').trigger('tasklist:changed') +describe 'reopen/close issue', -> + fixture.preload('issues_show.html') + beforeEach -> + fixture.load('issues_show.html') + @issue = new Issue() + it 'closes an issue', -> + $.ajax = (obj) -> + expect(obj.type).toBe('PUT') + expect(obj.url).toBe('http://gitlab.com/issues/6/close') + obj.success saved: true + + $btnClose = $('a.btn-close') + $btnReopen = $('a.btn-reopen') + expect($btnReopen).toBeHidden() + expect($btnClose.text()).toBe('Close') + expect(typeof $btnClose.prop('disabled')).toBe('undefined') + + $btnClose.trigger('click') + + expect($btnReopen).toBeVisible() + expect($btnClose).toBeHidden() + expect($('div.status-box-closed')).toBeVisible() + expect($('div.status-box-open')).toBeHidden() + + it 'fails to closes an issue with success:false', -> + + $.ajax = (obj) -> + expect(obj.type).toBe('PUT') + expect(obj.url).toBe('http://goesnowhere.nothing/whereami') + obj.success saved: false + + $btnClose = $('a.btn-close') + $btnReopen = $('a.btn-reopen') + $btnClose.attr('href','http://goesnowhere.nothing/whereami') + expect($btnReopen).toBeHidden() + expect($btnClose.text()).toBe('Close') + expect(typeof $btnClose.prop('disabled')).toBe('undefined') + + $btnClose.trigger('click') + + expect($btnReopen).toBeHidden() + expect($btnClose).toBeVisible() + expect($('div.status-box-closed')).toBeHidden() + expect($('div.status-box-open')).toBeVisible() + expect($('div.flash-alert')).toBeVisible() + expect($('div.flash-alert').text()).toBe('Unable to update this issue at this time.') + + it 'fails to closes an issue with HTTP error', -> + + $.ajax = (obj) -> + expect(obj.type).toBe('PUT') + expect(obj.url).toBe('http://goesnowhere.nothing/whereami') + obj.error() + + $btnClose = $('a.btn-close') + $btnReopen = $('a.btn-reopen') + $btnClose.attr('href','http://goesnowhere.nothing/whereami') + expect($btnReopen).toBeHidden() + expect($btnClose.text()).toBe('Close') + expect(typeof $btnClose.prop('disabled')).toBe('undefined') + + $btnClose.trigger('click') + + expect($btnReopen).toBeHidden() + expect($btnClose).toBeVisible() + expect($('div.status-box-closed')).toBeHidden() + expect($('div.status-box-open')).toBeVisible() + expect($('div.flash-alert')).toBeVisible() + expect($('div.flash-alert').text()).toBe('Unable to update this issue at this time.') + + it 'reopens an issue', -> + $.ajax = (obj) -> + expect(obj.type).toBe('PUT') + expect(obj.url).toBe('http://gitlab.com/issues/6/reopen') + obj.success saved: true + + $btnClose = $('a.btn-close') + $btnReopen = $('a.btn-reopen') + expect($btnReopen.text()).toBe('Reopen') + + $btnReopen.trigger('click') + + expect($btnReopen).toBeHidden() + expect($btnClose).toBeVisible() + expect($('div.status-box-open')).toBeVisible() + expect($('div.status-box-closed')).toBeHidden()
\ No newline at end of file diff --git a/spec/lib/banzai/filter/user_reference_filter_spec.rb b/spec/lib/banzai/filter/user_reference_filter_spec.rb index 3534bf97784..8bdebae1841 100644 --- a/spec/lib/banzai/filter/user_reference_filter_spec.rb +++ b/spec/lib/banzai/filter/user_reference_filter_spec.rb @@ -37,9 +37,22 @@ describe Banzai::Filter::UserReferenceFilter, lib: true do .to eq urls.namespace_project_url(project.namespace, project) end - it 'adds to the results hash' do - result = reference_pipeline_result("Hey #{reference}") - expect(result[:references][:user]).to eq [project.creator] + context "when the author is a member of the project" do + + it 'adds to the results hash' do + result = reference_pipeline_result("Hey #{reference}", author: project.creator) + expect(result[:references][:user]).to eq [project.creator] + end + end + + context "when the author is not a member of the project" do + + let(:other_user) { create(:user) } + + it "doesn't add to the results hash" do + result = reference_pipeline_result("Hey #{reference}", author: other_user) + expect(result[:references][:user]).to eq [] + end end end diff --git a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb index c90133fbf03..d15100fc6d8 100644 --- a/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb +++ b/spec/lib/ci/gitlab_ci_yaml_processor_spec.rb @@ -3,7 +3,7 @@ require 'spec_helper' module Ci describe GitlabCiYamlProcessor, lib: true do let(:path) { 'path' } - + describe "#builds_for_ref" do let(:type) { 'test' } @@ -29,7 +29,7 @@ module Ci when: "on_success" }) end - + describe :only do it "does not return builds if only has another branch" do config = YAML.dump({ @@ -517,7 +517,7 @@ module Ci end.to raise_error(GitlabCiYamlProcessor::ValidationError, "Unknown parameter: extra") end - it "returns errors if there is no any jobs defined" do + it "returns errors if there are no jobs defined" do config = YAML.dump({ before_script: ["bundle update"] }) expect do GitlabCiYamlProcessor.new(config, path) diff --git a/spec/lib/gitlab/metrics/delta_spec.rb b/spec/lib/gitlab/metrics/delta_spec.rb new file mode 100644 index 00000000000..718387cdee1 --- /dev/null +++ b/spec/lib/gitlab/metrics/delta_spec.rb @@ -0,0 +1,16 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Delta do + let(:delta) { described_class.new } + + describe '#compared_with' do + it 'returns the delta as a Numeric' do + expect(delta.compared_with(5)).to eq(5) + end + + it 'bases the delta on a previously used value' do + expect(delta.compared_with(5)).to eq(5) + expect(delta.compared_with(15)).to eq(10) + end + end +end diff --git a/spec/lib/gitlab/metrics/instrumentation_spec.rb b/spec/lib/gitlab/metrics/instrumentation_spec.rb new file mode 100644 index 00000000000..a7eab9d11cc --- /dev/null +++ b/spec/lib/gitlab/metrics/instrumentation_spec.rb @@ -0,0 +1,234 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Instrumentation do + let(:transaction) { Gitlab::Metrics::Transaction.new } + + before do + @dummy = Class.new do + def self.foo(text = 'foo') + text + end + + def bar(text = 'bar') + text + end + end + + allow(@dummy).to receive(:name).and_return('Dummy') + end + + describe '.configure' do + it 'yields self' do + described_class.configure do |c| + expect(c).to eq(described_class) + end + end + end + + describe '.instrument_method' do + describe 'with metrics enabled' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(true) + + described_class.instrument_method(@dummy, :foo) + end + + it 'renames the original method' do + expect(@dummy).to respond_to(:_original_foo) + end + + it 'calls the instrumented method with the correct arguments' do + expect(@dummy.foo).to eq('foo') + end + + it 'tracks the call duration upon calling the method' do + allow(Gitlab::Metrics).to receive(:method_call_threshold). + and_return(0) + + allow(described_class).to receive(:transaction). + and_return(transaction) + + expect(transaction).to receive(:add_metric). + with(described_class::SERIES, an_instance_of(Hash), + method: 'Dummy.foo') + + @dummy.foo + end + + it 'does not track method calls below a given duration threshold' do + allow(Gitlab::Metrics).to receive(:method_call_threshold). + and_return(100) + + expect(transaction).to_not receive(:add_metric) + + @dummy.foo + end + end + + describe 'with metrics disabled' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(false) + end + + it 'does not instrument the method' do + described_class.instrument_method(@dummy, :foo) + + expect(@dummy).to_not respond_to(:_original_foo) + end + end + end + + describe '.instrument_instance_method' do + describe 'with metrics enabled' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(true) + + described_class. + instrument_instance_method(@dummy, :bar) + end + + it 'renames the original method' do + expect(@dummy.method_defined?(:_original_bar)).to eq(true) + end + + it 'calls the instrumented method with the correct arguments' do + expect(@dummy.new.bar).to eq('bar') + end + + it 'tracks the call duration upon calling the method' do + allow(Gitlab::Metrics).to receive(:method_call_threshold). + and_return(0) + + allow(described_class).to receive(:transaction). + and_return(transaction) + + expect(transaction).to receive(:add_metric). + with(described_class::SERIES, an_instance_of(Hash), + method: 'Dummy#bar') + + @dummy.new.bar + end + + it 'does not track method calls below a given duration threshold' do + allow(Gitlab::Metrics).to receive(:method_call_threshold). + and_return(100) + + expect(transaction).to_not receive(:add_metric) + + @dummy.new.bar + end + end + + describe 'with metrics disabled' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(false) + end + + it 'does not instrument the method' do + described_class. + instrument_instance_method(@dummy, :bar) + + expect(@dummy.method_defined?(:_original_bar)).to eq(false) + end + end + end + + describe '.instrument_class_hierarchy' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(true) + + @child1 = Class.new(@dummy) do + def self.child1_foo; end + def child1_bar; end + end + + @child2 = Class.new(@child1) do + def self.child2_foo; end + def child2_bar; end + end + end + + it 'recursively instruments a class hierarchy' do + described_class.instrument_class_hierarchy(@dummy) + + expect(@child1).to respond_to(:_original_child1_foo) + expect(@child2).to respond_to(:_original_child2_foo) + + expect(@child1.method_defined?(:_original_child1_bar)).to eq(true) + expect(@child2.method_defined?(:_original_child2_bar)).to eq(true) + end + + it 'does not instrument the root module' do + described_class.instrument_class_hierarchy(@dummy) + + expect(@dummy).to_not respond_to(:_original_foo) + expect(@dummy.method_defined?(:_original_bar)).to eq(false) + end + end + + describe '.instrument_methods' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(true) + end + + it 'instruments all public class methods' do + described_class.instrument_methods(@dummy) + + expect(@dummy).to respond_to(:_original_foo) + end + + it 'only instruments methods directly defined in the module' do + mod = Module.new do + def kittens + end + end + + @dummy.extend(mod) + + described_class.instrument_methods(@dummy) + + expect(@dummy).to_not respond_to(:_original_kittens) + end + + it 'can take a block to determine if a method should be instrumented' do + described_class.instrument_methods(@dummy) do + false + end + + expect(@dummy).to_not respond_to(:_original_foo) + end + end + + describe '.instrument_instance_methods' do + before do + allow(Gitlab::Metrics).to receive(:enabled?).and_return(true) + end + + it 'instruments all public instance methods' do + described_class.instrument_instance_methods(@dummy) + + expect(@dummy.method_defined?(:_original_bar)).to eq(true) + end + + it 'only instruments methods directly defined in the module' do + mod = Module.new do + def kittens + end + end + + @dummy.include(mod) + + described_class.instrument_instance_methods(@dummy) + + expect(@dummy.method_defined?(:_original_kittens)).to eq(false) + end + + it 'can take a block to determine if a method should be instrumented' do + described_class.instrument_instance_methods(@dummy) do + false + end + + expect(@dummy.method_defined?(:_original_bar)).to eq(false) + end + end +end diff --git a/spec/lib/gitlab/metrics/metric_spec.rb b/spec/lib/gitlab/metrics/metric_spec.rb new file mode 100644 index 00000000000..ec39bc9cce8 --- /dev/null +++ b/spec/lib/gitlab/metrics/metric_spec.rb @@ -0,0 +1,57 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Metric do + let(:metric) do + described_class.new('foo', { number: 10 }, { host: 'localtoast' }) + end + + describe '#series' do + subject { metric.series } + + it { is_expected.to eq('foo') } + end + + describe '#values' do + subject { metric.values } + + it { is_expected.to eq({ number: 10 }) } + end + + describe '#tags' do + subject { metric.tags } + + it { is_expected.to eq({ host: 'localtoast' }) } + end + + describe '#to_hash' do + it 'returns a Hash' do + expect(metric.to_hash).to be_an_instance_of(Hash) + end + + describe 'the returned Hash' do + let(:hash) { metric.to_hash } + + it 'includes the series' do + expect(hash[:series]).to eq('foo') + end + + it 'includes the tags' do + expect(hash[:tags]).to be_an_instance_of(Hash) + + expect(hash[:tags][:hostname]).to be_an_instance_of(String) + expect(hash[:tags][:ruby_engine]).to be_an_instance_of(String) + expect(hash[:tags][:ruby_version]).to be_an_instance_of(String) + expect(hash[:tags][:gitlab_version]).to be_an_instance_of(String) + expect(hash[:tags][:process_type]).to be_an_instance_of(String) + end + + it 'includes the values' do + expect(hash[:values]).to eq({ number: 10 }) + end + + it 'includes the timestamp' do + expect(hash[:timestamp]).to be_an_instance_of(Fixnum) + end + end + end +end diff --git a/spec/lib/gitlab/metrics/obfuscated_sql_spec.rb b/spec/lib/gitlab/metrics/obfuscated_sql_spec.rb new file mode 100644 index 00000000000..2b681c9fe34 --- /dev/null +++ b/spec/lib/gitlab/metrics/obfuscated_sql_spec.rb @@ -0,0 +1,93 @@ +require 'spec_helper' + +describe Gitlab::Metrics::ObfuscatedSQL do + describe '#to_s' do + it 'replaces newlines with a space' do + sql = described_class.new("SELECT x\nFROM y") + + expect(sql.to_s).to eq('SELECT x FROM y') + end + + describe 'using single values' do + it 'replaces a single integer' do + sql = described_class.new('SELECT x FROM y WHERE a = 10') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + + it 'replaces a single float' do + sql = described_class.new('SELECT x FROM y WHERE a = 10.5') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + + it 'replaces a single quoted string' do + sql = described_class.new("SELECT x FROM y WHERE a = 'foo'") + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + + if Gitlab::Database.mysql? + it 'replaces a double quoted string' do + sql = described_class.new('SELECT x FROM y WHERE a = "foo"') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + end + + it 'replaces a single regular expression' do + sql = described_class.new('SELECT x FROM y WHERE a = /foo/') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + + it 'replaces regular expressions using escaped slashes' do + sql = described_class.new('SELECT x FROM y WHERE a = /foo\/bar/') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE a = ?') + end + end + + describe 'using consecutive values' do + it 'replaces multiple integers' do + sql = described_class.new('SELECT x FROM y WHERE z IN (10, 20, 30)') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE z IN (3 values)') + end + + it 'replaces multiple floats' do + sql = described_class.new('SELECT x FROM y WHERE z IN (1.5, 2.5, 3.5)') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE z IN (3 values)') + end + + it 'replaces multiple single quoted strings' do + sql = described_class.new("SELECT x FROM y WHERE z IN ('foo', 'bar')") + + expect(sql.to_s).to eq('SELECT x FROM y WHERE z IN (2 values)') + end + + if Gitlab::Database.mysql? + it 'replaces multiple double quoted strings' do + sql = described_class.new('SELECT x FROM y WHERE z IN ("foo", "bar")') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE z IN (2 values)') + end + end + + it 'replaces multiple regular expressions' do + sql = described_class.new('SELECT x FROM y WHERE z IN (/foo/, /bar/)') + + expect(sql.to_s).to eq('SELECT x FROM y WHERE z IN (2 values)') + end + end + + if Gitlab::Database.postgresql? + it 'replaces double quotes' do + sql = described_class.new('SELECT "x" FROM "y"') + + expect(sql.to_s).to eq('SELECT x FROM y') + end + end + end +end diff --git a/spec/lib/gitlab/metrics/rack_middleware_spec.rb b/spec/lib/gitlab/metrics/rack_middleware_spec.rb new file mode 100644 index 00000000000..a143fe4cfcd --- /dev/null +++ b/spec/lib/gitlab/metrics/rack_middleware_spec.rb @@ -0,0 +1,63 @@ +require 'spec_helper' + +describe Gitlab::Metrics::RackMiddleware do + let(:app) { double(:app) } + + let(:middleware) { described_class.new(app) } + + let(:env) { { 'REQUEST_METHOD' => 'GET', 'REQUEST_URI' => '/foo' } } + + describe '#call' do + before do + expect_any_instance_of(Gitlab::Metrics::Transaction).to receive(:finish) + end + + it 'tracks a transaction' do + expect(app).to receive(:call).with(env).and_return('yay') + + expect(middleware.call(env)).to eq('yay') + end + + it 'tags a transaction with the name and action of a controller' do + klass = double(:klass, name: 'TestController') + controller = double(:controller, class: klass, action_name: 'show') + + env['action_controller.instance'] = controller + + allow(app).to receive(:call).with(env) + + expect(middleware).to receive(:tag_controller). + with(an_instance_of(Gitlab::Metrics::Transaction), env) + + middleware.call(env) + end + end + + describe '#transaction_from_env' do + let(:transaction) { middleware.transaction_from_env(env) } + + it 'returns a Transaction' do + expect(transaction).to be_an_instance_of(Gitlab::Metrics::Transaction) + end + + it 'tags the transaction with the request method and URI' do + expect(transaction.tags[:request_method]).to eq('GET') + expect(transaction.tags[:request_uri]).to eq('/foo') + end + end + + describe '#tag_controller' do + let(:transaction) { middleware.transaction_from_env(env) } + + it 'tags a transaction with the name and action of a controller' do + klass = double(:klass, name: 'TestController') + controller = double(:controller, class: klass, action_name: 'show') + + env['action_controller.instance'] = controller + + middleware.tag_controller(transaction, env) + + expect(transaction.tags[:action]).to eq('TestController#show') + end + end +end diff --git a/spec/lib/gitlab/metrics/sampler_spec.rb b/spec/lib/gitlab/metrics/sampler_spec.rb new file mode 100644 index 00000000000..51a941c48cd --- /dev/null +++ b/spec/lib/gitlab/metrics/sampler_spec.rb @@ -0,0 +1,97 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Sampler do + let(:sampler) { described_class.new(5) } + + after do + Allocations.stop if Gitlab::Metrics.mri? + end + + describe '#start' do + it 'gathers a sample at a given interval' do + expect(sampler).to receive(:sleep).with(5) + expect(sampler).to receive(:sample) + expect(sampler).to receive(:loop).and_yield + + sampler.start.join + end + end + + describe '#sample' do + it 'samples various statistics' do + expect(sampler).to receive(:sample_memory_usage) + expect(sampler).to receive(:sample_file_descriptors) + expect(sampler).to receive(:sample_objects) + expect(sampler).to receive(:sample_gc) + expect(sampler).to receive(:flush) + + sampler.sample + end + + it 'clears any GC profiles' do + expect(sampler).to receive(:flush) + expect(GC::Profiler).to receive(:clear) + + sampler.sample + end + end + + describe '#flush' do + it 'schedules the metrics using Sidekiq' do + expect(Gitlab::Metrics).to receive(:submit_metrics). + with([an_instance_of(Hash)]) + + sampler.sample_memory_usage + sampler.flush + end + end + + describe '#sample_memory_usage' do + it 'adds a metric containing the memory usage' do + expect(Gitlab::Metrics::System).to receive(:memory_usage). + and_return(9000) + + expect(Gitlab::Metrics::Metric).to receive(:new). + with('memory_usage', value: 9000). + and_call_original + + sampler.sample_memory_usage + end + end + + describe '#sample_file_descriptors' do + it 'adds a metric containing the amount of open file descriptors' do + expect(Gitlab::Metrics::System).to receive(:file_descriptor_count). + and_return(4) + + expect(Gitlab::Metrics::Metric).to receive(:new). + with('file_descriptors', value: 4). + and_call_original + + sampler.sample_file_descriptors + end + end + + describe '#sample_objects' do + it 'adds a metric containing the amount of allocated objects' do + expect(Gitlab::Metrics::Metric).to receive(:new). + with('object_counts', an_instance_of(Hash), an_instance_of(Hash)). + at_least(:once). + and_call_original + + sampler.sample_objects + end + end + + describe '#sample_gc' do + it 'adds a metric containing garbage collection statistics' do + expect(GC::Profiler).to receive(:total_time).and_return(0.24) + + expect(Gitlab::Metrics::Metric).to receive(:new). + with('gc_statistics', an_instance_of(Hash)). + and_call_original + + sampler.sample_gc + end + end +end diff --git a/spec/lib/gitlab/metrics/sidekiq_middleware_spec.rb b/spec/lib/gitlab/metrics/sidekiq_middleware_spec.rb new file mode 100644 index 00000000000..5882e7d81c7 --- /dev/null +++ b/spec/lib/gitlab/metrics/sidekiq_middleware_spec.rb @@ -0,0 +1,26 @@ +require 'spec_helper' + +describe Gitlab::Metrics::SidekiqMiddleware do + let(:middleware) { described_class.new } + + describe '#call' do + it 'tracks the transaction' do + worker = Class.new.new + + expect_any_instance_of(Gitlab::Metrics::Transaction).to receive(:finish) + + middleware.call(worker, 'test', :test) { nil } + end + end + + describe '#tag_worker' do + it 'adds the worker class and action to the transaction' do + trans = Gitlab::Metrics::Transaction.new + worker = double(:worker, class: double(:class, name: 'TestWorker')) + + expect(trans).to receive(:add_tag).with(:action, 'TestWorker#perform') + + middleware.tag_worker(trans, worker) + end + end +end diff --git a/spec/lib/gitlab/metrics/subscribers/action_view_spec.rb b/spec/lib/gitlab/metrics/subscribers/action_view_spec.rb new file mode 100644 index 00000000000..c6cd584663f --- /dev/null +++ b/spec/lib/gitlab/metrics/subscribers/action_view_spec.rb @@ -0,0 +1,37 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Subscribers::ActionView do + let(:transaction) { Gitlab::Metrics::Transaction.new } + + let(:subscriber) { described_class.new } + + let(:event) do + root = Rails.root.to_s + + double(:event, duration: 2.1, + payload: { identifier: "#{root}/app/views/x.html.haml" }) + end + + before do + allow(subscriber).to receive(:current_transaction).and_return(transaction) + + allow(Gitlab::Metrics).to receive(:last_relative_application_frame). + and_return(['app/views/x.html.haml', 4]) + end + + describe '#render_template' do + it 'tracks rendering of a template' do + values = { duration: 2.1 } + tags = { + view: 'app/views/x.html.haml', + file: 'app/views/x.html.haml', + line: 4 + } + + expect(transaction).to receive(:add_metric). + with(described_class::SERIES, values, tags) + + subscriber.render_template(event) + end + end +end diff --git a/spec/lib/gitlab/metrics/subscribers/active_record_spec.rb b/spec/lib/gitlab/metrics/subscribers/active_record_spec.rb new file mode 100644 index 00000000000..05b6cc14716 --- /dev/null +++ b/spec/lib/gitlab/metrics/subscribers/active_record_spec.rb @@ -0,0 +1,32 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Subscribers::ActiveRecord do + let(:transaction) { Gitlab::Metrics::Transaction.new } + + let(:subscriber) { described_class.new } + + let(:event) do + double(:event, duration: 0.2, + payload: { sql: 'SELECT * FROM users WHERE id = 10' }) + end + + before do + allow(subscriber).to receive(:current_transaction).and_return(transaction) + + allow(Gitlab::Metrics).to receive(:last_relative_application_frame). + and_return(['app/models/foo.rb', 4]) + end + + describe '#sql' do + it 'tracks the execution of a SQL query' do + sql = 'SELECT * FROM users WHERE id = ?' + values = { duration: 0.2 } + tags = { sql: sql, file: 'app/models/foo.rb', line: 4 } + + expect(transaction).to receive(:add_metric). + with(described_class::SERIES, values, tags) + + subscriber.sql(event) + end + end +end diff --git a/spec/lib/gitlab/metrics/system_spec.rb b/spec/lib/gitlab/metrics/system_spec.rb new file mode 100644 index 00000000000..f8c1d956ca1 --- /dev/null +++ b/spec/lib/gitlab/metrics/system_spec.rb @@ -0,0 +1,29 @@ +require 'spec_helper' + +describe Gitlab::Metrics::System do + if File.exist?('/proc') + describe '.memory_usage' do + it "returns the process' memory usage in bytes" do + expect(described_class.memory_usage).to be > 0 + end + end + + describe '.file_descriptor_count' do + it 'returns the amount of open file descriptors' do + expect(described_class.file_descriptor_count).to be > 0 + end + end + else + describe '.memory_usage' do + it 'returns 0.0' do + expect(described_class.memory_usage).to eq(0.0) + end + end + + describe '.file_descriptor_count' do + it 'returns 0' do + expect(described_class.file_descriptor_count).to eq(0) + end + end + end +end diff --git a/spec/lib/gitlab/metrics/transaction_spec.rb b/spec/lib/gitlab/metrics/transaction_spec.rb new file mode 100644 index 00000000000..6862fc9e2d1 --- /dev/null +++ b/spec/lib/gitlab/metrics/transaction_spec.rb @@ -0,0 +1,77 @@ +require 'spec_helper' + +describe Gitlab::Metrics::Transaction do + let(:transaction) { described_class.new } + + describe '#duration' do + it 'returns the duration of a transaction in seconds' do + transaction.run { sleep(0.5) } + + expect(transaction.duration).to be >= 0.5 + end + end + + describe '#run' do + it 'yields the supplied block' do + expect { |b| transaction.run(&b) }.to yield_control + end + + it 'stores the transaction in the current thread' do + transaction.run do + expect(Thread.current[described_class::THREAD_KEY]).to eq(transaction) + end + end + + it 'removes the transaction from the current thread upon completion' do + transaction.run { } + + expect(Thread.current[described_class::THREAD_KEY]).to be_nil + end + end + + describe '#add_metric' do + it 'adds a metric tagged with the transaction UUID' do + expect(Gitlab::Metrics::Metric).to receive(:new). + with('foo', { number: 10 }, { transaction_id: transaction.uuid }) + + transaction.add_metric('foo', number: 10) + end + end + + describe '#add_tag' do + it 'adds a tag' do + transaction.add_tag(:foo, 'bar') + + expect(transaction.tags).to eq({ foo: 'bar' }) + end + end + + describe '#finish' do + it 'tracks the transaction details and submits them to Sidekiq' do + expect(transaction).to receive(:track_self) + expect(transaction).to receive(:submit) + + transaction.finish + end + end + + describe '#track_self' do + it 'adds a metric for the transaction itself' do + expect(transaction).to receive(:add_metric). + with(described_class::SERIES, { duration: transaction.duration }, {}) + + transaction.track_self + end + end + + describe '#submit' do + it 'submits the metrics to Sidekiq' do + transaction.track_self + + expect(Gitlab::Metrics).to receive(:submit_metrics). + with([an_instance_of(Hash)]) + + transaction.submit + end + end +end diff --git a/spec/lib/gitlab/metrics_spec.rb b/spec/lib/gitlab/metrics_spec.rb new file mode 100644 index 00000000000..6c0682cac4d --- /dev/null +++ b/spec/lib/gitlab/metrics_spec.rb @@ -0,0 +1,84 @@ +require 'spec_helper' + +describe Gitlab::Metrics do + describe '.pool_size' do + it 'returns a Fixnum' do + expect(described_class.pool_size).to be_an_instance_of(Fixnum) + end + end + + describe '.timeout' do + it 'returns a Fixnum' do + expect(described_class.timeout).to be_an_instance_of(Fixnum) + end + end + + describe '.enabled?' do + it 'returns a boolean' do + expect([true, false].include?(described_class.enabled?)).to eq(true) + end + end + + describe '.hostname' do + it 'returns a String containing the hostname' do + expect(described_class.hostname).to eq(Socket.gethostname) + end + end + + describe '.last_relative_application_frame' do + it 'returns an Array containing a file path and line number' do + file, line = described_class.last_relative_application_frame + + expect(line).to eq(__LINE__ - 2) + expect(file).to eq('spec/lib/gitlab/metrics_spec.rb') + end + end + + describe '#submit_metrics' do + it 'prepares and writes the metrics to InfluxDB' do + connection = double(:connection) + pool = double(:pool) + + expect(pool).to receive(:with).and_yield(connection) + expect(connection).to receive(:write_points).with(an_instance_of(Array)) + expect(Gitlab::Metrics).to receive(:pool).and_return(pool) + + described_class.submit_metrics([{ 'series' => 'kittens', 'tags' => {} }]) + end + end + + describe '#prepare_metrics' do + it 'returns a Hash with the keys as Symbols' do + metrics = described_class. + prepare_metrics([{ 'values' => {}, 'tags' => {} }]) + + expect(metrics).to eq([{ values: {}, tags: {} }]) + end + + it 'escapes tag values' do + metrics = described_class.prepare_metrics([ + { 'values' => {}, 'tags' => { 'foo' => 'bar=' } } + ]) + + expect(metrics).to eq([{ values: {}, tags: { 'foo' => 'bar\\=' } }]) + end + + it 'drops empty tags' do + metrics = described_class.prepare_metrics([ + { 'values' => {}, 'tags' => { 'cats' => '', 'dogs' => nil } } + ]) + + expect(metrics).to eq([{ values: {}, tags: {} }]) + end + end + + describe '#escape_value' do + it 'escapes an equals sign' do + expect(described_class.escape_value('foo=')).to eq('foo\\=') + end + + it 'casts values to Strings' do + expect(described_class.escape_value(10)).to eq('10') + end + end +end diff --git a/spec/models/application_setting_spec.rb b/spec/models/application_setting_spec.rb index 5f64453a35f..35d8220ae54 100644 --- a/spec/models/application_setting_spec.rb +++ b/spec/models/application_setting_spec.rb @@ -27,6 +27,7 @@ # admin_notification_email :string(255) # shared_runners_enabled :boolean default(TRUE), not null # max_artifacts_size :integer default(100), not null +# runners_registration_token :string(255) # require 'spec_helper' diff --git a/spec/models/concerns/issuable_spec.rb b/spec/models/concerns/issuable_spec.rb index 0f13c4410cd..021d62cdf0c 100644 --- a/spec/models/concerns/issuable_spec.rb +++ b/spec/models/concerns/issuable_spec.rb @@ -81,4 +81,36 @@ describe Issue, "Issuable" do expect(hook_data[:object_attributes]).to eq(issue.hook_attrs) end end + + describe '#card_attributes' do + it 'includes the author name' do + allow(issue).to receive(:author).and_return(double(name: 'Robert')) + allow(issue).to receive(:assignee).and_return(nil) + + expect(issue.card_attributes). + to eq({ 'Author' => 'Robert', 'Assignee' => nil }) + end + + it 'includes the assignee name' do + allow(issue).to receive(:author).and_return(double(name: 'Robert')) + allow(issue).to receive(:assignee).and_return(double(name: 'Douwe')) + + expect(issue.card_attributes). + to eq({ 'Author' => 'Robert', 'Assignee' => 'Douwe' }) + end + end + + describe "votes" do + before do + author = create :user + project = create :empty_project + issue.notes.awards.create!(note: "thumbsup", author: author, project: project) + issue.notes.awards.create!(note: "thumbsdown", author: author, project: project) + end + + it "returns correct values" do + expect(issue.upvotes).to eq(1) + expect(issue.downvotes).to eq(1) + end + end end diff --git a/spec/models/concerns/mentionable_spec.rb b/spec/models/concerns/mentionable_spec.rb index 6653621a83e..20f0c561e44 100644 --- a/spec/models/concerns/mentionable_spec.rb +++ b/spec/models/concerns/mentionable_spec.rb @@ -3,6 +3,10 @@ require 'spec_helper' describe Mentionable do include Mentionable + def author + nil + end + describe :references do let(:project) { create(:project) } diff --git a/spec/models/note_spec.rb b/spec/models/note_spec.rb index b7006fa5e68..593d8f76215 100644 --- a/spec/models/note_spec.rb +++ b/spec/models/note_spec.rb @@ -137,9 +137,14 @@ describe Note, models: true do create :note, note: "smile", is_award: true end - it "returns grouped array of notes" do - expect(Note.grouped_awards.first.first).to eq("smile") - expect(Note.grouped_awards.first.last).to match_array(Note.all) + it "returns grouped hash of notes" do + expect(Note.grouped_awards.keys.size).to eq(3) + expect(Note.grouped_awards["smile"]).to match_array(Note.all) + end + + it "returns thumbsup and thumbsdown always" do + expect(Note.grouped_awards["thumbsup"]).to match_array(Note.none) + expect(Note.grouped_awards["thumbsdown"]).to match_array(Note.none) end end diff --git a/spec/requests/api/users_spec.rb b/spec/requests/api/users_spec.rb index 2f609c63330..4f278551d07 100644 --- a/spec/requests/api/users_spec.rb +++ b/spec/requests/api/users_spec.rb @@ -27,6 +27,13 @@ describe API::API, api: true do user['username'] == username end['username']).to eq(username) end + + it "should return one user" do + get api("/users?username=#{omniauth_user.username}", user) + expect(response.status).to eq(200) + expect(json_response).to be_an Array + expect(json_response.first['username']).to eq(omniauth_user.username) + end end context "when admin" do diff --git a/spec/services/notification_service_spec.rb b/spec/services/notification_service_spec.rb index d7a898e85ff..c103752198d 100644 --- a/spec/services/notification_service_spec.rb +++ b/spec/services/notification_service_spec.rb @@ -115,6 +115,7 @@ describe NotificationService, services: true do before do build_team(note.project) + note.project.team << [note.author, :master] ActionMailer::Base.deliveries.clear end @@ -126,6 +127,8 @@ describe NotificationService, services: true do note.project.team.members.each do |member| # User with disabled notification should not be notified next if member.id == @u_disabled.id + # Author should not be notified + next if member.id == note.author.id should_email(member) end |
