diff options
| author | Mayra Cabrera <mcabrera@gitlab.com> | 2018-06-07 17:48:32 +0000 |
|---|---|---|
| committer | Mayra Cabrera <mcabrera@gitlab.com> | 2018-06-07 17:48:32 +0000 |
| commit | a73ff5985ca3a9da87c81c67a22967540655888c (patch) | |
| tree | 9038eb88f61013ad5f987622dbdb77698b520891 | |
| parent | c3c14bef67a7b092f81b9f1b44a7ae29d2567cf7 (diff) | |
| parent | cd5f483f7d22b9824bd062f25b8d641bd40c474b (diff) | |
| download | gitlab-ce-a73ff5985ca3a9da87c81c67a22967540655888c.tar.gz | |
Merge branch '11-0-stable-prepare-rc6' into '11-0-stable'
Prepare 11.0 RC6 release
See merge request gitlab-org/gitlab-ce!19527
585 files changed, 11321 insertions, 2930 deletions
diff --git a/.gitlab/issue_templates/Feature Proposal.md b/.gitlab/issue_templates/Feature Proposal.md index 5b55eb1374b..c4065d3c4ea 100644 --- a/.gitlab/issue_templates/Feature Proposal.md +++ b/.gitlab/issue_templates/Feature Proposal.md @@ -1,9 +1,15 @@ -### Description +### Problem to solve -(Include problem, use cases, benefits, and/or goals) +### Further details + +(Include use cases, benefits, and/or goals) ### Proposal +### What does success look like, and how can we measure that? + +(If no way to measure success, link to an issue that will implement a way to measure this) + ### Links / references /label ~"feature proposal" diff --git a/CHANGELOG.md b/CHANGELOG.md index ec92829f7d1..0d843c3f318 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ documentation](doc/development/changelog.md) for instructions on adding your own entry. +## 10.8.4 (2018-06-06) + +- No changes. + ## 10.8.3 (2018-05-30) ### Fixed (4 changes) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 64470a1f087..82d1abff4a4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -182,7 +182,7 @@ Assigning a team label makes sure issues get the attention of the appropriate people. The current team labels are ~Distribution, ~"CI/CD", ~Discussion, ~Documentation, ~Quality, -~Geo, ~Gitaly, ~Monitoring, ~Platform, ~Release, ~"Security Products" and ~"UX". +~Geo, ~Gitaly, ~Monitoring, ~Platform, ~Release, ~"Security Products", ~"Configuration", and ~"UX". The descriptions on the [labels page][labels-page] explain what falls under the responsibility of each team. @@ -349,7 +349,7 @@ on those issues. Please select someone with relevant experience from the [GitLab team][team]. If there is nobody mentioned with that expertise look in the commit history for the affected files to find someone. -[described in our handbook]: https://about.gitlab.com/handbook/engineering/issues/issue-triage-policies/ +[described in our handbook]: https://about.gitlab.com/handbook/engineering/issue-triage/ [issue bash events]: https://gitlab.com/gitlab-org/gitlab-ce/issues/17815 ### Feature proposals @@ -512,7 +512,7 @@ request is as follows: 1. Write [tests](https://docs.gitlab.com/ee/development/rake_tasks.html#run-tests) and code 1. [Generate a changelog entry with `bin/changelog`][changelog] 1. If you are writing documentation, make sure to follow the - [documentation styleguide][doc-styleguide] + [documentation guidelines][doc-guidelines] 1. If you have multiple commits please combine them into a few logically organized commits by [squashing them][git-squash] 1. Push the commit(s) to your fork @@ -727,7 +727,7 @@ available at [http://contributor-covenant.org/version/1/1/0/](http://contributor [rss-source]: https://github.com/bbatsov/ruby-style-guide/blob/master/README.md#source-code-layout [rss-naming]: https://github.com/bbatsov/ruby-style-guide/blob/master/README.md#naming [changelog]: doc/development/changelog.md "Generate a changelog entry" -[doc-styleguide]: doc/development/doc_styleguide.md "Documentation styleguide" +[doc-guidelines]: doc/development/documentation/index.md "Documentation guidelines" [js-styleguide]: doc/development/fe_guide/style_guide_js.md "JavaScript styleguide" [scss-styleguide]: doc/development/fe_guide/style_guide_scss.md "SCSS styleguide" [newlines-styleguide]: doc/development/newlines_styleguide.md "Newlines styleguide" diff --git a/GITALY_SERVER_VERSION b/GITALY_SERVER_VERSION index e49057b3302..f1b9cc4cd95 100644 --- a/GITALY_SERVER_VERSION +++ b/GITALY_SERVER_VERSION @@ -1 +1 @@ -0.104.0 +0.105.0 diff --git a/GITLAB_SHELL_VERSION b/GITLAB_SHELL_VERSION index 1996c504476..b7f8ee41e69 100644 --- a/GITLAB_SHELL_VERSION +++ b/GITLAB_SHELL_VERSION @@ -1 +1 @@ -7.1.3 +7.1.4 @@ -93,6 +93,10 @@ gem 'grape', '~> 1.0' gem 'grape-entity', '~> 0.7.1' gem 'rack-cors', '~> 1.0.0', require: 'rack/cors' +# GraphQL API +gem 'graphql', '~> 1.8.0' +gem 'graphiql-rails', '~> 1.4.10' + # Disable strong_params so that Mash does not respond to :permitted? gem 'hashie-forbidden_attributes' @@ -374,7 +378,7 @@ end group :test do gem 'shoulda-matchers', '~> 3.1.2', require: false - gem 'email_spec', '~> 1.6.0' + gem 'email_spec', '~> 2.2.0' gem 'json-schema', '~> 2.8.0' gem 'webmock', '~> 2.3.2' gem 'rails-controller-testing' if rails5? # Rails5 only gem. @@ -404,13 +408,12 @@ gem 'vmstat', '~> 2.3.0' gem 'sys-filesystem', '~> 1.1.6' # SSH host key support -gem 'net-ssh', '~> 4.2.0' +gem 'net-ssh', '~> 5.0' gem 'sshkey', '~> 1.9.0' # Required for ED25519 SSH host key support group :ed25519 do - gem 'rbnacl-libsodium' - gem 'rbnacl', '~> 4.0' + gem 'ed25519', '~> 1.2' gem 'bcrypt_pbkdf', '~> 1.0' end diff --git a/Gemfile.lock b/Gemfile.lock index e6e8f3d11bc..20fe5daa58e 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -172,15 +172,17 @@ GEM unf (>= 0.0.5, < 1.0.0) doorkeeper (4.3.2) railties (>= 4.2) - doorkeeper-openid_connect (1.3.0) + doorkeeper-openid_connect (1.4.0) doorkeeper (~> 4.3) json-jwt (~> 1.6) dropzonejs-rails (0.7.2) rails (> 3.1) + ed25519 (1.2.4) email_reply_trimmer (0.1.6) - email_spec (1.6.0) + email_spec (2.2.0) + htmlentities (~> 4.3.3) launchy (~> 2.1) - mail (~> 2.2) + mail (~> 2.7) encryptor (3.0.0) equalizer (0.0.11) erubis (2.7.0) @@ -358,12 +360,16 @@ GEM grape-entity (0.7.1) activesupport (>= 4.0) multi_json (>= 1.3.2) - grape-path-helpers (1.0.1) + grape-path-helpers (1.0.2) activesupport (~> 4) grape (~> 1.0) rake (~> 12) grape_logging (1.7.0) grape + graphiql-rails (1.4.10) + railties + sprockets-rails + graphql (1.8.1) grpc (1.11.0) google-protobuf (~> 3.1) googleapis-common-protos-types (~> 1.0.0) @@ -505,7 +511,7 @@ GEM mustermann (~> 1.0.0) mysql2 (0.4.10) net-ldap (0.16.0) - net-ssh (4.2.0) + net-ssh (5.0.1) netrc (0.11.0) nokogiri (1.8.2) mini_portile2 (~> 2.3.0) @@ -691,10 +697,6 @@ GEM ffi (>= 0.5.0, < 2) rblineprof (0.3.6) debugger-ruby_core_source (~> 1.3) - rbnacl (4.0.2) - ffi - rbnacl-libsodium (1.0.11) - rbnacl (>= 3.0.1) rdoc (6.0.4) re2 (1.1.1) recaptcha (3.0.0) @@ -1011,8 +1013,9 @@ DEPENDENCIES doorkeeper (~> 4.3) doorkeeper-openid_connect (~> 1.3) dropzonejs-rails (~> 0.7.1) + ed25519 (~> 1.2) email_reply_trimmer (~> 0.1) - email_spec (~> 1.6.0) + email_spec (~> 2.2.0) factory_bot_rails (~> 4.8.2) faraday (~> 0.12) fast_blank @@ -1052,6 +1055,8 @@ DEPENDENCIES grape-entity (~> 0.7.1) grape-path-helpers (~> 1.0) grape_logging (~> 1.7) + graphiql-rails (~> 1.4.10) + graphql (~> 1.8.0) grpc (~> 1.11.0) haml_lint (~> 0.26.0) hamlit (~> 2.6.1) @@ -1081,7 +1086,7 @@ DEPENDENCIES mousetrap-rails (~> 1.4.6) mysql2 (~> 0.4.10) net-ldap - net-ssh (~> 4.2.0) + net-ssh (~> 5.0) nokogiri (~> 1.8.2) oauth2 (~> 1.4) octokit (~> 4.9) @@ -1123,8 +1128,6 @@ DEPENDENCIES rainbow (~> 2.2) raindrops (~> 0.18) rblineprof (~> 0.3.6) - rbnacl (~> 4.0) - rbnacl-libsodium rdoc (~> 6.0) re2 (~> 1.1.1) recaptcha (~> 3.0) diff --git a/Gemfile.rails5.lock b/Gemfile.rails5.lock index af7305619eb..bee13dd1d53 100644 --- a/Gemfile.rails5.lock +++ b/Gemfile.rails5.lock @@ -72,8 +72,6 @@ GEM attr_encrypted (3.1.0) encryptor (~> 3.0.0) attr_required (1.0.1) - autoprefixer-rails (8.1.0.1) - execjs awesome_print (1.2.0) axiom-types (0.1.1) descendants_tracker (~> 0.0.4) @@ -93,9 +91,6 @@ GEM binding_of_caller (0.7.3) debug_inspector (>= 0.0.1) blankslate (2.1.2.4) - bootstrap-sass (3.3.7) - autoprefixer-rails (>= 5.2.1) - sass (>= 3.3.4) bootstrap_form (2.7.0) brakeman (4.2.1) browser (2.5.3) @@ -175,7 +170,7 @@ GEM diff-lcs (1.3) diffy (3.1.0) docile (1.1.5) - domain_name (0.5.20170404) + domain_name (0.5.20180417) unf (>= 0.0.5, < 1.0.0) doorkeeper (4.3.1) railties (>= 4.2) @@ -184,10 +179,12 @@ GEM json-jwt (~> 1.6) dropzonejs-rails (0.7.4) rails (> 3.1) + ed25519 (1.2.4) email_reply_trimmer (0.1.10) - email_spec (1.6.0) + email_spec (2.2.0) + htmlentities (~> 4.3.3) launchy (~> 2.1) - mail (~> 2.2) + mail (~> 2.7) encryptor (3.0.0) equalizer (0.0.11) erubis (2.7.0) @@ -288,7 +285,7 @@ GEM gettext_i18n_rails (>= 0.7.1) po_to_json (>= 1.0.0) rails (>= 3.2.0) - gitaly-proto (0.99.0) + gitaly-proto (0.100.0) google-protobuf (~> 3.1) grpc (~> 1.10) github-linguist (5.3.3) @@ -365,9 +362,9 @@ GEM grape-entity (0.7.1) activesupport (>= 4.0) multi_json (>= 1.3.2) - grape-route-helpers (2.1.0) + grape-path-helpers (1.0.0) activesupport - grape (>= 0.16.0) + grape (~> 1.0) rake grape_logging (1.7.0) grape @@ -417,6 +414,7 @@ GEM httpclient (2.8.3) i18n (1.0.1) concurrent-ruby (~> 1.0) + icalendar (2.4.1) ice_nine (0.11.2) influxdb (0.5.3) ipaddress (0.8.3) @@ -450,9 +448,9 @@ GEM kgio (2.11.2) knapsack (1.16.0) rake - kubeclient (3.0.0) + kubeclient (3.1.1) http (~> 2.2.2) - recursive-open-struct (~> 1.0.4) + recursive-open-struct (~> 1.0, >= 1.0.4) rest-client (~> 2.0) launchy (2.4.3) addressable (~> 2.3) @@ -508,7 +506,7 @@ GEM mustermann (~> 1.0.0) mysql2 (0.4.10) net-ldap (0.16.1) - net-ssh (4.2.0) + net-ssh (5.0.1) netrc (0.11.0) nio4r (2.3.1) nokogiri (1.8.2) @@ -521,15 +519,16 @@ GEM multi_json (~> 1.3) multi_xml (~> 0.5) rack (>= 1.2, < 3) - octokit (4.8.0) + octokit (4.9.0) sawyer (~> 0.8.0, >= 0.5.3) omniauth (1.8.1) hashie (>= 3.4.6, < 3.6.0) rack (>= 1.6.2, < 3) omniauth-auth0 (2.0.0) omniauth-oauth2 (~> 1.4) - omniauth-authentiq (0.3.1) - omniauth-oauth2 (~> 1.3, >= 1.3.1) + omniauth-authentiq (0.3.3) + jwt (>= 1.5) + omniauth-oauth2 (>= 1.5) omniauth-azure-oauth2 (0.0.9) jwt (~> 1.0) omniauth (~> 1.0) @@ -628,7 +627,7 @@ GEM parser unparser procto (0.0.3) - prometheus-client-mmap (0.9.2) + prometheus-client-mmap (0.9.3) pry (0.11.3) coderay (~> 1.1.0) method_source (~> 0.9.0) @@ -698,15 +697,11 @@ GEM ffi (>= 0.5.0, < 2) rblineprof (0.3.7) debugger-ruby_core_source (~> 1.3) - rbnacl (4.0.2) - ffi - rbnacl-libsodium (1.0.16) - rbnacl (>= 3.0.1) - rdoc (4.3.0) + rdoc (6.0.4) re2 (1.1.1) recaptcha (3.4.0) json - recursive-open-struct (1.0.5) + recursive-open-struct (1.1.0) redcarpet (3.4.0) redis (3.3.5) redis-actionpack (5.0.2) @@ -716,8 +711,8 @@ GEM redis-activesupport (5.0.4) activesupport (>= 3, < 6) redis-store (>= 1.3, < 2) - redis-namespace (1.5.3) - redis (~> 3.0, >= 3.0.4) + redis-namespace (1.6.0) + redis (>= 3.0.4) redis-rack (2.0.4) rack (>= 1.5, < 3) redis-store (>= 1.2, < 2) @@ -836,7 +831,7 @@ GEM activesupport (>= 3.1) select2-rails (3.5.10) thor (~> 0.14) - selenium-webdriver (3.11.0) + selenium-webdriver (3.12.0) childprocess (~> 0.5) rubyzip (~> 1.2) sentry-raven (2.7.2) @@ -986,7 +981,7 @@ DEPENDENCIES asciidoctor-plantuml (= 0.0.8) asset_sync (~> 2.4) attr_encrypted (~> 3.1.0) - awesome_print (~> 1.2.0) + awesome_print babosa (~> 1.0.2) base32 (~> 0.3.0) batch-loader (~> 1.2.1) @@ -994,7 +989,6 @@ DEPENDENCIES benchmark-ips (~> 2.3.0) better_errors (~> 2.1.0) binding_of_caller (~> 0.7.2) - bootstrap-sass (~> 3.3.0) bootstrap_form (~> 2.7.0) brakeman (~> 4.2) browser (~> 2.2) @@ -1020,8 +1014,9 @@ DEPENDENCIES doorkeeper (~> 4.3) doorkeeper-openid_connect (~> 1.3) dropzonejs-rails (~> 0.7.1) + ed25519 (~> 1.2) email_reply_trimmer (~> 0.1) - email_spec (~> 1.6.0) + email_spec (~> 2.2.0) factory_bot_rails (~> 4.8.2) faraday (~> 0.12) fast_blank @@ -1045,7 +1040,7 @@ DEPENDENCIES gettext (~> 3.2.2) gettext_i18n_rails (~> 1.8.0) gettext_i18n_rails_js (~> 1.3) - gitaly-proto (~> 0.99.0) + gitaly-proto (~> 0.100.0) github-linguist (~> 5.3.3) gitlab-flowdock-git-hook (~> 1.0.1) gitlab-gollum-lib (~> 4.2) @@ -1059,7 +1054,7 @@ DEPENDENCIES gpgme grape (~> 1.0) grape-entity (~> 0.7.1) - grape-route-helpers (~> 2.1.0) + grape-path-helpers (~> 1.0) grape_logging (~> 1.7) grpc (~> 1.11.0) haml_lint (~> 0.26.0) @@ -1070,6 +1065,7 @@ DEPENDENCIES html-pipeline (~> 2.7.1) html2text httparty (~> 0.13.3) + icalendar influxdb (~> 0.2) jira-ruby (~> 1.4) jquery-atwho-rails (~> 1.3.2) @@ -1077,7 +1073,7 @@ DEPENDENCIES jwt (~> 1.5.6) kaminari (~> 1.0) knapsack (~> 1.16) - kubeclient (~> 3.0) + kubeclient (~> 3.1.0) letter_opener_web (~> 1.3.0) license_finder (~> 3.1) licensee (~> 8.9) @@ -1089,13 +1085,13 @@ DEPENDENCIES mousetrap-rails (~> 1.4.6) mysql2 (~> 0.4.10) net-ldap - net-ssh (~> 4.2.0) + net-ssh (~> 5.0) nokogiri (~> 1.8.2) oauth2 (~> 1.4) - octokit (~> 4.8) + octokit (~> 4.9) omniauth (~> 1.8) omniauth-auth0 (~> 2.0.0) - omniauth-authentiq (~> 0.3.1) + omniauth-authentiq (~> 0.3.3) omniauth-azure-oauth2 (~> 0.0.9) omniauth-cas3 (~> 1.1.4) omniauth-facebook (~> 4.0.0) @@ -1118,7 +1114,7 @@ DEPENDENCIES peek-sidekiq (~> 1.0.3) pg (~> 0.18.2) premailer-rails (~> 1.9.7) - prometheus-client-mmap (~> 0.9.2) + prometheus-client-mmap (~> 0.9.3) pry-byebug (~> 3.4.1) pry-rails (~> 0.3.4) rack-attack (~> 4.4.1) @@ -1132,14 +1128,12 @@ DEPENDENCIES rainbow (~> 2.2) raindrops (~> 0.18) rblineprof (~> 0.3.6) - rbnacl (~> 4.0) - rbnacl-libsodium - rdoc (~> 4.2) + rdoc (~> 6.0) re2 (~> 1.1.1) recaptcha (~> 3.0) redcarpet (~> 3.4) redis (~> 3.2) - redis-namespace (~> 1.5.2) + redis-namespace (~> 1.6.0) redis-rails (~> 5.0.2) request_store (~> 1.3) responders (~> 2.0) @@ -1154,6 +1148,7 @@ DEPENDENCIES rubocop-rspec (~> 1.22.1) ruby-fogbugz (~> 0.2.1) ruby-prof (~> 0.17.0) + ruby-progressbar ruby_parser (~> 3.8) rufus-scheduler (~> 3.4) rugged (~> 0.27) @@ -1162,12 +1157,12 @@ DEPENDENCIES scss_lint (~> 0.56.0) seed-fu (~> 2.3.7) select2-rails (~> 3.5.9) - selenium-webdriver (~> 3.5) + selenium-webdriver (~> 3.12) sentry-raven (~> 2.7) settingslogic (~> 2.0.9) sham_rack (~> 1.3.6) shoulda-matchers (~> 3.1.2) - sidekiq (~> 5.0) + sidekiq (~> 5.1) sidekiq-cron (~> 0.6.0) sidekiq-limit_fetch (~> 3.4) simple_po_parser (~> 1.1.2) @@ -1199,4 +1194,4 @@ DEPENDENCIES wikicloth (= 0.8.1) BUNDLED WITH - 1.16.1 + 1.16.2 diff --git a/PROCESS.md b/PROCESS.md index f206506f7c5..7438df8014b 100644 --- a/PROCESS.md +++ b/PROCESS.md @@ -168,6 +168,7 @@ the stable branch are: * Fixes for [regressions](#regressions) * Fixes for security issues +* Fixes or improvements to automated QA scenarios * New or updated translations (as long as they do not touch application code) During the feature freeze all merge requests that are meant to go into the diff --git a/README.md b/README.md index 0266fe82c82..8bd667b3dac 100644 --- a/README.md +++ b/README.md @@ -126,5 +126,5 @@ Please see [Getting help for GitLab](https://about.gitlab.com/getting-help/) on ## Is it awesome? -Thanks for [asking this question](https://twitter.com/supersloth/status/489462789384056832) Joshua. [These people](https://twitter.com/gitlab/likes) seem to like it. + diff --git a/app/assets/javascripts/clusters/components/application_row.vue b/app/assets/javascripts/clusters/components/application_row.vue index 30567993322..98c0b9c22a8 100644 --- a/app/assets/javascripts/clusters/components/application_row.vue +++ b/app/assets/javascripts/clusters/components/application_row.vue @@ -187,7 +187,7 @@ role="row" > <div - class="alert alert-danger alert-block append-bottom-0" + class="alert alert-danger alert-block append-bottom-0 clusters-error-alert" role="gridcell" > <div> diff --git a/app/assets/javascripts/ide/components/ide_review.vue b/app/assets/javascripts/ide/components/ide_review.vue index 0c9ec3b00f0..99fa2465a84 100644 --- a/app/assets/javascripts/ide/components/ide_review.vue +++ b/app/assets/javascripts/ide/components/ide_review.vue @@ -11,17 +11,20 @@ export default { }, computed: { ...mapGetters(['currentMergeRequest']), - ...mapState(['viewer']), + ...mapState(['viewer', 'currentMergeRequestId']), showLatestChangesText() { - return !this.currentMergeRequest || this.viewer === viewerTypes.diff; + return !this.currentMergeRequestId || this.viewer === viewerTypes.diff; }, showMergeRequestText() { - return this.currentMergeRequest && this.viewer === viewerTypes.mr; + return this.currentMergeRequestId && this.viewer === viewerTypes.mr; + }, + mergeRequestId() { + return `!${this.currentMergeRequest.iid}`; }, }, mounted() { this.$nextTick(() => { - this.updateViewer(this.currentMergeRequest ? viewerTypes.mr : viewerTypes.diff); + this.updateViewer(this.currentMergeRequestId ? viewerTypes.mr : viewerTypes.diff); }); }, methods: { @@ -54,7 +57,11 @@ export default { </template> <template v-else-if="showMergeRequestText"> {{ __('Merge request') }} - (<a :href="currentMergeRequest.web_url">!{{ currentMergeRequest.iid }}</a>) + (<a + v-if="currentMergeRequest" + :href="currentMergeRequest.web_url" + v-text="mergeRequestId" + ></a>) </template> </div> </template> diff --git a/app/assets/javascripts/ide/components/ide_side_bar.vue b/app/assets/javascripts/ide/components/ide_side_bar.vue index 3f980203911..1dc2170edde 100644 --- a/app/assets/javascripts/ide/components/ide_side_bar.vue +++ b/app/assets/javascripts/ide/components/ide_side_bar.vue @@ -1,4 +1,5 @@ <script> +import $ from 'jquery'; import { mapState, mapGetters } from 'vuex'; import ProjectAvatarImage from '~/vue_shared/components/project_avatar/image.vue'; import Icon from '~/vue_shared/components/icon.vue'; @@ -13,6 +14,7 @@ import CommitSection from './repo_commit_section.vue'; import CommitForm from './commit_sidebar/form.vue'; import IdeReview from './ide_review.vue'; import SuccessMessage from './commit_sidebar/success_message.vue'; +import MergeRequestDropdown from './merge_requests/dropdown.vue'; import { activityBarViews } from '../constants'; export default { @@ -32,10 +34,12 @@ export default { CommitForm, IdeReview, SuccessMessage, + MergeRequestDropdown, }, data() { return { showTooltip: false, + showMergeRequestsDropdown: false, }; }, computed: { @@ -46,6 +50,7 @@ export default { 'changedFiles', 'stagedFiles', 'lastCommitMsg', + 'currentMergeRequestId', ]), ...mapGetters(['currentProject', 'someUncommitedChanges']), showSuccessMessage() { @@ -61,9 +66,39 @@ export default { watch: { currentBranchId() { this.$nextTick(() => { + if (!this.$refs.branchId) return; + this.showTooltip = this.$refs.branchId.scrollWidth > this.$refs.branchId.offsetWidth; }); }, + loading() { + this.$nextTick(() => { + this.addDropdownListeners(); + }); + }, + }, + mounted() { + this.addDropdownListeners(); + }, + beforeDestroy() { + $(this.$refs.mergeRequestDropdown) + .off('show.bs.dropdown') + .off('hide.bs.dropdown'); + }, + methods: { + addDropdownListeners() { + if (!this.$refs.mergeRequestDropdown) return; + + $(this.$refs.mergeRequestDropdown) + .on('show.bs.dropdown', () => { + this.toggleMergeRequestDropdown(); + }).on('hide.bs.dropdown', () => { + this.toggleMergeRequestDropdown(); + }); + }, + toggleMergeRequestDropdown() { + this.showMergeRequestsDropdown = !this.showMergeRequestsDropdown; + }, }, }; </script> @@ -88,9 +123,13 @@ export default { </div> </template> <template v-else> - <div class="context-header ide-context-header"> - <a - :href="currentProject.web_url" + <div + class="context-header ide-context-header dropdown" + ref="mergeRequestDropdown" + > + <button + type="button" + data-toggle="dropdown" > <div v-if="currentProject.avatar_url" @@ -114,19 +153,41 @@ export default { <div class="sidebar-context-title"> {{ currentProject.name }} </div> - <div - class="sidebar-context-title ide-sidebar-branch-title" - ref="branchId" - v-tooltip - :title="branchTooltipTitle" - > - <icon - name="branch" - css-classes="append-right-5" - />{{ currentBranchId }} + <div class="d-flex"> + <div + v-if="currentBranchId" + class="sidebar-context-title ide-sidebar-branch-title" + ref="branchId" + v-tooltip + :title="branchTooltipTitle" + > + <icon + name="branch" + css-classes="append-right-5" + />{{ currentBranchId }} + </div> + <div + v-if="currentMergeRequestId" + class="sidebar-context-title ide-sidebar-branch-title" + :class="{ + 'prepend-left-8': currentBranchId + }" + > + <icon + name="git-merge" + css-classes="append-right-5" + />!{{ currentMergeRequestId }} + </div> </div> </div> - </a> + <icon + class="ml-auto" + name="chevron-down" + /> + </button> + <merge-request-dropdown + :show="showMergeRequestsDropdown" + /> </div> <div class="multi-file-commit-panel-inner-scroll"> <component diff --git a/app/assets/javascripts/ide/components/ide_status_bar.vue b/app/assets/javascripts/ide/components/ide_status_bar.vue index 368a2995ed9..e40f137d998 100644 --- a/app/assets/javascripts/ide/components/ide_status_bar.vue +++ b/app/assets/javascripts/ide/components/ide_status_bar.vue @@ -35,9 +35,7 @@ export default { }, watch: { lastCommit() { - if (!this.isPollingInitialized) { - this.initPipelinePolling(); - } + this.initPipelinePolling(); }, }, mounted() { @@ -47,9 +45,8 @@ export default { if (this.intervalId) { clearInterval(this.intervalId); } - if (this.isPollingInitialized) { - this.stopPipelinePolling(); - } + + this.stopPipelinePolling(); }, methods: { ...mapActions('pipelines', ['fetchLatestPipeline', 'stopPipelinePolling']), @@ -59,8 +56,9 @@ export default { }, 1000); }, initPipelinePolling() { - this.fetchLatestPipeline(); - this.isPollingInitialized = true; + if (this.lastCommit) { + this.fetchLatestPipeline(); + } }, commitAgeUpdate() { if (this.lastCommit) { diff --git a/app/assets/javascripts/ide/components/jobs/detail.vue b/app/assets/javascripts/ide/components/jobs/detail.vue new file mode 100644 index 00000000000..4d234a36fe5 --- /dev/null +++ b/app/assets/javascripts/ide/components/jobs/detail.vue @@ -0,0 +1,136 @@ +<script> +import { mapActions, mapState } from 'vuex'; +import _ from 'underscore'; +import { __ } from '../../../locale'; +import tooltip from '../../../vue_shared/directives/tooltip'; +import Icon from '../../../vue_shared/components/icon.vue'; +import ScrollButton from './detail/scroll_button.vue'; +import JobDescription from './detail/description.vue'; + +const scrollPositions = { + top: 0, + bottom: 1, +}; + +export default { + directives: { + tooltip, + }, + components: { + Icon, + ScrollButton, + JobDescription, + }, + data() { + return { + scrollPos: scrollPositions.top, + }; + }, + computed: { + ...mapState('pipelines', ['detailJob']), + isScrolledToBottom() { + return this.scrollPos === scrollPositions.bottom; + }, + isScrolledToTop() { + return this.scrollPos === scrollPositions.top; + }, + jobOutput() { + return this.detailJob.output || __('No messages were logged'); + }, + }, + mounted() { + this.getTrace(); + }, + methods: { + ...mapActions('pipelines', ['fetchJobTrace', 'setDetailJob']), + scrollDown() { + if (this.$refs.buildTrace) { + this.$refs.buildTrace.scrollTo(0, this.$refs.buildTrace.scrollHeight); + } + }, + scrollUp() { + if (this.$refs.buildTrace) { + this.$refs.buildTrace.scrollTo(0, 0); + } + }, + scrollBuildLog: _.throttle(function buildLogScrollDebounce() { + const { scrollTop } = this.$refs.buildTrace; + const { offsetHeight, scrollHeight } = this.$refs.buildTrace; + + if (scrollTop + offsetHeight === scrollHeight) { + this.scrollPos = scrollPositions.bottom; + } else if (scrollTop === 0) { + this.scrollPos = scrollPositions.top; + } else { + this.scrollPos = ''; + } + }), + getTrace() { + return this.fetchJobTrace().then(() => this.scrollDown()); + }, + }, +}; +</script> + +<template> + <div class="ide-pipeline build-page d-flex flex-column flex-fill"> + <header class="ide-job-header d-flex align-items-center"> + <button + class="btn btn-default btn-sm d-flex" + @click="setDetailJob(null)" + > + <icon + name="chevron-left" + /> + {{ __('View jobs') }} + </button> + </header> + <div class="top-bar d-flex border-left-0"> + <job-description + :job="detailJob" + /> + <div class="controllers ml-auto"> + <a + v-tooltip + :title="__('Show complete raw log')" + data-placement="top" + data-container="body" + class="controllers-buttons" + :href="detailJob.rawPath" + target="_blank" + > + <i + aria-hidden="true" + class="fa fa-file-text-o" + ></i> + </a> + <scroll-button + direction="up" + :disabled="isScrolledToTop" + @click="scrollUp" + /> + <scroll-button + direction="down" + :disabled="isScrolledToBottom" + @click="scrollDown" + /> + </div> + </div> + <pre + class="build-trace mb-0 h-100" + ref="buildTrace" + @scroll="scrollBuildLog" + > + <code + class="bash" + v-html="jobOutput" + > + </code> + <div + v-show="detailJob.isLoading" + class="build-loader-animation" + > + </div> + </pre> + </div> +</template> diff --git a/app/assets/javascripts/ide/components/jobs/detail/description.vue b/app/assets/javascripts/ide/components/jobs/detail/description.vue new file mode 100644 index 00000000000..def6bac3157 --- /dev/null +++ b/app/assets/javascripts/ide/components/jobs/detail/description.vue @@ -0,0 +1,47 @@ +<script> +import Icon from '../../../../vue_shared/components/icon.vue'; +import CiIcon from '../../../../vue_shared/components/ci_icon.vue'; + +export default { + components: { + Icon, + CiIcon, + }, + props: { + job: { + type: Object, + required: true, + }, + }, + computed: { + jobId() { + return `#${this.job.id}`; + }, + }, +}; +</script> + +<template> + <div class="d-flex align-items-center"> + <ci-icon + class="d-flex" + :status="job.status" + :borderless="true" + :size="24" + /> + <span class="prepend-left-8"> + {{ job.name }} + <a + :href="job.path" + target="_blank" + class="ide-external-link" + > + {{ jobId }} + <icon + name="external-link" + :size="12" + /> + </a> + </span> + </div> +</template> diff --git a/app/assets/javascripts/ide/components/jobs/detail/scroll_button.vue b/app/assets/javascripts/ide/components/jobs/detail/scroll_button.vue new file mode 100644 index 00000000000..4e19e6e9c84 --- /dev/null +++ b/app/assets/javascripts/ide/components/jobs/detail/scroll_button.vue @@ -0,0 +1,66 @@ +<script> +import { __ } from '../../../../locale'; +import Icon from '../../../../vue_shared/components/icon.vue'; +import tooltip from '../../../../vue_shared/directives/tooltip'; + +const directions = { + up: 'up', + down: 'down', +}; + +export default { + directives: { + tooltip, + }, + components: { + Icon, + }, + props: { + direction: { + type: String, + required: true, + validator(value) { + return Object.keys(directions).includes(value); + }, + }, + disabled: { + type: Boolean, + required: true, + }, + }, + computed: { + tooltipTitle() { + return this.direction === directions.up ? __('Scroll to top') : __('Scroll to bottom'); + }, + iconName() { + return `scroll_${this.direction}`; + }, + }, + methods: { + clickedScroll() { + this.$emit('click'); + }, + }, +}; +</script> + +<template> + <div + v-tooltip + class="controllers-buttons" + data-container="body" + data-placement="top" + :title="tooltipTitle" + > + <button + class="btn-scroll btn-transparent btn-blank" + type="button" + :disabled="disabled" + @click="clickedScroll" + > + <icon + :name="iconName" + /> + </button> + </div> +</template> diff --git a/app/assets/javascripts/ide/components/jobs/item.vue b/app/assets/javascripts/ide/components/jobs/item.vue index c33936021d4..c8e621504f0 100644 --- a/app/assets/javascripts/ide/components/jobs/item.vue +++ b/app/assets/javascripts/ide/components/jobs/item.vue @@ -1,11 +1,9 @@ <script> -import Icon from '../../../vue_shared/components/icon.vue'; -import CiIcon from '../../../vue_shared/components/ci_icon.vue'; +import JobDescription from './detail/description.vue'; export default { components: { - Icon, - CiIcon, + JobDescription, }, props: { job: { @@ -18,29 +16,29 @@ export default { return `#${this.job.id}`; }, }, + methods: { + clickViewLog() { + this.$emit('clickViewLog', this.job); + }, + }, }; </script> <template> <div class="ide-job-item"> - <ci-icon - :status="job.status" - :borderless="true" - :size="24" + <job-description + class="append-right-default" + :job="job" /> - <span class="prepend-left-8"> - {{ job.name }} - <a - :href="job.path" - target="_blank" - class="ide-external-link" + <div class="ml-auto align-self-center"> + <button + v-if="job.started" + type="button" + class="btn btn-default btn-sm" + @click="clickViewLog" > - {{ jobId }} - <icon - name="external-link" - :size="12" - /> - </a> - </span> + {{ __('View log') }} + </button> + </div> </div> </template> diff --git a/app/assets/javascripts/ide/components/jobs/list.vue b/app/assets/javascripts/ide/components/jobs/list.vue index bdd0364c9b9..3b16b860ecd 100644 --- a/app/assets/javascripts/ide/components/jobs/list.vue +++ b/app/assets/javascripts/ide/components/jobs/list.vue @@ -19,7 +19,7 @@ export default { }, }, methods: { - ...mapActions('pipelines', ['fetchJobs', 'toggleStageCollapsed']), + ...mapActions('pipelines', ['fetchJobs', 'toggleStageCollapsed', 'setDetailJob']), }, }; </script> @@ -38,6 +38,7 @@ export default { :stage="stage" @fetch="fetchJobs" @toggleCollapsed="toggleStageCollapsed" + @clickViewLog="setDetailJob" /> </template> </div> diff --git a/app/assets/javascripts/ide/components/jobs/stage.vue b/app/assets/javascripts/ide/components/jobs/stage.vue index 5b24bb1f5a7..b1428f885fb 100644 --- a/app/assets/javascripts/ide/components/jobs/stage.vue +++ b/app/assets/javascripts/ide/components/jobs/stage.vue @@ -48,6 +48,9 @@ export default { toggleCollapsed() { this.$emit('toggleCollapsed', this.stage.id); }, + clickViewLog(job) { + this.$emit('clickViewLog', job); + }, }, }; </script> @@ -101,6 +104,7 @@ export default { v-for="job in stage.jobs" :key="job.id" :job="job" + @clickViewLog="clickViewLog" /> </template> </div> diff --git a/app/assets/javascripts/ide/components/merge_requests/dropdown.vue b/app/assets/javascripts/ide/components/merge_requests/dropdown.vue new file mode 100644 index 00000000000..8cc8345db2e --- /dev/null +++ b/app/assets/javascripts/ide/components/merge_requests/dropdown.vue @@ -0,0 +1,63 @@ +<script> +import { mapGetters } from 'vuex'; +import Tabs from '../../../vue_shared/components/tabs/tabs'; +import Tab from '../../../vue_shared/components/tabs/tab.vue'; +import List from './list.vue'; + +export default { + components: { + Tabs, + Tab, + List, + }, + props: { + show: { + type: Boolean, + required: true, + }, + }, + computed: { + ...mapGetters('mergeRequests', ['assignedData', 'createdData']), + createdMergeRequestLength() { + return this.createdData.mergeRequests.length; + }, + assignedMergeRequestLength() { + return this.assignedData.mergeRequests.length; + }, + }, +}; +</script> + +<template> + <div class="dropdown-menu ide-merge-requests-dropdown p-0"> + <tabs + v-if="show" + stop-propagation + > + <tab active> + <template slot="title"> + {{ __('Created by me') }} + <span class="badge badge-pill"> + {{ createdMergeRequestLength }} + </span> + </template> + <list + type="created" + :empty-text="__('You have not created any merge requests')" + /> + </tab> + <tab> + <template slot="title"> + {{ __('Assigned to me') }} + <span class="badge badge-pill"> + {{ assignedMergeRequestLength }} + </span> + </template> + <list + type="assigned" + :empty-text="__('You do not have any assigned merge requests')" + /> + </tab> + </tabs> + </div> +</template> diff --git a/app/assets/javascripts/ide/components/merge_requests/item.vue b/app/assets/javascripts/ide/components/merge_requests/item.vue new file mode 100644 index 00000000000..b50fc8a3dbb --- /dev/null +++ b/app/assets/javascripts/ide/components/merge_requests/item.vue @@ -0,0 +1,63 @@ +<script> +import Icon from '../../../vue_shared/components/icon.vue'; + +export default { + components: { + Icon, + }, + props: { + item: { + type: Object, + required: true, + }, + currentId: { + type: String, + required: true, + }, + currentProjectId: { + type: String, + required: true, + }, + }, + computed: { + isActive() { + return ( + this.item.iid === parseInt(this.currentId, 10) && + this.currentProjectId === this.item.projectPathWithNamespace + ); + }, + pathWithID() { + return `${this.item.projectPathWithNamespace}!${this.item.iid}`; + }, + }, + methods: { + clickItem() { + this.$emit('click', this.item); + }, + }, +}; +</script> + +<template> + <button + type="button" + class="btn-link d-flex align-items-center" + @click="clickItem" + > + <span class="d-flex append-right-default ide-merge-request-current-icon"> + <icon + v-if="isActive" + name="mobile-issue-close" + :size="18" + /> + </span> + <span> + <strong> + {{ item.title }} + </strong> + <span class="ide-merge-request-project-path d-block mt-1"> + {{ pathWithID }} + </span> + </span> + </button> +</template> diff --git a/app/assets/javascripts/ide/components/merge_requests/list.vue b/app/assets/javascripts/ide/components/merge_requests/list.vue new file mode 100644 index 00000000000..5896e3a147d --- /dev/null +++ b/app/assets/javascripts/ide/components/merge_requests/list.vue @@ -0,0 +1,132 @@ +<script> +import { mapActions, mapGetters, mapState } from 'vuex'; +import _ from 'underscore'; +import LoadingIcon from '../../../vue_shared/components/loading_icon.vue'; +import Item from './item.vue'; + +export default { + components: { + LoadingIcon, + Item, + }, + props: { + type: { + type: String, + required: true, + }, + emptyText: { + type: String, + required: true, + }, + }, + data() { + return { + search: '', + }; + }, + computed: { + ...mapGetters('mergeRequests', ['getData']), + ...mapState(['currentMergeRequestId', 'currentProjectId']), + data() { + return this.getData(this.type); + }, + isLoading() { + return this.data.isLoading; + }, + mergeRequests() { + return this.data.mergeRequests; + }, + hasMergeRequests() { + return this.mergeRequests.length !== 0; + }, + hasNoSearchResults() { + return this.search !== '' && !this.hasMergeRequests; + }, + }, + watch: { + isLoading: { + handler: 'focusSearch', + }, + }, + mounted() { + this.loadMergeRequests(); + }, + methods: { + ...mapActions('mergeRequests', ['fetchMergeRequests', 'openMergeRequest']), + loadMergeRequests() { + this.fetchMergeRequests({ type: this.type, search: this.search }); + }, + viewMergeRequest(item) { + this.openMergeRequest({ + projectPath: item.projectPathWithNamespace, + id: item.iid, + }); + }, + searchMergeRequests: _.debounce(function debounceSearch() { + this.loadMergeRequests(); + }, 250), + focusSearch() { + if (!this.isLoading) { + this.$nextTick(() => { + this.$refs.searchInput.focus(); + }); + } + }, + }, +}; +</script> + +<template> + <div> + <div class="dropdown-input mt-3 pb-3 mb-0 border-bottom"> + <input + type="search" + class="dropdown-input-field" + :placeholder="__('Search merge requests')" + v-model="search" + @input="searchMergeRequests" + ref="searchInput" + /> + <i + aria-hidden="true" + class="fa fa-search dropdown-input-search" + ></i> + </div> + <div class="dropdown-content ide-merge-requests-dropdown-content d-flex"> + <loading-icon + class="mt-3 mb-3 align-self-center ml-auto mr-auto" + v-if="isLoading" + size="2" + /> + <ul + v-else + class="mb-3 w-100" + > + <template v-if="hasMergeRequests"> + <li + v-for="item in mergeRequests" + :key="item.id" + > + <item + :item="item" + :current-id="currentMergeRequestId" + :current-project-id="currentProjectId" + @click="viewMergeRequest" + /> + </li> + </template> + <li + v-else + class="ide-merge-requests-empty d-flex align-items-center justify-content-center" + > + <template v-if="hasNoSearchResults"> + {{ __('No merge requests found') }} + </template> + <template v-else> + {{ emptyText }} + </template> + </li> + </ul> + </div> + </div> +</template> diff --git a/app/assets/javascripts/ide/components/panes/right.vue b/app/assets/javascripts/ide/components/panes/right.vue index 703c4a70cfa..aafd6a15a78 100644 --- a/app/assets/javascripts/ide/components/panes/right.vue +++ b/app/assets/javascripts/ide/components/panes/right.vue @@ -4,6 +4,7 @@ import tooltip from '../../../vue_shared/directives/tooltip'; import Icon from '../../../vue_shared/components/icon.vue'; import { rightSidebarViews } from '../../constants'; import PipelinesList from '../pipelines/list.vue'; +import JobsDetail from '../jobs/detail.vue'; export default { directives: { @@ -12,9 +13,16 @@ export default { components: { Icon, PipelinesList, + JobsDetail, }, computed: { ...mapState(['rightPane']), + pipelinesActive() { + return ( + this.rightPane === rightSidebarViews.pipelines || + this.rightPane === rightSidebarViews.jobsDetail + ); + }, }, methods: { ...mapActions(['setRightPane']), @@ -48,7 +56,7 @@ export default { :title="__('Pipelines')" class="ide-sidebar-link is-right" :class="{ - active: rightPane === $options.rightSidebarViews.pipelines + active: pipelinesActive }" type="button" @click="clickTab($event, $options.rightSidebarViews.pipelines)" diff --git a/app/assets/javascripts/ide/constants.js b/app/assets/javascripts/ide/constants.js index 33cd20caf52..65886c02b92 100644 --- a/app/assets/javascripts/ide/constants.js +++ b/app/assets/javascripts/ide/constants.js @@ -23,4 +23,5 @@ export const viewerTypes = { export const rightSidebarViews = { pipelines: 'pipelines-list', + jobsDetail: 'jobs-detail', }; diff --git a/app/assets/javascripts/ide/stores/actions/merge_request.js b/app/assets/javascripts/ide/stores/actions/merge_request.js index 5ec9bd661bb..edb20ff96fc 100644 --- a/app/assets/javascripts/ide/stores/actions/merge_request.js +++ b/app/assets/javascripts/ide/stores/actions/merge_request.js @@ -17,9 +17,7 @@ export const getMergeRequestData = ( mergeRequestId, mergeRequest: data, }); - if (!state.currentMergeRequestId) { - commit(types.SET_CURRENT_MERGE_REQUEST, mergeRequestId); - } + commit(types.SET_CURRENT_MERGE_REQUEST, mergeRequestId); resolve(data); }) .catch(() => { diff --git a/app/assets/javascripts/ide/stores/actions/project.js b/app/assets/javascripts/ide/stores/actions/project.js index 46af47d2f81..0b99bce4a8e 100644 --- a/app/assets/javascripts/ide/stores/actions/project.js +++ b/app/assets/javascripts/ide/stores/actions/project.js @@ -13,8 +13,7 @@ export const getProjectData = ({ commit, state }, { namespace, projectId, force .then(data => { commit(types.TOGGLE_LOADING, { entry: state }); commit(types.SET_PROJECT, { projectPath: `${namespace}/${projectId}`, project: data }); - if (!state.currentProjectId) - commit(types.SET_CURRENT_PROJECT, `${namespace}/${projectId}`); + commit(types.SET_CURRENT_PROJECT, `${namespace}/${projectId}`); resolve(data); }) .catch(() => { diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/actions.js b/app/assets/javascripts/ide/stores/modules/merge_requests/actions.js index d3050183bd3..5beb8fac71f 100644 --- a/app/assets/javascripts/ide/stores/modules/merge_requests/actions.js +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/actions.js @@ -1,25 +1,42 @@ import { __ } from '../../../../locale'; import Api from '../../../../api'; import flash from '../../../../flash'; +import router from '../../../ide_router'; +import { scopes } from './constants'; import * as types from './mutation_types'; +import * as rootTypes from '../../mutation_types'; -export const requestMergeRequests = ({ commit }) => commit(types.REQUEST_MERGE_REQUESTS); -export const receiveMergeRequestsError = ({ commit }) => { +export const requestMergeRequests = ({ commit }, type) => + commit(types.REQUEST_MERGE_REQUESTS, type); +export const receiveMergeRequestsError = ({ commit }, type) => { flash(__('Error loading merge requests.')); - commit(types.RECEIVE_MERGE_REQUESTS_ERROR); + commit(types.RECEIVE_MERGE_REQUESTS_ERROR, type); }; -export const receiveMergeRequestsSuccess = ({ commit }, data) => - commit(types.RECEIVE_MERGE_REQUESTS_SUCCESS, data); +export const receiveMergeRequestsSuccess = ({ commit }, { type, data }) => + commit(types.RECEIVE_MERGE_REQUESTS_SUCCESS, { type, data }); -export const fetchMergeRequests = ({ dispatch, state: { scope, state } }, search = '') => { - dispatch('requestMergeRequests'); - dispatch('resetMergeRequests'); +export const fetchMergeRequests = ({ dispatch, state: { state } }, { type, search = '' }) => { + const scope = scopes[type]; + dispatch('requestMergeRequests', type); + dispatch('resetMergeRequests', type); Api.mergeRequests({ scope, state, search }) - .then(({ data }) => dispatch('receiveMergeRequestsSuccess', data)) - .catch(() => dispatch('receiveMergeRequestsError')); + .then(({ data }) => dispatch('receiveMergeRequestsSuccess', { type, data })) + .catch(() => dispatch('receiveMergeRequestsError', type)); }; -export const resetMergeRequests = ({ commit }) => commit(types.RESET_MERGE_REQUESTS); +export const resetMergeRequests = ({ commit }, type) => commit(types.RESET_MERGE_REQUESTS, type); + +export const openMergeRequest = ({ commit, dispatch }, { projectPath, id }) => { + commit(rootTypes.CLEAR_PROJECTS, null, { root: true }); + commit(rootTypes.SET_CURRENT_MERGE_REQUEST, `${id}`, { root: true }); + commit(rootTypes.RESET_OPEN_FILES, null, { root: true }); + dispatch('pipelines/stopPipelinePolling', null, { root: true }); + dispatch('pipelines/clearEtagPoll', null, { root: true }); + dispatch('pipelines/resetLatestPipeline', null, { root: true }); + dispatch('setCurrentBranchId', '', { root: true }); + + router.push(`/project/${projectPath}/merge_requests/${id}`); +}; export default () => {}; diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/constants.js b/app/assets/javascripts/ide/stores/modules/merge_requests/constants.js index 64b7763f257..a7085c7d04c 100644 --- a/app/assets/javascripts/ide/stores/modules/merge_requests/constants.js +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/constants.js @@ -1,6 +1,6 @@ export const scopes = { - assignedToMe: 'assigned-to-me', - createdByMe: 'created-by-me', + assigned: 'assigned-to-me', + created: 'created-by-me', }; export const states = { diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/getters.js b/app/assets/javascripts/ide/stores/modules/merge_requests/getters.js new file mode 100644 index 00000000000..8e2b234be8d --- /dev/null +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/getters.js @@ -0,0 +1,4 @@ +export const getData = state => type => state[type]; + +export const assignedData = state => state.assigned; +export const createdData = state => state.created; diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/index.js b/app/assets/javascripts/ide/stores/modules/merge_requests/index.js index 04e7e0f08f1..2e6dfb420f4 100644 --- a/app/assets/javascripts/ide/stores/modules/merge_requests/index.js +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/index.js @@ -1,5 +1,6 @@ import state from './state'; import * as actions from './actions'; +import * as getters from './getters'; import mutations from './mutations'; export default { @@ -7,4 +8,5 @@ export default { state: state(), actions, mutations, + getters, }; diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/mutations.js b/app/assets/javascripts/ide/stores/modules/merge_requests/mutations.js index 98102a68e08..971da0806bd 100644 --- a/app/assets/javascripts/ide/stores/modules/merge_requests/mutations.js +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/mutations.js @@ -2,15 +2,15 @@ import * as types from './mutation_types'; export default { - [types.REQUEST_MERGE_REQUESTS](state) { - state.isLoading = true; + [types.REQUEST_MERGE_REQUESTS](state, type) { + state[type].isLoading = true; }, - [types.RECEIVE_MERGE_REQUESTS_ERROR](state) { - state.isLoading = false; + [types.RECEIVE_MERGE_REQUESTS_ERROR](state, type) { + state[type].isLoading = false; }, - [types.RECEIVE_MERGE_REQUESTS_SUCCESS](state, data) { - state.isLoading = false; - state.mergeRequests = data.map(mergeRequest => ({ + [types.RECEIVE_MERGE_REQUESTS_SUCCESS](state, { type, data }) { + state[type].isLoading = false; + state[type].mergeRequests = data.map(mergeRequest => ({ id: mergeRequest.id, iid: mergeRequest.iid, title: mergeRequest.title, @@ -20,7 +20,7 @@ export default { .replace(`/merge_requests/${mergeRequest.iid}`, ''), })); }, - [types.RESET_MERGE_REQUESTS](state) { - state.mergeRequests = []; + [types.RESET_MERGE_REQUESTS](state, type) { + state[type].mergeRequests = []; }, }; diff --git a/app/assets/javascripts/ide/stores/modules/merge_requests/state.js b/app/assets/javascripts/ide/stores/modules/merge_requests/state.js index 2947b686c1c..57eb6b04283 100644 --- a/app/assets/javascripts/ide/stores/modules/merge_requests/state.js +++ b/app/assets/javascripts/ide/stores/modules/merge_requests/state.js @@ -1,8 +1,13 @@ -import { scopes, states } from './constants'; +import { states } from './constants'; export default () => ({ - isLoading: false, - mergeRequests: [], - scope: scopes.assignedToMe, + created: { + isLoading: false, + mergeRequests: [], + }, + assigned: { + isLoading: false, + mergeRequests: [], + }, state: states.opened, }); diff --git a/app/assets/javascripts/ide/stores/modules/pipelines/actions.js b/app/assets/javascripts/ide/stores/modules/pipelines/actions.js index 1ebe487263b..0a4ea80c4c1 100644 --- a/app/assets/javascripts/ide/stores/modules/pipelines/actions.js +++ b/app/assets/javascripts/ide/stores/modules/pipelines/actions.js @@ -4,6 +4,7 @@ import { __ } from '../../../../locale'; import flash from '../../../../flash'; import Poll from '../../../../lib/utils/poll'; import service from '../../../services'; +import { rightSidebarViews } from '../../../constants'; import * as types from './mutation_types'; let eTagPoll; @@ -77,4 +78,31 @@ export const fetchJobs = ({ dispatch }, stage) => { export const toggleStageCollapsed = ({ commit }, stageId) => commit(types.TOGGLE_STAGE_COLLAPSE, stageId); +export const setDetailJob = ({ commit, dispatch }, job) => { + commit(types.SET_DETAIL_JOB, job); + dispatch('setRightPane', job ? rightSidebarViews.jobsDetail : rightSidebarViews.pipelines, { + root: true, + }); +}; + +export const requestJobTrace = ({ commit }) => commit(types.REQUEST_JOB_TRACE); +export const receiveJobTraceError = ({ commit }) => { + flash(__('Error fetching job trace')); + commit(types.RECEIVE_JOB_TRACE_ERROR); +}; +export const receiveJobTraceSuccess = ({ commit }, data) => + commit(types.RECEIVE_JOB_TRACE_SUCCESS, data); + +export const fetchJobTrace = ({ dispatch, state }) => { + dispatch('requestJobTrace'); + + return axios + .get(`${state.detailJob.path}/trace`, { params: { format: 'json' } }) + .then(({ data }) => dispatch('receiveJobTraceSuccess', data)) + .catch(() => dispatch('receiveJobTraceError')); +}; + +export const resetLatestPipeline = ({ commit }) => + commit(types.RECEIVE_LASTEST_PIPELINE_SUCCESS, null); + export default () => {}; diff --git a/app/assets/javascripts/ide/stores/modules/pipelines/mutation_types.js b/app/assets/javascripts/ide/stores/modules/pipelines/mutation_types.js index 3ddc8409c5b..f4c36b9d96f 100644 --- a/app/assets/javascripts/ide/stores/modules/pipelines/mutation_types.js +++ b/app/assets/javascripts/ide/stores/modules/pipelines/mutation_types.js @@ -7,3 +7,9 @@ export const RECEIVE_JOBS_ERROR = 'RECEIVE_JOBS_ERROR'; export const RECEIVE_JOBS_SUCCESS = 'RECEIVE_JOBS_SUCCESS'; export const TOGGLE_STAGE_COLLAPSE = 'TOGGLE_STAGE_COLLAPSE'; + +export const SET_DETAIL_JOB = 'SET_DETAIL_JOB'; + +export const REQUEST_JOB_TRACE = 'REQUEST_JOB_TRACE'; +export const RECEIVE_JOB_TRACE_ERROR = 'RECEIVE_JOB_TRACE_ERROR'; +export const RECEIVE_JOB_TRACE_SUCCESS = 'RECEIVE_JOB_TRACE_SUCCESS'; diff --git a/app/assets/javascripts/ide/stores/modules/pipelines/mutations.js b/app/assets/javascripts/ide/stores/modules/pipelines/mutations.js index 745797e1ee5..5a2213bbe89 100644 --- a/app/assets/javascripts/ide/stores/modules/pipelines/mutations.js +++ b/app/assets/javascripts/ide/stores/modules/pipelines/mutations.js @@ -63,4 +63,17 @@ export default { isCollapsed: stage.id === id ? !stage.isCollapsed : stage.isCollapsed, })); }, + [types.SET_DETAIL_JOB](state, job) { + state.detailJob = { ...job }; + }, + [types.REQUEST_JOB_TRACE](state) { + state.detailJob.isLoading = true; + }, + [types.RECEIVE_JOB_TRACE_ERROR](state) { + state.detailJob.isLoading = false; + }, + [types.RECEIVE_JOB_TRACE_SUCCESS](state, data) { + state.detailJob.isLoading = false; + state.detailJob.output = data.html; + }, }; diff --git a/app/assets/javascripts/ide/stores/modules/pipelines/state.js b/app/assets/javascripts/ide/stores/modules/pipelines/state.js index 0f83b315fff..8651e267b53 100644 --- a/app/assets/javascripts/ide/stores/modules/pipelines/state.js +++ b/app/assets/javascripts/ide/stores/modules/pipelines/state.js @@ -3,4 +3,5 @@ export default () => ({ isLoadingJobs: false, latestPipeline: null, stages: [], + detailJob: null, }); diff --git a/app/assets/javascripts/ide/stores/modules/pipelines/utils.js b/app/assets/javascripts/ide/stores/modules/pipelines/utils.js index 9f4b0d7d726..a6caca2d2dc 100644 --- a/app/assets/javascripts/ide/stores/modules/pipelines/utils.js +++ b/app/assets/javascripts/ide/stores/modules/pipelines/utils.js @@ -4,4 +4,8 @@ export const normalizeJob = job => ({ name: job.name, status: job.status, path: job.build_path, + rawPath: `${job.build_path}/raw`, + started: job.started, + output: '', + isLoading: false, }); diff --git a/app/assets/javascripts/ide/stores/mutation_types.js b/app/assets/javascripts/ide/stores/mutation_types.js index fbfb92105d6..99b315ac4db 100644 --- a/app/assets/javascripts/ide/stores/mutation_types.js +++ b/app/assets/javascripts/ide/stores/mutation_types.js @@ -68,3 +68,6 @@ export const TOGGLE_FILE_FINDER = 'TOGGLE_FILE_FINDER'; export const BURST_UNUSED_SEAL = 'BURST_UNUSED_SEAL'; export const SET_RIGHT_PANE = 'SET_RIGHT_PANE'; + +export const CLEAR_PROJECTS = 'CLEAR_PROJECTS'; +export const RESET_OPEN_FILES = 'RESET_OPEN_FILES'; diff --git a/app/assets/javascripts/ide/stores/mutations.js b/app/assets/javascripts/ide/stores/mutations.js index eeaa7cb0ec3..48f1da4eccf 100644 --- a/app/assets/javascripts/ide/stores/mutations.js +++ b/app/assets/javascripts/ide/stores/mutations.js @@ -157,6 +157,12 @@ export default { [types.SET_LINKS](state, links) { Object.assign(state, { links }); }, + [types.CLEAR_PROJECTS](state) { + Object.assign(state, { projects: {}, trees: {} }); + }, + [types.RESET_OPEN_FILES](state) { + Object.assign(state, { openFiles: [] }); + }, ...projectMutations, ...mergeRequestMutation, ...fileMutations, diff --git a/app/assets/javascripts/importer_status.js b/app/assets/javascripts/importer_status.js index 52455885248..f9ff0722c01 100644 --- a/app/assets/javascripts/importer_status.js +++ b/app/assets/javascripts/importer_status.js @@ -67,7 +67,15 @@ class ImporterStatus { false, )); }) - .catch(() => flash(__('An error occurred while importing project'))); + .catch((error) => { + let details = error; + + if (error.response && error.response.data && error.response.data.errors) { + details = error.response.data.errors; + } + + flash(__(`An error occurred while importing project: ${details}`)); + }); } autoUpdate() { diff --git a/app/assets/javascripts/init_changes_dropdown.js b/app/assets/javascripts/init_changes_dropdown.js index 09cca1dc7d9..5c5a6e01848 100644 --- a/app/assets/javascripts/init_changes_dropdown.js +++ b/app/assets/javascripts/init_changes_dropdown.js @@ -1,5 +1,5 @@ import $ from 'jquery'; -import stickyMonitor from './lib/utils/sticky'; +import { stickyMonitor } from './lib/utils/sticky'; export default (stickyTop) => { stickyMonitor(document.querySelector('.js-diff-files-changed'), stickyTop); diff --git a/app/assets/javascripts/job.js b/app/assets/javascripts/job.js index 611e8200b4d..fc13f467675 100644 --- a/app/assets/javascripts/job.js +++ b/app/assets/javascripts/job.js @@ -1,14 +1,17 @@ import $ from 'jquery'; import _ from 'underscore'; -import StickyFill from 'stickyfilljs'; +import { polyfillSticky } from './lib/utils/sticky'; import axios from './lib/utils/axios_utils'; import { visitUrl } from './lib/utils/url_utility'; import bp from './breakpoints'; import { numberToHumanSize } from './lib/utils/number_utils'; import { setCiStatusFavicon } from './lib/utils/common_utils'; +import { isScrolledToBottom, scrollDown } from './lib/utils/scroll_utils'; +import LogOutputBehaviours from './lib/utils/logoutput_behaviours'; -export default class Job { +export default class Job extends LogOutputBehaviours { constructor(options) { + super(); this.timeout = null; this.state = null; this.fetchingStatusFavicon = false; @@ -29,10 +32,6 @@ export default class Job { this.$buildTraceOutput = $('.js-build-output'); this.$topBar = $('.js-top-bar'); - // Scroll controllers - this.$scrollTopBtn = $('.js-scroll-up'); - this.$scrollBottomBtn = $('.js-scroll-down'); - clearTimeout(this.timeout); this.initSidebar(); @@ -48,23 +47,14 @@ export default class Job { .off('click', '.stage-item') .on('click', '.stage-item', this.updateDropdown); - // add event listeners to the scroll buttons - this.$scrollTopBtn - .off('click') - .on('click', this.scrollToTop.bind(this)); - - this.$scrollBottomBtn - .off('click') - .on('click', this.scrollToBottom.bind(this)); - this.scrollThrottled = _.throttle(this.toggleScroll.bind(this), 100); this.$window .off('scroll') .on('scroll', () => { - if (!this.isScrolledToBottom()) { + if (!isScrolledToBottom()) { this.toggleScrollAnimation(false); - } else if (this.isScrolledToBottom() && !this.isLogComplete) { + } else if (isScrolledToBottom() && !this.isLogComplete) { this.toggleScrollAnimation(true); } this.scrollThrottled(); @@ -80,70 +70,11 @@ export default class Job { } initAffixTopArea() { - /** - If the browser does not support position sticky, it returns the position as static. - If the browser does support sticky, then we allow the browser to handle it, if not - then we use a polyfill - */ - if (this.$topBar.css('position') !== 'static') return; - - StickyFill.add(this.$topBar); - } - - // eslint-disable-next-line class-methods-use-this - canScroll() { - return $(document).height() > $(window).height(); - } - - toggleScroll() { - const $document = $(document); - const currentPosition = $document.scrollTop(); - const scrollHeight = $document.height(); - - const windowHeight = $(window).height(); - if (this.canScroll()) { - if (currentPosition > 0 && - (scrollHeight - currentPosition !== windowHeight)) { - // User is in the middle of the log - - this.toggleDisableButton(this.$scrollTopBtn, false); - this.toggleDisableButton(this.$scrollBottomBtn, false); - } else if (currentPosition === 0) { - // User is at Top of Log - - this.toggleDisableButton(this.$scrollTopBtn, true); - this.toggleDisableButton(this.$scrollBottomBtn, false); - } else if (this.isScrolledToBottom()) { - // User is at the bottom of the build log. - - this.toggleDisableButton(this.$scrollTopBtn, false); - this.toggleDisableButton(this.$scrollBottomBtn, true); - } - } else { - this.toggleDisableButton(this.$scrollTopBtn, true); - this.toggleDisableButton(this.$scrollBottomBtn, true); - } - } - // eslint-disable-next-line class-methods-use-this - isScrolledToBottom() { - const $document = $(document); - - const currentPosition = $document.scrollTop(); - const scrollHeight = $document.height(); - - const windowHeight = $(window).height(); - - return scrollHeight - currentPosition === windowHeight; - } - - // eslint-disable-next-line class-methods-use-this - scrollDown() { - const $document = $(document); - $document.scrollTop($document.height()); + polyfillSticky(this.$topBar); } scrollToBottom() { - this.scrollDown(); + scrollDown(); this.hasBeenScrolled = true; this.toggleScroll(); } @@ -154,12 +85,6 @@ export default class Job { this.toggleScroll(); } - // eslint-disable-next-line class-methods-use-this - toggleDisableButton($button, disable) { - if (disable && $button.prop('disabled')) return; - $button.prop('disabled', disable); - } - toggleScrollAnimation(toggle) { this.$scrollBottomBtn.toggleClass('animate', toggle); } @@ -191,7 +116,7 @@ export default class Job { this.state = log.state; } - this.isScrollInBottom = this.isScrolledToBottom(); + this.isScrollInBottom = isScrolledToBottom(); if (log.append) { this.$buildTraceOutput.append(log.html); @@ -231,7 +156,7 @@ export default class Job { }) .then(() => { if (this.isScrollInBottom) { - this.scrollDown(); + scrollDown(); } }) .then(() => this.toggleScroll()); diff --git a/app/assets/javascripts/jobs/components/header.vue b/app/assets/javascripts/jobs/components/header.vue index c1044f4cd42..5704d753277 100644 --- a/app/assets/javascripts/jobs/components/header.vue +++ b/app/assets/javascripts/jobs/components/header.vue @@ -42,6 +42,9 @@ export default { jobStarted() { return !this.job.started === false; }, + headerTime() { + return this.jobStarted ? this.job.started : this.job.created_at; + }, }, watch: { job() { @@ -73,7 +76,7 @@ export default { :status="status" item-name="Job" :item-id="job.id" - :time="job.created_at" + :time="headerTime" :user="job.user" :actions="actions" :has-sidebar-button="true" diff --git a/app/assets/javascripts/lib/utils/logoutput_behaviours.js b/app/assets/javascripts/lib/utils/logoutput_behaviours.js new file mode 100644 index 00000000000..1bf99d935ef --- /dev/null +++ b/app/assets/javascripts/lib/utils/logoutput_behaviours.js @@ -0,0 +1,46 @@ +import $ from 'jquery'; +import { canScroll, isScrolledToBottom, toggleDisableButton } from './scroll_utils'; + +export default class LogOutputBehaviours { + constructor() { + // Scroll buttons + this.$scrollTopBtn = $('.js-scroll-up'); + this.$scrollBottomBtn = $('.js-scroll-down'); + + this.$scrollTopBtn.off('click').on('click', this.scrollToTop.bind(this)); + this.$scrollBottomBtn.off('click').on('click', this.scrollToBottom.bind(this)); + } + + toggleScroll() { + const $document = $(document); + const currentPosition = $document.scrollTop(); + const scrollHeight = $document.height(); + + const windowHeight = $(window).height(); + if (canScroll()) { + if (currentPosition > 0 && scrollHeight - currentPosition !== windowHeight) { + // User is in the middle of the log + + toggleDisableButton(this.$scrollTopBtn, false); + toggleDisableButton(this.$scrollBottomBtn, false); + } else if (currentPosition === 0) { + // User is at Top of Log + + toggleDisableButton(this.$scrollTopBtn, true); + toggleDisableButton(this.$scrollBottomBtn, false); + } else if (isScrolledToBottom()) { + // User is at the bottom of the build log. + + toggleDisableButton(this.$scrollTopBtn, false); + toggleDisableButton(this.$scrollBottomBtn, true); + } + } else { + toggleDisableButton(this.$scrollTopBtn, true); + toggleDisableButton(this.$scrollBottomBtn, true); + } + } + + toggleScrollAnimation(toggle) { + this.$scrollBottomBtn.toggleClass('animate', toggle); + } +} diff --git a/app/assets/javascripts/lib/utils/scroll_utils.js b/app/assets/javascripts/lib/utils/scroll_utils.js new file mode 100644 index 00000000000..9313b570863 --- /dev/null +++ b/app/assets/javascripts/lib/utils/scroll_utils.js @@ -0,0 +1,29 @@ +import $ from 'jquery'; + +export const canScroll = () => $(document).height() > $(window).height(); + +/** + * Checks if the entire page is scrolled down all the way to the bottom + */ +export const isScrolledToBottom = () => { + const $document = $(document); + + const currentPosition = $document.scrollTop(); + const scrollHeight = $document.height(); + + const windowHeight = $(window).height(); + + return scrollHeight - currentPosition === windowHeight; +}; + +export const scrollDown = () => { + const $document = $(document); + $document.scrollTop($document.height()); +}; + +export const toggleDisableButton = ($button, disable) => { + if (disable && $button.prop('disabled')) return; + $button.prop('disabled', disable); +}; + +export default {}; diff --git a/app/assets/javascripts/lib/utils/sticky.js b/app/assets/javascripts/lib/utils/sticky.js index 098afcfa1b4..15a4dd62012 100644 --- a/app/assets/javascripts/lib/utils/sticky.js +++ b/app/assets/javascripts/lib/utils/sticky.js @@ -1,3 +1,5 @@ +import StickyFill from 'stickyfilljs'; + export const createPlaceholder = () => { const placeholder = document.createElement('div'); placeholder.classList.add('sticky-placeholder'); @@ -28,7 +30,16 @@ export const isSticky = (el, scrollY, stickyTop, insertPlaceholder) => { } }; -export default (el, stickyTop, insertPlaceholder = true) => { +/** + * Create a listener that will toggle a 'is-stuck' class, based on the current scroll position. + * + * - If the current environment does not support `position: sticky`, do nothing. + * + * @param {HTMLElement} el The `position: sticky` element. + * @param {Number} stickyTop Used to determine when an element is stuck. + * @param {Boolean} insertPlaceholder Should a placeholder element be created when element is stuck? + */ +export const stickyMonitor = (el, stickyTop, insertPlaceholder = true) => { if (!el) return; if (typeof CSS === 'undefined' || !(CSS.supports('(position: -webkit-sticky) or (position: sticky)'))) return; @@ -37,3 +48,13 @@ export default (el, stickyTop, insertPlaceholder = true) => { passive: true, }); }; + +/** + * Polyfill the `position: sticky` behavior. + * + * - If the current environment supports `position: sticky`, do nothing. + * - Can receive an iterable element list (NodeList, jQuery collection, etc.) or single HTMLElement. + */ +export const polyfillSticky = (el) => { + StickyFill.add(el); +}; diff --git a/app/assets/javascripts/monitoring/components/dashboard.vue b/app/assets/javascripts/monitoring/components/dashboard.vue index f5572be5fbf..21934021852 100644 --- a/app/assets/javascripts/monitoring/components/dashboard.vue +++ b/app/assets/javascripts/monitoring/components/dashboard.vue @@ -174,7 +174,10 @@ export default { :tags-path="tagsPath" :show-legend="showLegend" :small-graph="forceSmallGraph" - /> + > + <!-- EE content --> + {{ null }} + </graph> </graph-group> </div> <empty-state diff --git a/app/assets/javascripts/monitoring/components/graph.vue b/app/assets/javascripts/monitoring/components/graph.vue index de6755e0414..503ee1ce3d1 100644 --- a/app/assets/javascripts/monitoring/components/graph.vue +++ b/app/assets/javascripts/monitoring/components/graph.vue @@ -232,9 +232,14 @@ export default { @mouseover="showFlagContent = true" @mouseleave="showFlagContent = false" > - <h5 class="text-center graph-title"> - {{ graphData.title }} - </h5> + <div class="prometheus-graph-header"> + <h5 class="prometheus-graph-title"> + {{ graphData.title }} + </h5> + <div class="prometheus-graph-widgets"> + <slot></slot> + </div> + </div> <div class="prometheus-svg-container" :style="paddingBottomRootSvg" diff --git a/app/assets/javascripts/notes/constants.js b/app/assets/javascripts/notes/constants.js index c4de4826eda..5b5b1e89058 100644 --- a/app/assets/javascripts/notes/constants.js +++ b/app/assets/javascripts/notes/constants.js @@ -14,6 +14,7 @@ export const EPIC_NOTEABLE_TYPE = 'epic'; export const MERGE_REQUEST_NOTEABLE_TYPE = 'merge_request'; export const UNRESOLVE_NOTE_METHOD_NAME = 'delete'; export const RESOLVE_NOTE_METHOD_NAME = 'post'; +export const DESCRIPTION_TYPE = 'changed the description'; export const NOTEABLE_TYPE_MAPPING = { Issue: ISSUE_NOTEABLE_TYPE, diff --git a/app/assets/javascripts/notes/stores/collapse_utils.js b/app/assets/javascripts/notes/stores/collapse_utils.js new file mode 100644 index 00000000000..fa4a1c56b20 --- /dev/null +++ b/app/assets/javascripts/notes/stores/collapse_utils.js @@ -0,0 +1,108 @@ +import { n__, s__, sprintf } from '~/locale'; +import { DESCRIPTION_TYPE } from '../constants'; + +/** + * Changes the description from a note, returns 'changed the description n number of times' + */ +export const changeDescriptionNote = (note, descriptionChangedTimes, timeDifferenceMinutes) => { + const descriptionNote = Object.assign({}, note); + + descriptionNote.note_html = sprintf( + s__(`MergeRequest| + %{paragraphStart}changed the description %{descriptionChangedTimes} times %{timeDifferenceMinutes}%{paragraphEnd}`), + { + paragraphStart: '<p dir="auto">', + paragraphEnd: '</p>', + descriptionChangedTimes, + timeDifferenceMinutes: n__('within %d minute ', 'within %d minutes ', timeDifferenceMinutes), + }, + false, + ); + + descriptionNote.times_updated = descriptionChangedTimes; + + return descriptionNote; +}; + +/** + * Checks the time difference between two notes from their 'created_at' dates + * returns an integer + */ + +export const getTimeDifferenceMinutes = (noteBeggining, noteEnd) => { + const descriptionNoteBegin = new Date(noteBeggining.created_at); + const descriptionNoteEnd = new Date(noteEnd.created_at); + const timeDifferenceMinutes = (descriptionNoteEnd - descriptionNoteBegin) / 1000 / 60; + + return Math.ceil(timeDifferenceMinutes); +}; + +/** + * Checks if a note is a system note and if the content is description + * + * @param {Object} note + * @returns {Boolean} + */ +export const isDescriptionSystemNote = note => note.system && note.note === DESCRIPTION_TYPE; + +/** + * Collapses the system notes of a description type, e.g. Changed the description, n minutes ago + * the notes will collapse as long as they happen no more than 10 minutes away from each away + * in between the notes can be anything, another type of system note + * (such as 'changed the weight') or a comment. + * + * @param {Array} notes + * @returns {Array} + */ +export const collapseSystemNotes = notes => { + let lastDescriptionSystemNote = null; + let lastDescriptionSystemNoteIndex = -1; + let descriptionChangedTimes = 1; + + return notes.slice(0).reduce((acc, currentNote) => { + const note = currentNote.notes[0]; + + if (isDescriptionSystemNote(note)) { + // is it the first one? + if (!lastDescriptionSystemNote) { + lastDescriptionSystemNote = note; + lastDescriptionSystemNoteIndex = acc.length; + } else if (lastDescriptionSystemNote) { + const timeDifferenceMinutes = getTimeDifferenceMinutes( + lastDescriptionSystemNote, + note, + ); + + // are they less than 10 minutes appart? + if (timeDifferenceMinutes > 10) { + // reset counter + descriptionChangedTimes = 1; + // update the previous system note + lastDescriptionSystemNote = note; + lastDescriptionSystemNoteIndex = acc.length; + } else { + // increase counter + descriptionChangedTimes += 1; + + // delete the previous one + acc.splice(lastDescriptionSystemNoteIndex, 1); + + // replace the text of the current system note with the collapsed note. + currentNote.notes.splice( + 0, + 1, + changeDescriptionNote(note, descriptionChangedTimes, timeDifferenceMinutes), + ); + + // update the previous system note index + lastDescriptionSystemNoteIndex = acc.length; + } + } + } + acc.push(currentNote); + return acc; + }, []); +}; + +// for babel-rewire +export default {}; diff --git a/app/assets/javascripts/notes/stores/getters.js b/app/assets/javascripts/notes/stores/getters.js index 787be6f4c99..bc373e0d0fc 100644 --- a/app/assets/javascripts/notes/stores/getters.js +++ b/app/assets/javascripts/notes/stores/getters.js @@ -1,6 +1,8 @@ import _ from 'underscore'; +import { collapseSystemNotes } from './collapse_utils'; + +export const notes = state => collapseSystemNotes(state.notes); -export const notes = state => state.notes; export const targetNoteHash = state => state.targetNoteHash; export const getNotesData = state => state.notesData; diff --git a/app/assets/javascripts/performance_bar/components/detailed_metric.vue b/app/assets/javascripts/performance_bar/components/detailed_metric.vue index db8a0055acd..96189e7033a 100644 --- a/app/assets/javascripts/performance_bar/components/detailed_metric.vue +++ b/app/assets/javascripts/performance_bar/components/detailed_metric.vue @@ -56,6 +56,7 @@ export default { <gl-modal :id="`modal-peek-${metric}-details`" :header-title-text="header" + modal-size="lg" class="performance-bar-modal" > <table @@ -70,7 +71,7 @@ export default { <td v-for="key in keys" :key="key" - class="break-word" + class="break-word all-words" > {{ item[key] }} </td> diff --git a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_machine_type_dropdown.vue b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_machine_type_dropdown.vue index ab7d2d41ece..6ed35c0a981 100644 --- a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_machine_type_dropdown.vue +++ b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_machine_type_dropdown.vue @@ -89,14 +89,13 @@ export default { <div> <div class="js-gcp-machine-type-dropdown dropdown" - :class="{ 'gl-show-field-errors': hasErrors }" > <dropdown-hidden-input :name="fieldName" :value="selectedMachineType" /> <dropdown-button - :class="{ 'gl-field-error-outline': hasErrors }" + :class="{ 'border-danger': hasErrors }" :is-disabled="isDisabled" :is-loading="isLoading" :toggle-text="toggleText" @@ -132,8 +131,11 @@ export default { </div> </div> <span - class="form-text text-muted" - :class="{ 'gl-field-error': hasErrors }" + class="form-text" + :class="{ + 'text-danger': hasErrors, + 'text-muted': !hasErrors + }" v-if="hasErrors" > {{ errorMessage }} diff --git a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_project_id_dropdown.vue b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_project_id_dropdown.vue index 25350ef0fa9..542d4d21a22 100644 --- a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_project_id_dropdown.vue +++ b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_project_id_dropdown.vue @@ -147,7 +147,6 @@ export default { <div> <div class="js-gcp-project-id-dropdown dropdown" - :class="{ 'gl-show-field-errors': hasErrors }" > <dropdown-hidden-input :name="fieldName" @@ -155,7 +154,7 @@ export default { /> <dropdown-button :class="{ - 'gl-field-error-outline': hasErrors, + 'border-danger': hasErrors, 'read-only': hasOneProject }" :is-disabled="isDisabled" @@ -193,8 +192,11 @@ export default { </div> </div> <span - class="form-text text-muted" - :class="{ 'gl-field-error': hasErrors }" + class="form-text" + :class="{ + 'text-danger': hasErrors, + 'text-muted': !hasErrors + }" v-html="helpText" ></span> </div> diff --git a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_zone_dropdown.vue b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_zone_dropdown.vue index 8ee4eefcd91..bc28f8b5df4 100644 --- a/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_zone_dropdown.vue +++ b/app/assets/javascripts/projects/gke_cluster_dropdowns/components/gke_zone_dropdown.vue @@ -63,14 +63,13 @@ export default { <div> <div class="js-gcp-zone-dropdown dropdown" - :class="{ 'gl-show-field-errors': hasErrors }" > <dropdown-hidden-input :name="fieldName" :value="selectedZone" /> <dropdown-button - :class="{ 'gl-field-error-outline': hasErrors }" + :class="{ 'border-danger': hasErrors }" :is-disabled="isDisabled" :is-loading="isLoading" :toggle-text="toggleText" @@ -106,8 +105,11 @@ export default { </div> </div> <span - class="form-text text-muted" - :class="{ 'gl-field-error': hasErrors }" + class="form-text" + :class="{ + 'text-danger': hasErrors, + 'text-muted': !hasErrors + }" v-if="hasErrors" > {{ errorMessage }} diff --git a/app/assets/javascripts/vue_merge_request_widget/mr_widget_options.vue b/app/assets/javascripts/vue_merge_request_widget/mr_widget_options.vue index f69fe03fcb3..c20d07a169d 100644 --- a/app/assets/javascripts/vue_merge_request_widget/mr_widget_options.vue +++ b/app/assets/javascripts/vue_merge_request_widget/mr_widget_options.vue @@ -265,10 +265,10 @@ export default { /> <section - v-if="mr.maintainerEditAllowed" + v-if="mr.allowCollaboration" class="mr-info-list mr-links" > - {{ s__("mrWidget|Allows edits from maintainers") }} + {{ s__("mrWidget|Allows commits from members who can merge to the target branch") }} </section> <mr-widget-related-links diff --git a/app/assets/javascripts/vue_merge_request_widget/stores/mr_widget_store.js b/app/assets/javascripts/vue_merge_request_widget/stores/mr_widget_store.js index e5b7e1f1c68..134aaacf9d2 100644 --- a/app/assets/javascripts/vue_merge_request_widget/stores/mr_widget_store.js +++ b/app/assets/javascripts/vue_merge_request_widget/stores/mr_widget_store.js @@ -83,7 +83,7 @@ export default class MergeRequestStore { this.canBeMerged = data.can_be_merged || false; this.isMergeAllowed = data.mergeable || false; this.mergeOngoing = data.merge_ongoing; - this.maintainerEditAllowed = data.allow_maintainer_to_push; + this.allowCollaboration = data.allow_collaboration; // Cherry-pick and Revert actions related this.canCherryPickInCurrentMR = currentUser.can_cherry_pick_on_current_merge_request || false; diff --git a/app/assets/javascripts/vue_shared/components/gl_modal.vue b/app/assets/javascripts/vue_shared/components/gl_modal.vue index d5d5a7d3798..7ba58bd5959 100644 --- a/app/assets/javascripts/vue_shared/components/gl_modal.vue +++ b/app/assets/javascripts/vue_shared/components/gl_modal.vue @@ -1,15 +1,21 @@ <script> const buttonVariants = ['danger', 'primary', 'success', 'warning']; +const sizeVariants = ['sm', 'md', 'lg']; export default { name: 'GlModal', - props: { id: { type: String, required: false, default: null, }, + modalSize: { + type: String, + required: false, + default: 'md', + validator: value => sizeVariants.includes(value), + }, headerTitleText: { type: String, required: false, @@ -27,7 +33,11 @@ export default { default: '', }, }, - + computed: { + modalSizeClass() { + return this.modalSize === 'md' ? '' : `modal-${this.modalSize}`; + }, + }, methods: { emitCancel(event) { this.$emit('cancel', event); @@ -48,6 +58,7 @@ export default { > <div class="modal-dialog" + :class="modalSizeClass" role="document" > <div class="modal-content"> diff --git a/app/assets/javascripts/vue_shared/components/tabs/tab.vue b/app/assets/javascripts/vue_shared/components/tabs/tab.vue index 2a35d6bc151..9b2f46186ac 100644 --- a/app/assets/javascripts/vue_shared/components/tabs/tab.vue +++ b/app/assets/javascripts/vue_shared/components/tabs/tab.vue @@ -26,6 +26,11 @@ export default { created() { this.isTab = true; }, + updated() { + if (this.$parent) { + this.$parent.$forceUpdate(); + } + }, }; </script> diff --git a/app/assets/javascripts/vue_shared/components/tabs/tabs.js b/app/assets/javascripts/vue_shared/components/tabs/tabs.js index 4362264caa5..9b9e4bb47bd 100644 --- a/app/assets/javascripts/vue_shared/components/tabs/tabs.js +++ b/app/assets/javascripts/vue_shared/components/tabs/tabs.js @@ -1,4 +1,11 @@ export default { + props: { + stopPropagation: { + type: Boolean, + required: false, + default: false, + }, + }, data() { return { currentIndex: 0, @@ -13,7 +20,12 @@ export default { this.tabs = this.$children.filter(child => child.isTab); this.currentIndex = this.tabs.findIndex(tab => tab.localActive); }, - setTab(index) { + setTab(e, index) { + if (this.stopPropagation) { + e.stopPropagation(); + e.preventDefault(); + } + this.tabs[this.currentIndex].localActive = false; this.tabs[index].localActive = true; @@ -36,7 +48,7 @@ export default { href: '#', }, on: { - click: () => this.setTab(i), + click: e => this.setTab(e, i), }, }, tab.$slots.title || tab.title, diff --git a/app/assets/stylesheets/bootstrap_migration.scss b/app/assets/stylesheets/bootstrap_migration.scss index fc4a4250acc..79f580546c3 100644 --- a/app/assets/stylesheets/bootstrap_migration.scss +++ b/app/assets/stylesheets/bootstrap_migration.scss @@ -32,13 +32,46 @@ legend { button, html [type="button"], [type="reset"], -[type="submit"] { +[type="submit"], +[role="button"] { // Override bootstrap reboot -webkit-appearance: inherit; + cursor: pointer; } -[role="button"] { - cursor: pointer; +h1, +h2, +h3, +h4, +h5, +h6 { + color: $gl-text-color; + font-weight: 600; +} + +h1, +.h1, +h2, +.h2, +h3, +.h3 { + margin-top: 20px; + margin-bottom: 10px; +} + +h4, +.h4, +h5, +.h5, +h6, +.h6 { + margin-top: 10px; + margin-bottom: 10px; +} + +h5, +.h5 { + font-size: $gl-font-size; } input[type="file"] { @@ -64,6 +97,10 @@ a { } } +kbd { + display: inline-block; +} + code { padding: 2px 4px; color: $red-600; @@ -113,6 +150,16 @@ table { color: $gl-text-color-secondary !important; } +.bg-success, +.bg-primary, +.bg-info, +.bg-danger, +.bg-warning { + .card-header { + color: $white-light; + } +} + // Polyfill deprecated selectors .hidden { @@ -187,6 +234,9 @@ table { } .nav-tabs { + // Override bootstrap's default border + border-bottom: 0; + .nav-link { border-top: 0; border-left: 0; @@ -201,3 +251,13 @@ table { pre code { white-space: pre-wrap; } + +.alert-danger { + background-color: $red-500; + border-color: $red-500; + color: $white-light; + + h4 { + color: $white-light; + } +} diff --git a/app/assets/stylesheets/framework/common.scss b/app/assets/stylesheets/framework/common.scss index 1e7b9534275..996e5c1512d 100644 --- a/app/assets/stylesheets/framework/common.scss +++ b/app/assets/stylesheets/framework/common.scss @@ -448,6 +448,10 @@ img.emoji { .break-word { word-wrap: break-word; + + &.all-words { + word-break: break-word; + } } /** COMMON CLASSES **/ diff --git a/app/assets/stylesheets/framework/contextual_sidebar.scss b/app/assets/stylesheets/framework/contextual_sidebar.scss index 1a415e1b852..9cbaaa5dc8d 100644 --- a/app/assets/stylesheets/framework/contextual_sidebar.scss +++ b/app/assets/stylesheets/framework/contextual_sidebar.scss @@ -26,19 +26,25 @@ margin-right: 2px; width: $contextual-sidebar-width; - a { + > a, + > button { transition: padding $sidebar-transition-duration; font-weight: $gl-font-weight-bold; display: flex; + width: 100%; align-items: center; padding: 10px 16px 10px 10px; color: $gl-text-color; - } + background-color: transparent; + border: 0; + text-align: left; - &:hover, - a:hover { - background-color: $link-hover-background; - color: $gl-text-color; + &:hover, + &:focus { + background-color: $link-hover-background; + color: $gl-text-color; + outline: 0; + } } .avatar-container { diff --git a/app/assets/stylesheets/framework/filters.scss b/app/assets/stylesheets/framework/filters.scss index 0ee5748952a..551a7e852ae 100644 --- a/app/assets/stylesheets/framework/filters.scss +++ b/app/assets/stylesheets/framework/filters.scss @@ -299,6 +299,7 @@ height: 14px; width: 14px; vertical-align: middle; + margin-bottom: 4px; } .dropdown-toggle-text { diff --git a/app/assets/stylesheets/framework/gitlab_theme.scss b/app/assets/stylesheets/framework/gitlab_theme.scss index 14dd3879bdc..b40d02f381a 100644 --- a/app/assets/stylesheets/framework/gitlab_theme.scss +++ b/app/assets/stylesheets/framework/gitlab_theme.scss @@ -3,26 +3,26 @@ */ @mixin gitlab-theme( - $color-100, - $color-200, - $color-500, - $color-700, - $color-800, - $color-900, + $location-badge-color, + $search-and-nav-links, + $active-tab-border, + $border-and-box-shadow, + $sidebar-text, + $nav-svg-color, $color-alternate ) { // Header .navbar-gitlab { - background-color: $color-900; + background-color: $nav-svg-color; .navbar-collapse { - color: $color-200; + color: $search-and-nav-links; } .container-fluid { .navbar-toggler { - border-left: 1px solid lighten($color-700, 10%); + border-left: 1px solid lighten($border-and-box-shadow, 10%); } } @@ -31,40 +31,40 @@ > li { > a:hover, > a:focus { - background-color: rgba($color-200, 0.2); + background-color: rgba($search-and-nav-links, 0.2); } &.active > a, &.dropdown.show > a { - color: $color-900; + color: $nav-svg-color; background-color: $color-alternate; } &.line-separator { - border-left: 1px solid rgba($color-200, 0.2); + border-left: 1px solid rgba($search-and-nav-links, 0.2); } } } .navbar-sub-nav { - color: $color-200; + color: $search-and-nav-links; } .nav { > li { - color: $color-200; + color: $search-and-nav-links; > a { &.header-user-dropdown-toggle { .header-user-avatar { - border-color: $color-200; + border-color: $search-and-nav-links; } } &:hover, &:focus { @include media-breakpoint-up(sm) { - background-color: rgba($color-200, 0.2); + background-color: rgba($search-and-nav-links, 0.2); } svg { @@ -75,12 +75,12 @@ &.active > a, &.dropdown.show > a { - color: $color-900; + color: $nav-svg-color; background-color: $color-alternate; &:hover { svg { - fill: $color-900; + fill: $nav-svg-color; } } } @@ -88,7 +88,7 @@ .impersonated-user, .impersonated-user:hover { svg { - fill: $color-900; + fill: $nav-svg-color; } } } @@ -99,34 +99,34 @@ > a { &:hover, &:focus { - background-color: rgba($color-200, 0.2); + background-color: rgba($search-and-nav-links, 0.2); } } } .search { form { - background-color: rgba($color-200, 0.2); + background-color: rgba($search-and-nav-links, 0.2); &:hover { - background-color: rgba($color-200, 0.3); + background-color: rgba($search-and-nav-links, 0.3); } } .location-badge { - color: $color-100; - background-color: rgba($color-200, 0.1); - border-right: 1px solid $color-800; + color: $location-badge-color; + background-color: rgba($search-and-nav-links, 0.1); + border-right: 1px solid $sidebar-text; } .search-input::placeholder { - color: rgba($color-200, 0.8); + color: rgba($search-and-nav-links, 0.8); } .search-input-wrap { .search-icon, .clear-icon { - fill: rgba($color-200, 0.8); + fill: rgba($search-and-nav-links, 0.8); } } @@ -141,38 +141,34 @@ .search-input-wrap { .search-icon { - fill: rgba($color-200, 0.8); + fill: rgba($search-and-nav-links, 0.8); } } } } - .btn-sign-in { - background-color: $color-100; - color: $color-900; - } // Sidebar .nav-sidebar li.active { - box-shadow: inset 4px 0 0 $color-700; + box-shadow: inset 4px 0 0 $border-and-box-shadow; > a { - color: $color-800; + color: $sidebar-text; } svg { - fill: $color-800; + fill: $sidebar-text; } } .sidebar-top-level-items > li.active .badge.badge-pill { - color: $color-800; + color: $sidebar-text; } .nav-links li { &.active a, a.active { - border-bottom: 2px solid $color-500; + border-bottom: 2px solid $active-tab-border; .badge.badge-pill { font-weight: $gl-font-weight-bold; @@ -181,27 +177,27 @@ } .branch-header-title { - color: $color-700; + color: $border-and-box-shadow; } .ide-file-list .file.file-active { - color: $color-700; + color: $border-and-box-shadow; } .ide-sidebar-link { &.active { - color: $color-700; - box-shadow: inset 3px 0 $color-700; + color: $border-and-box-shadow; + box-shadow: inset 3px 0 $border-and-box-shadow; &.is-right { - box-shadow: inset -3px 0 $color-700; + box-shadow: inset -3px 0 $border-and-box-shadow; } } } } body { - &.ui_indigo { + &.ui-indigo { @include gitlab-theme( $indigo-100, $indigo-200, @@ -213,19 +209,19 @@ body { ); } - &.ui_dark { + &.ui-light-indigo { @include gitlab-theme( - $theme-gray-100, - $theme-gray-200, - $theme-gray-500, - $theme-gray-700, - $theme-gray-800, - $theme-gray-900, + $indigo-100, + $indigo-200, + $indigo-500, + $indigo-500, + $indigo-700, + $indigo-700, $white-light ); } - &.ui_blue { + &.ui-blue { @include gitlab-theme( $theme-blue-100, $theme-blue-200, @@ -237,7 +233,19 @@ body { ); } - &.ui_green { + &.ui-light-blue { + @include gitlab-theme( + $theme-light-blue-100, + $theme-light-blue-200, + $theme-light-blue-500, + $theme-light-blue-500, + $theme-light-blue-700, + $theme-light-blue-700, + $white-light + ); + } + + &.ui-green { @include gitlab-theme( $theme-green-100, $theme-green-200, @@ -249,7 +257,55 @@ body { ); } - &.ui_light { + &.ui-light-green { + @include gitlab-theme( + $theme-green-100, + $theme-green-200, + $theme-green-500, + $theme-green-500, + $theme-light-green-700, + $theme-light-green-700, + $white-light + ); + } + + &.ui-red { + @include gitlab-theme( + $theme-red-100, + $theme-red-200, + $theme-red-500, + $theme-red-700, + $theme-red-800, + $theme-red-900, + $white-light + ); + } + + &.ui-light-red { + @include gitlab-theme( + $theme-light-red-100, + $theme-light-red-200, + $theme-light-red-500, + $theme-light-red-500, + $theme-light-red-700, + $theme-light-red-700, + $white-light + ); + } + + &.ui-dark { + @include gitlab-theme( + $theme-gray-100, + $theme-gray-200, + $theme-gray-500, + $theme-gray-700, + $theme-gray-800, + $theme-gray-900, + $white-light + ); + } + + &.ui-light { @include gitlab-theme( $theme-gray-900, $theme-gray-700, diff --git a/app/assets/stylesheets/framework/header.scss b/app/assets/stylesheets/framework/header.scss index 094134b63b0..f2ac77819d5 100644 --- a/app/assets/stylesheets/framework/header.scss +++ b/app/assets/stylesheets/framework/header.scss @@ -437,6 +437,8 @@ } .btn-sign-in { + background-color: $indigo-100; + color: $indigo-900; margin-top: 3px; font-weight: $gl-font-weight-bold; diff --git a/app/assets/stylesheets/framework/layout.scss b/app/assets/stylesheets/framework/layout.scss index 0536c39cee7..55c0bc76f23 100644 --- a/app/assets/stylesheets/framework/layout.scss +++ b/app/assets/stylesheets/framework/layout.scss @@ -115,9 +115,3 @@ body { .with-performance-bar .layout-page { margin-top: $header-height + $performance-bar-height; } - -.vertical-center { - min-height: 100vh; - display: flex; - align-items: center; -} diff --git a/app/assets/stylesheets/framework/tables.scss b/app/assets/stylesheets/framework/tables.scss index a10bd1544c5..10c23f6c407 100644 --- a/app/assets/stylesheets/framework/tables.scss +++ b/app/assets/stylesheets/framework/tables.scss @@ -1,5 +1,6 @@ .table-holder { margin: 0; + overflow: auto; } table { diff --git a/app/assets/stylesheets/framework/toggle.scss b/app/assets/stylesheets/framework/toggle.scss index d5cc78a6680..20394cc1e52 100644 --- a/app/assets/stylesheets/framework/toggle.scss +++ b/app/assets/stylesheets/framework/toggle.scss @@ -42,6 +42,10 @@ background: none; } + &:focus { + outline: none; + } + .toggle-icon { position: relative; display: block; diff --git a/app/assets/stylesheets/framework/variables.scss b/app/assets/stylesheets/framework/variables.scss index 946223cfff0..e6e74d55f64 100644 --- a/app/assets/stylesheets/framework/variables.scss +++ b/app/assets/stylesheets/framework/variables.scss @@ -117,6 +117,15 @@ $theme-blue-800: #25496e; $theme-blue-900: #1a3652; $theme-blue-950: #0f2235; +$theme-light-blue-50: #f2f7fc; +$theme-light-blue-100: #ebf1f7; +$theme-light-blue-200: #c9dcf2; +$theme-light-blue-300: #83abd4; +$theme-light-blue-400: #4d86bf; +$theme-light-blue-500: #367cc2; +$theme-light-blue-600: #3771ab; +$theme-light-blue-700: #2261a1; + $theme-green-50: #f2faf6; $theme-green-100: #e4f3ea; $theme-green-200: #c0dfcd; @@ -129,6 +138,29 @@ $theme-green-800: #145d33; $theme-green-900: #0d4524; $theme-green-950: #072d16; +$theme-light-green-700: #156b39; + +$theme-red-50: #fcf4f2; +$theme-red-100: #fae9e6; +$theme-red-200: #ebcac5; +$theme-red-300: #d99b91; +$theme-red-400: #b0655a; +$theme-red-500: #ad4a3b; +$theme-red-600: #9e4133; +$theme-red-700: #912f20; +$theme-red-800: #78291d; +$theme-red-900: #691a16; +$theme-red-950: #36140f; + +$theme-light-red-50: #fff6f5; +$theme-light-red-100: #fae2de; +$theme-light-red-200: #f7d5d0; +$theme-light-red-300: #d9796a; +$theme-light-red-400: #cf604e; +$theme-light-red-500: #c24b38; +$theme-light-red-600: #b03927; +$theme-light-red-700: #a62e21; + $black: #000; $black-transparent: rgba(0, 0, 0, 0.3); $almost-black: #242424; diff --git a/app/assets/stylesheets/pages/boards.scss b/app/assets/stylesheets/pages/boards.scss index 1c3d312f7ac..b2416a3d5bc 100644 --- a/app/assets/stylesheets/pages/boards.scss +++ b/app/assets/stylesheets/pages/boards.scss @@ -282,9 +282,6 @@ box-shadow: 0 1px 2px $issue-boards-card-shadow; list-style: none; - // as a fallback, hide overflow content so that dragging and dropping still works - overflow: hidden; - &:not(:last-child) { margin-bottom: 5px; } diff --git a/app/assets/stylesheets/pages/builds.scss b/app/assets/stylesheets/pages/builds.scss index 9ee02ca1d83..9213ccd4cdf 100644 --- a/app/assets/stylesheets/pages/builds.scss +++ b/app/assets/stylesheets/pages/builds.scss @@ -75,6 +75,7 @@ .top-bar { height: 35px; + min-height: 35px; background: $gray-light; border: 1px solid $border-color; color: $gl-text-color; diff --git a/app/assets/stylesheets/pages/clusters.scss b/app/assets/stylesheets/pages/clusters.scss index 3e4d123242c..56beb7718a4 100644 --- a/app/assets/stylesheets/pages/clusters.scss +++ b/app/assets/stylesheets/pages/clusters.scss @@ -13,6 +13,10 @@ max-width: 100%; } +.clusters-error-alert { + width: 100%; +} + .clusters-container { .nav-bar-right { padding: $gl-padding-top $gl-padding; diff --git a/app/assets/stylesheets/pages/environments.scss b/app/assets/stylesheets/pages/environments.scss index cd0d67613c3..06f08ae2215 100644 --- a/app/assets/stylesheets/pages/environments.scss +++ b/app/assets/stylesheets/pages/environments.scss @@ -23,7 +23,6 @@ } .btn-group { - > a { color: $gl-text-color-secondary; } @@ -245,6 +244,7 @@ .prometheus-graph { flex: 1 0 auto; min-width: 450px; + max-width: 100%; padding: $gl-padding / 2; h5 { @@ -256,6 +256,17 @@ } } +.prometheus-graph-header { + display: flex; + align-items: center; + justify-content: space-between; + margin-bottom: $gl-padding-8; + + h5 { + margin: 0; + } +} + .prometheus-graph-cursor { position: absolute; background: $theme-gray-600; diff --git a/app/assets/stylesheets/pages/profiles/preferences.scss b/app/assets/stylesheets/pages/profiles/preferences.scss index 68d40b56133..babe81cb0f7 100644 --- a/app/assets/stylesheets/pages/profiles/preferences.scss +++ b/app/assets/stylesheets/pages/profiles/preferences.scss @@ -1,25 +1,3 @@ -@mixin application-theme-preview($color-1, $color-2, $color-3, $color-4) { - .one { - background-color: $color-1; - border-top-left-radius: $border-radius-default; - } - - .two { - background-color: $color-2; - border-top-right-radius: $border-radius-default; - } - - .three { - background-color: $color-3; - border-bottom-left-radius: $border-radius-default; - } - - .four { - background-color: $color-4; - border-bottom-right-radius: $border-radius-default; - } -} - .multi-file-editor-options { label { margin-right: 20px; @@ -38,44 +16,61 @@ .application-theme { label { - margin-right: 20px; + margin: 0 $gl-padding $gl-padding 0; text-align: center; } .preview { font-size: 0; - margin-bottom: 10px; + height: 48px; + border-radius: 4px; + min-width: 135px; + margin-bottom: $gl-padding-8; + + &.ui-indigo { + background-color: $indigo-900; + } + + &.ui-light-indigo { + background-color: $indigo-700; + } - &.indigo { - @include application-theme-preview($indigo-900, $indigo-700, $indigo-800, $indigo-500); + &.ui-blue { + background-color: $theme-blue-900; } - &.dark { - @include application-theme-preview($theme-gray-900, $theme-gray-700, $theme-gray-800, $theme-gray-600); + &.ui-light-blue { + background-color: $theme-light-blue-700; } - &.light { - @include application-theme-preview($theme-gray-600, $theme-gray-200, $theme-gray-400, $theme-gray-100); + &.ui-green { + background-color: $theme-green-900; } - &.blue { - @include application-theme-preview($theme-blue-900, $theme-blue-700, $theme-blue-800, $theme-blue-500); + &.ui-light-green { + background-color: $theme-light-green-700; } - &.green { - @include application-theme-preview($theme-green-900, $theme-green-700, $theme-green-800, $theme-green-500); + &.ui-red { + background-color: $theme-red-900; + } + + &.ui-light-red { + background-color: $theme-light-red-700; + } + + &.ui-dark { + background-color: $theme-gray-900; + } + + &.ui-light { + background-color: $theme-gray-200; } } .preview-row { display: block; } - - .quadrant { - display: inline-block; - height: 50px; - width: 80px; - } } .syntax-theme { diff --git a/app/assets/stylesheets/pages/repo.scss b/app/assets/stylesheets/pages/repo.scss index 2b3cc33c8ae..3c7edb0d4bb 100644 --- a/app/assets/stylesheets/pages/repo.scss +++ b/app/assets/stylesheets/pages/repo.scss @@ -458,14 +458,10 @@ width: auto; margin-right: 0; - a { + > a, + > button { height: 60px; } - - a:hover, - a:focus { - text-decoration: none; - } } .projects-sidebar { @@ -1135,6 +1131,11 @@ .avatar { flex: 0 0 40px; } + + .ide-merge-requests-dropdown.dropdown-menu { + width: 385px; + max-height: initial; + } } .ide-sidebar-project-title { @@ -1143,11 +1144,20 @@ .sidebar-context-title { white-space: nowrap; } + + .ide-sidebar-branch-title { + min-width: 50px; + } } .ide-external-link { + position: relative; + svg { display: none; + position: absolute; + top: 2px; + right: -$gl-padding; } &:hover, @@ -1178,6 +1188,8 @@ display: flex; flex-direction: column; height: 100%; + margin-top: -$grid-size; + margin-bottom: -$grid-size; .empty-state { margin-top: auto; @@ -1194,6 +1206,17 @@ margin: 0; } } + + .build-trace, + .top-bar { + margin-left: -$gl-padding; + } + + &.build-page .top-bar { + top: 0; + font-size: 12px; + border-top-right-radius: $border-radius-default; + } } .ide-pipeline-list { @@ -1202,7 +1225,7 @@ } .ide-pipeline-header { - min-height: 50px; + min-height: 55px; padding-left: $gl-padding; padding-right: $gl-padding; @@ -1222,8 +1245,7 @@ .ci-status-icon { display: flex; justify-content: center; - height: 20px; - margin-top: -2px; + min-width: 24px; overflow: hidden; } } @@ -1253,3 +1275,56 @@ overflow: hidden; text-overflow: ellipsis; } + +.ide-job-header { + min-height: 60px; +} + +.ide-merge-requests-dropdown { + .nav-links li { + width: 50%; + padding-left: 0; + padding-right: 0; + + a { + text-align: center; + + &:not(.active) { + background-color: $gray-light; + } + } + } + + .dropdown-input { + padding-left: $gl-padding; + padding-right: $gl-padding; + + .fa { + right: 26px; + } + } + + .btn-link { + padding-top: $gl-padding; + padding-bottom: $gl-padding; + } +} + +.ide-merge-request-current-icon { + min-width: 18px; +} + +.ide-merge-requests-empty { + height: 230px; +} + +.ide-merge-requests-dropdown-content { + min-height: 230px; + max-height: 470px; +} + +.ide-merge-request-project-path { + font-size: 12px; + line-height: 16px; + color: $gl-text-color-secondary; +} diff --git a/app/assets/stylesheets/pages/settings.scss b/app/assets/stylesheets/pages/settings.scss index 43c39c43bc1..16e999341da 100644 --- a/app/assets/stylesheets/pages/settings.scss +++ b/app/assets/stylesheets/pages/settings.scss @@ -102,10 +102,6 @@ .form-text.text-muted { margin-top: 0; } - - .label-light { - margin-bottom: 0; - } } .settings-list-icon { diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb index db8a8cdc0d2..041837c5410 100644 --- a/app/controllers/application_controller.rb +++ b/app/controllers/application_controller.rb @@ -91,6 +91,10 @@ class ApplicationController < ActionController::Base payload[:user_id] = logged_user.try(:id) payload[:username] = logged_user.try(:username) end + + if response.status == 422 && response.body.present? && response.content_type == 'application/json'.freeze + payload[:response] = response.body + end end # Controllers such as GitHttpController may use alternative methods @@ -130,12 +134,17 @@ class ApplicationController < ActionController::Base end def access_denied!(message = nil) + # If we display a custom access denied message to the user, we don't want to + # hide existence of the resource, rather tell them they cannot access it using + # the provided message + status = message.present? ? :forbidden : :not_found + respond_to do |format| - format.any { head :not_found } + format.any { head status } format.html do render "errors/access_denied", layout: "errors", - status: 404, + status: status, locals: { message: message } end end diff --git a/app/controllers/graphql_controller.rb b/app/controllers/graphql_controller.rb new file mode 100644 index 00000000000..0a1cf169aca --- /dev/null +++ b/app/controllers/graphql_controller.rb @@ -0,0 +1,45 @@ +class GraphqlController < ApplicationController + # Unauthenticated users have access to the API for public data + skip_before_action :authenticate_user! + + before_action :check_graphql_feature_flag! + + def execute + variables = Gitlab::Graphql::Variables.new(params[:variables]).to_h + query = params[:query] + operation_name = params[:operationName] + context = { + current_user: current_user + } + result = GitlabSchema.execute(query, variables: variables, context: context, operation_name: operation_name) + render json: result + end + + rescue_from StandardError do |exception| + log_exception(exception) + + render_error("Internal server error") + end + + rescue_from Gitlab::Graphql::Variables::Invalid do |exception| + render_error(exception.message, status: :unprocessable_entity) + end + + private + + # Overridden from the ApplicationController to make the response look like + # a GraphQL response. That is nicely picked up in Graphiql. + def render_404 + render_error("Not found!", status: :not_found) + end + + def render_error(message, status: 500) + error = { errors: [message: message] } + + render json: error, status: status + end + + def check_graphql_feature_flag! + render_404 unless Feature.enabled?(:graphql) + end +end diff --git a/app/controllers/groups/group_members_controller.rb b/app/controllers/groups/group_members_controller.rb index ef3eba80154..ef5d5e5c742 100644 --- a/app/controllers/groups/group_members_controller.rb +++ b/app/controllers/groups/group_members_controller.rb @@ -3,8 +3,12 @@ class Groups::GroupMembersController < Groups::ApplicationController include MembersPresentation include SortingHelper + def self.admin_not_required_endpoints + %i[index leave request_access] + end + # Authorize - before_action :authorize_admin_group_member!, except: [:index, :leave, :request_access] + before_action :authorize_admin_group_member!, except: admin_not_required_endpoints skip_cross_project_access_check :index, :create, :update, :destroy, :request_access, :approve_access_request, :leave, :resend_invite, diff --git a/app/controllers/groups/milestones_controller.rb b/app/controllers/groups/milestones_controller.rb index 5903689dc62..9bd51de7e97 100644 --- a/app/controllers/groups/milestones_controller.rb +++ b/app/controllers/groups/milestones_controller.rb @@ -76,12 +76,15 @@ class Groups::MilestonesController < Groups::ApplicationController def milestones milestones = MilestonesFinder.new(search_params).execute - legacy_milestones = GroupMilestone.build_collection(group, group_projects, params) @sort = params[:sort] || 'due_date_asc' MilestoneArray.sort(milestones + legacy_milestones, @sort) end + def legacy_milestones + GroupMilestone.build_collection(group, group_projects, params) + end + def milestone @milestone = if params[:title] diff --git a/app/controllers/groups/shared_projects_controller.rb b/app/controllers/groups/shared_projects_controller.rb index f2f835767e0..7dec1f5f402 100644 --- a/app/controllers/groups/shared_projects_controller.rb +++ b/app/controllers/groups/shared_projects_controller.rb @@ -24,7 +24,9 @@ module Groups # Make the `search` param consistent for the frontend, # which will be using `filter`. params[:search] ||= params[:filter] if params[:filter] - params.permit(:sort, :search) + # Don't show archived projects + params[:non_archived] = true + params.permit(:sort, :search, :non_archived) end end end diff --git a/app/controllers/import/base_controller.rb b/app/controllers/import/base_controller.rb index 663269a0f92..5766c6924cd 100644 --- a/app/controllers/import/base_controller.rb +++ b/app/controllers/import/base_controller.rb @@ -25,4 +25,8 @@ class Import::BaseController < ApplicationController current_user.namespace end + + def project_save_error(project) + project.errors.full_messages.join(', ') + end end diff --git a/app/controllers/import/bitbucket_controller.rb b/app/controllers/import/bitbucket_controller.rb index 77af5fb9c4f..fa31933e778 100644 --- a/app/controllers/import/bitbucket_controller.rb +++ b/app/controllers/import/bitbucket_controller.rb @@ -55,7 +55,7 @@ class Import::BitbucketController < Import::BaseController if project.persisted? render json: ProjectSerializer.new.represent(project) else - render json: { errors: project.errors.full_messages }, status: :unprocessable_entity + render json: { errors: project_save_error(project) }, status: :unprocessable_entity end else render json: { errors: 'This namespace has already been taken! Please choose another one.' }, status: :unprocessable_entity diff --git a/app/controllers/import/fogbugz_controller.rb b/app/controllers/import/fogbugz_controller.rb index 25ec13b8075..2d665e05ac3 100644 --- a/app/controllers/import/fogbugz_controller.rb +++ b/app/controllers/import/fogbugz_controller.rb @@ -66,7 +66,7 @@ class Import::FogbugzController < Import::BaseController if project.persisted? render json: ProjectSerializer.new.represent(project) else - render json: { errors: project.errors.full_messages }, status: :unprocessable_entity + render json: { errors: project_save_error(project) }, status: :unprocessable_entity end end diff --git a/app/controllers/import/github_controller.rb b/app/controllers/import/github_controller.rb index f67ec4c248b..c9870332c0f 100644 --- a/app/controllers/import/github_controller.rb +++ b/app/controllers/import/github_controller.rb @@ -48,7 +48,7 @@ class Import::GithubController < Import::BaseController if project.persisted? render json: ProjectSerializer.new.represent(project) else - render json: { errors: project.errors.full_messages }, status: :unprocessable_entity + render json: { errors: project_save_error(project) }, status: :unprocessable_entity end else render json: { errors: 'This namespace has already been taken! Please choose another one.' }, status: :unprocessable_entity diff --git a/app/controllers/import/gitlab_controller.rb b/app/controllers/import/gitlab_controller.rb index 39e2e9e094b..fccbdbca0f6 100644 --- a/app/controllers/import/gitlab_controller.rb +++ b/app/controllers/import/gitlab_controller.rb @@ -32,7 +32,7 @@ class Import::GitlabController < Import::BaseController if project.persisted? render json: ProjectSerializer.new.represent(project) else - render json: { errors: project.errors.full_messages }, status: :unprocessable_entity + render json: { errors: project_save_error(project) }, status: :unprocessable_entity end else render json: { errors: 'This namespace has already been taken! Please choose another one.' }, status: :unprocessable_entity diff --git a/app/controllers/import/google_code_controller.rb b/app/controllers/import/google_code_controller.rb index 9b26a00f7c7..3bce27e810a 100644 --- a/app/controllers/import/google_code_controller.rb +++ b/app/controllers/import/google_code_controller.rb @@ -92,7 +92,7 @@ class Import::GoogleCodeController < Import::BaseController if project.persisted? render json: ProjectSerializer.new.represent(project) else - render json: { errors: project.errors.full_messages }, status: :unprocessable_entity + render json: { errors: project_save_error(project) }, status: :unprocessable_entity end end diff --git a/app/controllers/projects/lfs_storage_controller.rb b/app/controllers/projects/lfs_storage_controller.rb index 43d8867a536..45c98d60822 100644 --- a/app/controllers/projects/lfs_storage_controller.rb +++ b/app/controllers/projects/lfs_storage_controller.rb @@ -18,7 +18,7 @@ class Projects::LfsStorageController < Projects::GitHttpClientController def upload_authorize set_workhorse_internal_api_content_type - authorized = LfsObjectUploader.workhorse_authorize + authorized = LfsObjectUploader.workhorse_authorize(has_length: true) authorized.merge!(LfsOid: oid, LfsSize: size) render json: authorized diff --git a/app/controllers/projects/merge_requests/application_controller.rb b/app/controllers/projects/merge_requests/application_controller.rb index 29632bef7e5..8e4aeec16dc 100644 --- a/app/controllers/projects/merge_requests/application_controller.rb +++ b/app/controllers/projects/merge_requests/application_controller.rb @@ -15,7 +15,7 @@ class Projects::MergeRequests::ApplicationController < Projects::ApplicationCont def merge_request_params_attributes [ - :allow_maintainer_to_push, + :allow_collaboration, :assignee_id, :description, :force_remove_source_branch, diff --git a/app/controllers/projects/merge_requests_controller.rb b/app/controllers/projects/merge_requests_controller.rb index ecea6e1b2bf..b452bfd7e6f 100644 --- a/app/controllers/projects/merge_requests_controller.rb +++ b/app/controllers/projects/merge_requests_controller.rb @@ -28,15 +28,14 @@ class Projects::MergeRequestsController < Projects::MergeRequests::ApplicationCo end def show - validates_merge_request - close_merge_request_without_source_project - check_if_can_be_merged - - # Return if the response has already been rendered - return if response_body + close_merge_request_if_no_source_project + mark_merge_request_mergeable respond_to do |format| format.html do + # use next to appease Rubocop + next render('invalid') if target_branch_missing? + # Build a note object for comment form @note = @project.notes.new(noteable: @merge_request) @@ -234,20 +233,6 @@ class Projects::MergeRequestsController < Projects::MergeRequests::ApplicationCo alias_method :issuable, :merge_request alias_method :awardable, :merge_request - def validates_merge_request - # Show git not found page - # if there is no saved commits between source & target branch - if @merge_request.has_no_commits? - # and if target branch doesn't exist - return invalid_mr unless @merge_request.target_branch_exists? - end - end - - def invalid_mr - # Render special view for MR with removed target branch - render 'invalid' - end - def merge_params params.permit(merge_params_attributes) end @@ -261,7 +246,7 @@ class Projects::MergeRequestsController < Projects::MergeRequests::ApplicationCo @merge_request.head_pipeline && @merge_request.head_pipeline.active? end - def close_merge_request_without_source_project + def close_merge_request_if_no_source_project if !@merge_request.source_project && @merge_request.open? @merge_request.close end @@ -269,7 +254,11 @@ class Projects::MergeRequestsController < Projects::MergeRequests::ApplicationCo private - def check_if_can_be_merged + def target_branch_missing? + @merge_request.has_no_commits? && !@merge_request.target_branch_exists? + end + + def mark_merge_request_mergeable @merge_request.check_if_can_be_merged end diff --git a/app/controllers/projects/milestones_controller.rb b/app/controllers/projects/milestones_controller.rb index c5a044541f1..2494b56981d 100644 --- a/app/controllers/projects/milestones_controller.rb +++ b/app/controllers/projects/milestones_controller.rb @@ -1,4 +1,5 @@ class Projects::MilestonesController < Projects::ApplicationController + include Gitlab::Utils::StrongMemoize include MilestoneActions before_action :check_issuables_available! @@ -103,7 +104,7 @@ class Projects::MilestonesController < Projects::ApplicationController protected def milestones - @milestones ||= begin + strong_memoize(:milestones) do MilestonesFinder.new(search_params).execute end end @@ -121,10 +122,10 @@ class Projects::MilestonesController < Projects::ApplicationController end def search_params - if @project.group && can?(current_user, :read_group, @project.group) - group = @project.group + if request.format.json? && @project.group && can?(current_user, :read_group, @project.group) + groups = @project.group.self_and_ancestors end - params.permit(:state).merge(project_ids: @project.id, group_ids: group&.id) + params.permit(:state).merge(project_ids: @project.id, group_ids: groups&.select(:id)) end end diff --git a/app/controllers/projects/pipelines_controller.rb b/app/controllers/projects/pipelines_controller.rb index 6b40fc2fe68..768595ceeb4 100644 --- a/app/controllers/projects/pipelines_controller.rb +++ b/app/controllers/projects/pipelines_controller.rb @@ -23,8 +23,6 @@ class Projects::PipelinesController < Projects::ApplicationController @finished_count = limited_pipelines_count(project, 'finished') @pipelines_count = limited_pipelines_count(project) - Gitlab::Ci::Pipeline::Preloader.preload(@pipelines) - respond_to do |format| format.html format.json do @@ -34,7 +32,7 @@ class Projects::PipelinesController < Projects::ApplicationController pipelines: PipelineSerializer .new(project: @project, current_user: @current_user) .with_pagination(request, response) - .represent(@pipelines, disable_coverage: true), + .represent(@pipelines, disable_coverage: true, preload: true), count: { all: @pipelines_count, running: @running_count, diff --git a/app/controllers/users/terms_controller.rb b/app/controllers/users/terms_controller.rb index ab685b9106e..f7c6d1d59db 100644 --- a/app/controllers/users/terms_controller.rb +++ b/app/controllers/users/terms_controller.rb @@ -13,6 +13,10 @@ module Users def index @redirect = redirect_path + + if @term.accepted_by_user?(current_user) + flash.now[:notice] = "You have already accepted the Terms of Service as #{current_user.to_reference}" + end end def accept diff --git a/app/graphql/functions/base_function.rb b/app/graphql/functions/base_function.rb new file mode 100644 index 00000000000..42fb8f99acc --- /dev/null +++ b/app/graphql/functions/base_function.rb @@ -0,0 +1,4 @@ +module Functions + class BaseFunction < GraphQL::Function + end +end diff --git a/app/graphql/functions/echo.rb b/app/graphql/functions/echo.rb new file mode 100644 index 00000000000..e5bf109b8d7 --- /dev/null +++ b/app/graphql/functions/echo.rb @@ -0,0 +1,13 @@ +module Functions + class Echo < BaseFunction + argument :text, GraphQL::STRING_TYPE + + description "Testing endpoint to validate the API with" + + def call(obj, args, ctx) + username = ctx[:current_user]&.username + + "#{username.inspect} says: #{args[:text]}" + end + end +end diff --git a/app/graphql/gitlab_schema.rb b/app/graphql/gitlab_schema.rb new file mode 100644 index 00000000000..de4fc1d8e32 --- /dev/null +++ b/app/graphql/gitlab_schema.rb @@ -0,0 +1,8 @@ +class GitlabSchema < GraphQL::Schema + use BatchLoader::GraphQL + use Gitlab::Graphql::Authorize + use Gitlab::Graphql::Present + + query(Types::QueryType) + # mutation(Types::MutationType) +end diff --git a/app/graphql/mutations/.keep b/app/graphql/mutations/.keep new file mode 100644 index 00000000000..e69de29bb2d --- /dev/null +++ b/app/graphql/mutations/.keep diff --git a/app/graphql/resolvers/base_resolver.rb b/app/graphql/resolvers/base_resolver.rb new file mode 100644 index 00000000000..89b7f9dad6f --- /dev/null +++ b/app/graphql/resolvers/base_resolver.rb @@ -0,0 +1,4 @@ +module Resolvers + class BaseResolver < GraphQL::Schema::Resolver + end +end diff --git a/app/graphql/resolvers/full_path_resolver.rb b/app/graphql/resolvers/full_path_resolver.rb new file mode 100644 index 00000000000..4eb28aaed6c --- /dev/null +++ b/app/graphql/resolvers/full_path_resolver.rb @@ -0,0 +1,19 @@ +module Resolvers + module FullPathResolver + extend ActiveSupport::Concern + + prepended do + argument :full_path, GraphQL::ID_TYPE, + required: true, + description: 'The full path of the project or namespace, e.g., "gitlab-org/gitlab-ce"' + end + + def model_by_full_path(model, full_path) + BatchLoader.for(full_path).batch(key: "#{model.model_name.param_key}:full_path") do |full_paths, loader| + # `with_route` avoids an N+1 calculating full_path + results = model.where_full_path_in(full_paths).with_route + results.each { |project| loader.call(project.full_path, project) } + end + end + end +end diff --git a/app/graphql/resolvers/merge_request_resolver.rb b/app/graphql/resolvers/merge_request_resolver.rb new file mode 100644 index 00000000000..b1857ab09f7 --- /dev/null +++ b/app/graphql/resolvers/merge_request_resolver.rb @@ -0,0 +1,21 @@ +module Resolvers + class MergeRequestResolver < BaseResolver + prepend FullPathResolver + + type Types::ProjectType, null: true + + argument :iid, GraphQL::ID_TYPE, + required: true, + description: 'The IID of the merge request, e.g., "1"' + + def resolve(full_path:, iid:) + project = model_by_full_path(Project, full_path) + return unless project.present? + + BatchLoader.for(iid.to_s).batch(key: project.id) do |iids, loader| + results = project.merge_requests.where(iid: iids) + results.each { |mr| loader.call(mr.iid.to_s, mr) } + end + end + end +end diff --git a/app/graphql/resolvers/project_resolver.rb b/app/graphql/resolvers/project_resolver.rb new file mode 100644 index 00000000000..ec115bad896 --- /dev/null +++ b/app/graphql/resolvers/project_resolver.rb @@ -0,0 +1,11 @@ +module Resolvers + class ProjectResolver < BaseResolver + prepend FullPathResolver + + type Types::ProjectType, null: true + + def resolve(full_path:) + model_by_full_path(Project, full_path) + end + end +end diff --git a/app/graphql/types/base_enum.rb b/app/graphql/types/base_enum.rb new file mode 100644 index 00000000000..b45a845f74f --- /dev/null +++ b/app/graphql/types/base_enum.rb @@ -0,0 +1,4 @@ +module Types + class BaseEnum < GraphQL::Schema::Enum + end +end diff --git a/app/graphql/types/base_field.rb b/app/graphql/types/base_field.rb new file mode 100644 index 00000000000..c5740a334d7 --- /dev/null +++ b/app/graphql/types/base_field.rb @@ -0,0 +1,5 @@ +module Types + class BaseField < GraphQL::Schema::Field + prepend Gitlab::Graphql::Authorize + end +end diff --git a/app/graphql/types/base_input_object.rb b/app/graphql/types/base_input_object.rb new file mode 100644 index 00000000000..309e336e6c8 --- /dev/null +++ b/app/graphql/types/base_input_object.rb @@ -0,0 +1,4 @@ +module Types + class BaseInputObject < GraphQL::Schema::InputObject + end +end diff --git a/app/graphql/types/base_interface.rb b/app/graphql/types/base_interface.rb new file mode 100644 index 00000000000..69e72dc5808 --- /dev/null +++ b/app/graphql/types/base_interface.rb @@ -0,0 +1,5 @@ +module Types + module BaseInterface + include GraphQL::Schema::Interface + end +end diff --git a/app/graphql/types/base_object.rb b/app/graphql/types/base_object.rb new file mode 100644 index 00000000000..e033ef96ce9 --- /dev/null +++ b/app/graphql/types/base_object.rb @@ -0,0 +1,7 @@ +module Types + class BaseObject < GraphQL::Schema::Object + prepend Gitlab::Graphql::Present + + field_class Types::BaseField + end +end diff --git a/app/graphql/types/base_scalar.rb b/app/graphql/types/base_scalar.rb new file mode 100644 index 00000000000..c0aa38be239 --- /dev/null +++ b/app/graphql/types/base_scalar.rb @@ -0,0 +1,4 @@ +module Types + class BaseScalar < GraphQL::Schema::Scalar + end +end diff --git a/app/graphql/types/base_union.rb b/app/graphql/types/base_union.rb new file mode 100644 index 00000000000..36337fc6ee5 --- /dev/null +++ b/app/graphql/types/base_union.rb @@ -0,0 +1,4 @@ +module Types + class BaseUnion < GraphQL::Schema::Union + end +end diff --git a/app/graphql/types/merge_request_type.rb b/app/graphql/types/merge_request_type.rb new file mode 100644 index 00000000000..d5d24952984 --- /dev/null +++ b/app/graphql/types/merge_request_type.rb @@ -0,0 +1,47 @@ +module Types + class MergeRequestType < BaseObject + present_using MergeRequestPresenter + + graphql_name 'MergeRequest' + + field :id, GraphQL::ID_TYPE, null: false + field :iid, GraphQL::ID_TYPE, null: false + field :title, GraphQL::STRING_TYPE, null: false + field :description, GraphQL::STRING_TYPE, null: true + field :state, GraphQL::STRING_TYPE, null: true + field :created_at, Types::TimeType, null: false + field :updated_at, Types::TimeType, null: false + field :source_project, Types::ProjectType, null: true + field :target_project, Types::ProjectType, null: false + # Alias for target_project + field :project, Types::ProjectType, null: false + field :project_id, GraphQL::INT_TYPE, null: false, method: :target_project_id + field :source_project_id, GraphQL::INT_TYPE, null: true + field :target_project_id, GraphQL::INT_TYPE, null: false + field :source_branch, GraphQL::STRING_TYPE, null: false + field :target_branch, GraphQL::STRING_TYPE, null: false + field :work_in_progress, GraphQL::BOOLEAN_TYPE, method: :work_in_progress?, null: false + field :merge_when_pipeline_succeeds, GraphQL::BOOLEAN_TYPE, null: true + field :diff_head_sha, GraphQL::STRING_TYPE, null: true + field :merge_commit_sha, GraphQL::STRING_TYPE, null: true + field :user_notes_count, GraphQL::INT_TYPE, null: true + field :should_remove_source_branch, GraphQL::BOOLEAN_TYPE, method: :should_remove_source_branch?, null: true + field :force_remove_source_branch, GraphQL::BOOLEAN_TYPE, method: :force_remove_source_branch?, null: true + field :merge_status, GraphQL::STRING_TYPE, null: true + field :in_progress_merge_commit_sha, GraphQL::STRING_TYPE, null: true + field :merge_error, GraphQL::STRING_TYPE, null: true + field :allow_collaboration, GraphQL::BOOLEAN_TYPE, null: true + field :should_be_rebased, GraphQL::BOOLEAN_TYPE, method: :should_be_rebased?, null: false + field :rebase_commit_sha, GraphQL::STRING_TYPE, null: true + field :rebase_in_progress, GraphQL::BOOLEAN_TYPE, method: :rebase_in_progress?, null: false + field :diff_head_sha, GraphQL::STRING_TYPE, null: true + field :merge_commit_message, GraphQL::STRING_TYPE, null: true + field :merge_ongoing, GraphQL::BOOLEAN_TYPE, method: :merge_ongoing?, null: false + field :source_branch_exists, GraphQL::BOOLEAN_TYPE, method: :source_branch_exists?, null: false + field :mergeable_discussions_state, GraphQL::BOOLEAN_TYPE, null: true + field :web_url, GraphQL::STRING_TYPE, null: true + field :upvotes, GraphQL::INT_TYPE, null: false + field :downvotes, GraphQL::INT_TYPE, null: false + field :subscribed, GraphQL::BOOLEAN_TYPE, method: :subscribed?, null: false + end +end diff --git a/app/graphql/types/mutation_type.rb b/app/graphql/types/mutation_type.rb new file mode 100644 index 00000000000..06ed91c1658 --- /dev/null +++ b/app/graphql/types/mutation_type.rb @@ -0,0 +1,7 @@ +module Types + class MutationType < BaseObject + graphql_name "Mutation" + + # TODO: Add Mutations as fields + end +end diff --git a/app/graphql/types/project_type.rb b/app/graphql/types/project_type.rb new file mode 100644 index 00000000000..9e885d5845a --- /dev/null +++ b/app/graphql/types/project_type.rb @@ -0,0 +1,65 @@ +module Types + class ProjectType < BaseObject + graphql_name 'Project' + + field :id, GraphQL::ID_TYPE, null: false + + field :full_path, GraphQL::ID_TYPE, null: false + field :path, GraphQL::STRING_TYPE, null: false + + field :name_with_namespace, GraphQL::STRING_TYPE, null: false + field :name, GraphQL::STRING_TYPE, null: false + + field :description, GraphQL::STRING_TYPE, null: true + + field :default_branch, GraphQL::STRING_TYPE, null: true + field :tag_list, GraphQL::STRING_TYPE, null: true + + field :ssh_url_to_repo, GraphQL::STRING_TYPE, null: true + field :http_url_to_repo, GraphQL::STRING_TYPE, null: true + field :web_url, GraphQL::STRING_TYPE, null: true + + field :star_count, GraphQL::INT_TYPE, null: false + field :forks_count, GraphQL::INT_TYPE, null: false + + field :created_at, Types::TimeType, null: true + field :last_activity_at, Types::TimeType, null: true + + field :archived, GraphQL::BOOLEAN_TYPE, null: true + + field :visibility, GraphQL::STRING_TYPE, null: true + + field :container_registry_enabled, GraphQL::BOOLEAN_TYPE, null: true + field :shared_runners_enabled, GraphQL::BOOLEAN_TYPE, null: true + field :lfs_enabled, GraphQL::BOOLEAN_TYPE, null: true + field :merge_requests_ff_only_enabled, GraphQL::BOOLEAN_TYPE, null: true + + field :avatar_url, GraphQL::STRING_TYPE, null: true, resolve: -> (project, args, ctx) do + project.avatar_url(only_path: false) + end + + %i[issues merge_requests wiki snippets].each do |feature| + field "#{feature}_enabled", GraphQL::BOOLEAN_TYPE, null: true, resolve: -> (project, args, ctx) do + project.feature_available?(feature, ctx[:current_user]) + end + end + + field :jobs_enabled, GraphQL::BOOLEAN_TYPE, null: true, resolve: -> (project, args, ctx) do + project.feature_available?(:builds, ctx[:current_user]) + end + + field :public_jobs, GraphQL::BOOLEAN_TYPE, method: :public_builds, null: true + + field :open_issues_count, GraphQL::INT_TYPE, null: true, resolve: -> (project, args, ctx) do + project.open_issues_count if project.feature_available?(:issues, ctx[:current_user]) + end + + field :import_status, GraphQL::STRING_TYPE, null: true + field :ci_config_path, GraphQL::STRING_TYPE, null: true + + field :only_allow_merge_if_pipeline_succeeds, GraphQL::BOOLEAN_TYPE, null: true + field :request_access_enabled, GraphQL::BOOLEAN_TYPE, null: true + field :only_allow_merge_if_all_discussions_are_resolved, GraphQL::BOOLEAN_TYPE, null: true + field :printing_merge_request_link_enabled, GraphQL::BOOLEAN_TYPE, null: true + end +end diff --git a/app/graphql/types/query_type.rb b/app/graphql/types/query_type.rb new file mode 100644 index 00000000000..be79c78bf67 --- /dev/null +++ b/app/graphql/types/query_type.rb @@ -0,0 +1,21 @@ +module Types + class QueryType < BaseObject + graphql_name 'Query' + + field :project, Types::ProjectType, + null: true, + resolver: Resolvers::ProjectResolver, + description: "Find a project" do + authorize :read_project + end + + field :merge_request, Types::MergeRequestType, + null: true, + resolver: Resolvers::MergeRequestResolver, + description: "Find a merge request" do + authorize :read_merge_request + end + + field :echo, GraphQL::STRING_TYPE, null: false, function: Functions::Echo.new + end +end diff --git a/app/graphql/types/time_type.rb b/app/graphql/types/time_type.rb new file mode 100644 index 00000000000..2333d82ad1e --- /dev/null +++ b/app/graphql/types/time_type.rb @@ -0,0 +1,14 @@ +module Types + class TimeType < BaseScalar + graphql_name 'Time' + description 'Time represented in ISO 8601' + + def self.coerce_input(value, ctx) + Time.parse(value) + end + + def self.coerce_result(value, ctx) + value.iso8601 + end + end +end diff --git a/app/helpers/merge_requests_helper.rb b/app/helpers/merge_requests_helper.rb index 74251c260f0..5ff06b3e0fc 100644 --- a/app/helpers/merge_requests_helper.rb +++ b/app/helpers/merge_requests_helper.rb @@ -126,8 +126,8 @@ module MergeRequestsHelper link_to(url[merge_request.project, merge_request], data: data_attrs, &block) end - def allow_maintainer_push_unavailable_reason(merge_request) - return if merge_request.can_allow_maintainer_to_push?(current_user) + def allow_collaboration_unavailable_reason(merge_request) + return if merge_request.can_allow_collaboration?(current_user) minimum_visibility = [merge_request.target_project.visibility_level, merge_request.source_project.visibility_level].min diff --git a/app/helpers/projects_helper.rb b/app/helpers/projects_helper.rb index 7b8cf13375b..cdbb572f80a 100644 --- a/app/helpers/projects_helper.rb +++ b/app/helpers/projects_helper.rb @@ -412,7 +412,10 @@ module ProjectsHelper exports_path = File.join(Settings.shared['path'], 'tmp/project_exports') filtered_message = message.strip.gsub(exports_path, "[REPO EXPORT PATH]") - disk_path = Gitlab.config.repositories.storages[project.repository_storage].legacy_disk_path + disk_path = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + Gitlab.config.repositories.storages[project.repository_storage].legacy_disk_path + end + filtered_message.gsub(disk_path.chomp('/'), "[REPOS PATH]") end diff --git a/app/helpers/workhorse_helper.rb b/app/helpers/workhorse_helper.rb index 9f78b80c71d..a82271ce0ee 100644 --- a/app/helpers/workhorse_helper.rb +++ b/app/helpers/workhorse_helper.rb @@ -6,7 +6,7 @@ module WorkhorseHelper headers.store(*Gitlab::Workhorse.send_git_blob(repository, blob)) headers['Content-Disposition'] = 'inline' headers['Content-Type'] = safe_content_type(blob) - head :ok # 'render nothing: true' messes up the Content-Type + render plain: "" end # Send a Git diff through Workhorse diff --git a/app/models/application_setting/term.rb b/app/models/application_setting/term.rb index e8ce0ccbb71..3b1dfe7e4ef 100644 --- a/app/models/application_setting/term.rb +++ b/app/models/application_setting/term.rb @@ -1,6 +1,7 @@ class ApplicationSetting class Term < ActiveRecord::Base include CacheMarkdownField + has_many :term_agreements validates :terms, presence: true @@ -9,5 +10,10 @@ class ApplicationSetting def self.latest order(:id).last end + + def accepted_by_user?(user) + user.accepted_term_id == id || + term_agreements.accepted.where(user: user).exists? + end end end diff --git a/app/models/ci/build.rb b/app/models/ci/build.rb index 75fd55a8f7b..2d675726939 100644 --- a/app/models/ci/build.rb +++ b/app/models/ci/build.rb @@ -55,12 +55,18 @@ module Ci where('(artifacts_file IS NOT NULL AND artifacts_file <> ?) OR EXISTS (?)', '', Ci::JobArtifact.select(1).where('ci_builds.id = ci_job_artifacts.job_id').archive) end + + scope :without_archived_trace, ->() do + where('NOT EXISTS (?)', Ci::JobArtifact.select(1).where('ci_builds.id = ci_job_artifacts.job_id').trace) + end + scope :with_artifacts_stored_locally, -> { with_artifacts_archive.where(artifacts_file_store: [nil, LegacyArtifactUploader::Store::LOCAL]) } scope :with_artifacts_not_expired, ->() { with_artifacts_archive.where('artifacts_expire_at IS NULL OR artifacts_expire_at > ?', Time.now) } scope :with_expired_artifacts, ->() { with_artifacts_archive.where('artifacts_expire_at < ?', Time.now) } scope :last_month, ->() { where('created_at > ?', Date.today - 1.month) } scope :manual_actions, ->() { where(when: :manual, status: COMPLETED_STATUSES + [:manual]) } scope :ref_protected, -> { where(protected: true) } + scope :with_live_trace, -> { where('EXISTS (?)', Ci::BuildTraceChunk.where('ci_builds.id = ci_build_trace_chunks.build_id').select(1)) } scope :matches_tag_ids, -> (tag_ids) do matcher = ::ActsAsTaggableOn::Tagging @@ -144,6 +150,7 @@ module Ci after_transition any => [:success] do |build| build.run_after_commit do BuildSuccessWorker.perform_async(id) + PagesWorker.perform_async(:deploy, id) if build.pages_generator? end end @@ -183,6 +190,11 @@ module Ci pipeline.manual_actions.where.not(name: name) end + def pages_generator? + Gitlab.config.pages.enabled && + self.name == 'pages' + end + def playable? action? && (manual? || retryable?) end @@ -402,8 +414,6 @@ module Ci build_data = Gitlab::DataBuilder::Build.build(self) project.execute_hooks(build_data.dup, :job_hooks) project.execute_services(build_data.dup, :job_hooks) - PagesService.new(build_data).execute - project.running_or_pending_build_count(force: true) end def browsable_artifacts? diff --git a/app/models/ci/group.rb b/app/models/ci/group.rb index 87898b086c6..9c1046e8715 100644 --- a/app/models/ci/group.rb +++ b/app/models/ci/group.rb @@ -31,6 +31,14 @@ module Ci end end + def self.fabricate(stage) + stage.statuses.ordered.latest + .sort_by(&:sortable_name).group_by(&:group_name) + .map do |group_name, grouped_statuses| + self.new(stage, name: group_name, jobs: grouped_statuses) + end + end + private def commit_statuses diff --git a/app/models/ci/legacy_stage.rb b/app/models/ci/legacy_stage.rb index 9b536af672b..ce691875e42 100644 --- a/app/models/ci/legacy_stage.rb +++ b/app/models/ci/legacy_stage.rb @@ -16,11 +16,7 @@ module Ci end def groups - @groups ||= statuses.ordered.latest - .sort_by(&:sortable_name).group_by(&:group_name) - .map do |group_name, grouped_statuses| - Ci::Group.new(self, name: group_name, jobs: grouped_statuses) - end + @groups ||= Ci::Group.fabricate(self) end def to_param diff --git a/app/models/ci/pipeline.rb b/app/models/ci/pipeline.rb index 53af87a271a..eecd86349e4 100644 --- a/app/models/ci/pipeline.rb +++ b/app/models/ci/pipeline.rb @@ -7,13 +7,18 @@ module Ci include Presentable include Gitlab::OptimisticLocking include Gitlab::Utils::StrongMemoize + include AtomicInternalId belongs_to :project, inverse_of: :pipelines belongs_to :user belongs_to :auto_canceled_by, class_name: 'Ci::Pipeline' belongs_to :pipeline_schedule, class_name: 'Ci::PipelineSchedule' - has_many :stages + has_internal_id :iid, scope: :project, presence: false, init: ->(s) do + s&.project&.pipelines&.maximum(:iid) || s&.project&.pipelines&.count + end + + has_many :stages, -> { order(position: :asc) }, inverse_of: :pipeline has_many :statuses, class_name: 'CommitStatus', foreign_key: :commit_id, inverse_of: :pipeline has_many :builds, foreign_key: :commit_id, inverse_of: :pipeline has_many :trigger_requests, dependent: :destroy, foreign_key: :commit_id # rubocop:disable Cop/ActiveRecordDependent @@ -249,6 +254,20 @@ module Ci stage unless stage.statuses_count.zero? end + ## + # TODO We do not completely switch to persisted stages because of + # race conditions with setting statuses gitlab-ce#23257. + # + def ordered_stages + return legacy_stages unless complete? + + if Feature.enabled?('ci_pipeline_persisted_stages') + stages + else + legacy_stages + end + end + def legacy_stages # TODO, this needs refactoring, see gitlab-ce#26481. @@ -411,7 +430,7 @@ module Ci def number_of_warnings BatchLoader.for(id).batch(default_value: 0) do |pipeline_ids, loader| - Build.where(commit_id: pipeline_ids) + ::Ci::Build.where(commit_id: pipeline_ids) .latest .failed_but_allowed .group(:commit_id) @@ -503,7 +522,8 @@ module Ci def update_status retry_optimistic_lock(self) do - case latest_builds_status + case latest_builds_status.to_s + when 'created' then nil when 'pending' then enqueue when 'running' then run when 'success' then succeed @@ -511,6 +531,9 @@ module Ci when 'canceled' then cancel when 'skipped' then skip when 'manual' then block + else + raise HasStatus::UnknownStatusError, + "Unknown status `#{latest_builds_status}`" end end end @@ -531,6 +554,7 @@ module Ci def predefined_variables Gitlab::Ci::Variables::Collection.new + .append(key: 'CI_PIPELINE_IID', value: iid.to_s) .append(key: 'CI_CONFIG_PATH', value: ci_yaml_file_path) .append(key: 'CI_PIPELINE_SOURCE', value: source.to_s) .append(key: 'CI_COMMIT_MESSAGE', value: git_commit_message) diff --git a/app/models/ci/runner.rb b/app/models/ci/runner.rb index 57edd6a4956..8c9aacca8de 100644 --- a/app/models/ci/runner.rb +++ b/app/models/ci/runner.rb @@ -219,10 +219,8 @@ module Ci cache_attributes(values) - if persist_cached_data? - self.assign_attributes(values) - self.save if self.changed? - end + # We save data without validation, it will always change due to `contacted_at` + self.update_columns(values) if persist_cached_data? end def pick_build!(build) diff --git a/app/models/ci/stage.rb b/app/models/ci/stage.rb index 5a1eeb966aa..ea07f37e6c1 100644 --- a/app/models/ci/stage.rb +++ b/app/models/ci/stage.rb @@ -68,16 +68,44 @@ module Ci def update_status retry_optimistic_lock(self) do case statuses.latest.status + when 'created' then nil when 'pending' then enqueue when 'running' then run when 'success' then succeed when 'failed' then drop when 'canceled' then cancel when 'manual' then block - when 'skipped' then skip - else skip + when 'skipped', nil then skip + else + raise HasStatus::UnknownStatusError, + "Unknown status `#{statuses.latest.status}`" end end end + + def groups + @groups ||= Ci::Group.fabricate(self) + end + + def has_warnings? + number_of_warnings.positive? + end + + def number_of_warnings + BatchLoader.for(id).batch(default_value: 0) do |stage_ids, loader| + ::Ci::Build.where(stage_id: stage_ids) + .latest + .failed_but_allowed + .group(:stage_id) + .count + .each { |id, amount| loader.call(id, amount) } + end + end + + def detailed_status(current_user) + Gitlab::Ci::Status::Stage::Factory + .new(self, current_user) + .fabricate! + end end end diff --git a/app/models/concerns/atomic_internal_id.rb b/app/models/concerns/atomic_internal_id.rb index 22f516a172f..164c704260e 100644 --- a/app/models/concerns/atomic_internal_id.rb +++ b/app/models/concerns/atomic_internal_id.rb @@ -25,9 +25,13 @@ module AtomicInternalId extend ActiveSupport::Concern module ClassMethods - def has_internal_id(column, scope:, init:) # rubocop:disable Naming/PredicateName - before_validation(on: :create) do + def has_internal_id(column, scope:, init:, presence: true) # rubocop:disable Naming/PredicateName + before_validation :"ensure_#{scope}_#{column}!", on: :create + validates column, presence: presence + + define_method("ensure_#{scope}_#{column}!") do scope_value = association(scope).reader + if read_attribute(column).blank? && scope_value scope_attrs = { scope_value.class.table_name.singularize.to_sym => scope_value } usage = self.class.table_name.to_sym @@ -35,13 +39,9 @@ module AtomicInternalId new_iid = InternalId.generate_next(self, scope_attrs, usage, init) write_attribute(column, new_iid) end - end - validates column, presence: true, numericality: true + read_attribute(column) + end end end - - def to_param - iid.to_s - end end diff --git a/app/models/concerns/avatarable.rb b/app/models/concerns/avatarable.rb index 13246a774e3..095897b08e3 100644 --- a/app/models/concerns/avatarable.rb +++ b/app/models/concerns/avatarable.rb @@ -4,11 +4,14 @@ module Avatarable included do prepend ShadowMethods include ObjectStorage::BackgroundMove + include Gitlab::Utils::StrongMemoize validate :avatar_type, if: ->(user) { user.avatar.present? && user.avatar_changed? } validates :avatar, file_size: { maximum: 200.kilobytes.to_i } mount_uploader :avatar, AvatarUploader + + after_initialize :add_avatar_to_batch end module ShadowMethods @@ -18,6 +21,17 @@ module Avatarable avatar_path(only_path: args.fetch(:only_path, true)) || super end + + def retrieve_upload(identifier, paths) + upload = retrieve_upload_from_batch(identifier) + + # This fallback is needed when deleting an upload, because we may have + # already been removed from the DB. We have to check an explicit `#nil?` + # because it's a BatchLoader instance. + upload = super if upload.nil? + + upload + end end def avatar_type @@ -52,4 +66,37 @@ module Avatarable url_base + avatar.local_url end + + # Path that is persisted in the tracking Upload model. Used to fetch the + # upload from the model. + def upload_paths(identifier) + avatar_mounter.blank_uploader.store_dirs.map { |store, path| File.join(path, identifier) } + end + + private + + def retrieve_upload_from_batch(identifier) + BatchLoader.for(identifier: identifier, model: self).batch(key: self.class) do |upload_params, loader, args| + model_class = args[:key] + paths = upload_params.flat_map do |params| + params[:model].upload_paths(params[:identifier]) + end + + Upload.where(uploader: AvatarUploader, path: paths).find_each do |upload| + model = model_class.instantiate('id' => upload.model_id) + + loader.call({ model: model, identifier: File.basename(upload.path) }, upload) + end + end + end + + def add_avatar_to_batch + return unless avatar_mounter + + avatar_mounter.read_identifiers.each { |identifier| retrieve_upload_from_batch(identifier) } + end + + def avatar_mounter + strong_memoize(:avatar_mounter) { _mounter(:avatar) } + end end diff --git a/app/models/concerns/has_status.rb b/app/models/concerns/has_status.rb index 7c3ed96bc28..72c236a0fc7 100644 --- a/app/models/concerns/has_status.rb +++ b/app/models/concerns/has_status.rb @@ -11,6 +11,8 @@ module HasStatus STATUSES_ENUM = { created: 0, pending: 1, running: 2, success: 3, failed: 4, canceled: 5, skipped: 6, manual: 7 }.freeze + UnknownStatusError = Class.new(StandardError) + class_methods do def status_sql scope_relevant = respond_to?(:exclude_ignored) ? exclude_ignored : all diff --git a/app/models/concerns/iid_routes.rb b/app/models/concerns/iid_routes.rb new file mode 100644 index 00000000000..246748cf52c --- /dev/null +++ b/app/models/concerns/iid_routes.rb @@ -0,0 +1,9 @@ +module IidRoutes + ## + # This automagically enforces all related routes to use `iid` instead of `id` + # If you want to use `iid` for some routes and `id` for other routes, this module should not to be included, + # instead you should define `iid` or `id` explictly at each route generators. e.g. pipeline_path(project.id, pipeline.iid) + def to_param + iid.to_s + end +end diff --git a/app/models/concerns/protected_ref_access.rb b/app/models/concerns/protected_ref_access.rb index bfda5b1678b..e3a7f2d5498 100644 --- a/app/models/concerns/protected_ref_access.rb +++ b/app/models/concerns/protected_ref_access.rb @@ -8,8 +8,8 @@ module ProtectedRefAccess ].freeze HUMAN_ACCESS_LEVELS = { - Gitlab::Access::MASTER => "Masters".freeze, - Gitlab::Access::DEVELOPER => "Developers + Masters".freeze, + Gitlab::Access::MASTER => "Maintainers".freeze, + Gitlab::Access::DEVELOPER => "Developers + Maintainers".freeze, Gitlab::Access::NO_ACCESS => "No one".freeze }.freeze diff --git a/app/models/concerns/with_uploads.rb b/app/models/concerns/with_uploads.rb index e7cfffb775b..4245d083a49 100644 --- a/app/models/concerns/with_uploads.rb +++ b/app/models/concerns/with_uploads.rb @@ -36,4 +36,8 @@ module WithUploads upload.destroy end end + + def retrieve_upload(_identifier, paths) + uploads.find_by(path: paths) + end end diff --git a/app/models/deployment.rb b/app/models/deployment.rb index 254764eefde..ac86e9e8de0 100644 --- a/app/models/deployment.rb +++ b/app/models/deployment.rb @@ -1,5 +1,6 @@ class Deployment < ActiveRecord::Base include AtomicInternalId + include IidRoutes belongs_to :project, required: true belongs_to :environment, required: true diff --git a/app/models/internal_id.rb b/app/models/internal_id.rb index f7f930e86ed..f50f28deffe 100644 --- a/app/models/internal_id.rb +++ b/app/models/internal_id.rb @@ -14,7 +14,7 @@ class InternalId < ActiveRecord::Base belongs_to :project belongs_to :namespace - enum usage: { issues: 0, merge_requests: 1, deployments: 2, milestones: 3, epics: 4 } + enum usage: { issues: 0, merge_requests: 1, deployments: 2, milestones: 3, epics: 4, ci_pipelines: 5 } validates :usage, presence: true diff --git a/app/models/issue.rb b/app/models/issue.rb index 41a290f34b4..d136700836d 100644 --- a/app/models/issue.rb +++ b/app/models/issue.rb @@ -2,6 +2,7 @@ require 'carrierwave/orm/activerecord' class Issue < ActiveRecord::Base include AtomicInternalId + include IidRoutes include Issuable include Noteable include Referable diff --git a/app/models/merge_request.rb b/app/models/merge_request.rb index 79fc155fd3c..535a2c362f2 100644 --- a/app/models/merge_request.rb +++ b/app/models/merge_request.rb @@ -1,5 +1,6 @@ class MergeRequest < ActiveRecord::Base include AtomicInternalId + include IidRoutes include Issuable include Noteable include Referable @@ -1124,21 +1125,21 @@ class MergeRequest < ActiveRecord::Base project.merge_requests.merged.where(author_id: author_id).empty? end - def allow_maintainer_to_push - maintainer_push_possible? && super + def allow_collaboration + collaborative_push_possible? && super end - alias_method :allow_maintainer_to_push?, :allow_maintainer_to_push + alias_method :allow_collaboration?, :allow_collaboration - def maintainer_push_possible? + def collaborative_push_possible? source_project.present? && for_fork? && target_project.visibility_level > Gitlab::VisibilityLevel::PRIVATE && source_project.visibility_level > Gitlab::VisibilityLevel::PRIVATE && !ProtectedBranch.protected?(source_project, source_branch) end - def can_allow_maintainer_to_push?(user) - maintainer_push_possible? && + def can_allow_collaboration?(user) + collaborative_push_possible? && Ability.allowed?(user, :push_code, source_project) end diff --git a/app/models/milestone.rb b/app/models/milestone.rb index d14e3a4ded5..d05dcfd083a 100644 --- a/app/models/milestone.rb +++ b/app/models/milestone.rb @@ -9,6 +9,7 @@ class Milestone < ActiveRecord::Base include CacheMarkdownField include AtomicInternalId + include IidRoutes include Sortable include Referable include StripAttribute diff --git a/app/models/note.rb b/app/models/note.rb index 02f7a9b1e4f..41c04ae0571 100644 --- a/app/models/note.rb +++ b/app/models/note.rb @@ -435,6 +435,10 @@ class Note < ActiveRecord::Base super.merge(noteable: noteable) end + def retrieve_upload(_identifier, paths) + Upload.find_by(model: self, path: paths) + end + private def keep_around_commit diff --git a/app/models/personal_snippet.rb b/app/models/personal_snippet.rb index 82c1c4de3a0..355624fd552 100644 --- a/app/models/personal_snippet.rb +++ b/app/models/personal_snippet.rb @@ -1,2 +1,3 @@ class PersonalSnippet < Snippet + include WithUploads end diff --git a/app/models/project.rb b/app/models/project.rb index 32298fc7f5c..562198e2369 100644 --- a/app/models/project.rb +++ b/app/models/project.rb @@ -228,6 +228,7 @@ class Project < ActiveRecord::Base has_many :commit_statuses has_many :pipelines, class_name: 'Ci::Pipeline', inverse_of: :project + has_many :stages, class_name: 'Ci::Stage', inverse_of: :project # Ci::Build objects store data on the file system such as artifact files and # build traces. Currently there's no efficient way of removing this data in @@ -1425,8 +1426,14 @@ class Project < ActiveRecord::Base Ci::Runner.from("(#{union.to_sql}) ci_runners") end + def active_runners + strong_memoize(:active_runners) do + all_runners.active + end + end + def any_runners?(&block) - all_runners.active.any?(&block) + active_runners.any?(&block) end def valid_runners_token?(token) @@ -1649,12 +1656,6 @@ class Project < ActiveRecord::Base import_state.update_column(:jid, nil) end - def running_or_pending_build_count(force: false) - Rails.cache.fetch(['projects', id, 'running_or_pending_build_count'], force: force) do - builds.running_or_pending.count(:all) - end - end - # Lazy loading of the `pipeline_status` attribute def pipeline_status @pipeline_status ||= Gitlab::Cache::Ci::ProjectPipelineStatus.load_for_project(self) @@ -1974,18 +1975,18 @@ class Project < ActiveRecord::Base .limit(1) .select(1) source_of_merge_requests.opened - .where(allow_maintainer_to_push: true) + .where(allow_collaboration: true) .where('EXISTS (?)', developer_access_exists) end - def branch_allows_maintainer_push?(user, branch_name) + def branch_allows_collaboration?(user, branch_name) return false unless user cache_key = "user:#{user.id}:#{branch_name}:branch_allows_push" - memoized_results = strong_memoize(:branch_allows_maintainer_push) do + memoized_results = strong_memoize(:branch_allows_collaboration) do Hash.new do |result, cache_key| - result[cache_key] = fetch_branch_allows_maintainer_push?(user, branch_name) + result[cache_key] = fetch_branch_allows_collaboration?(user, branch_name) end end @@ -2127,18 +2128,18 @@ class Project < ActiveRecord::Base raise ex end - def fetch_branch_allows_maintainer_push?(user, branch_name) + def fetch_branch_allows_collaboration?(user, branch_name) check_access = -> do next false if empty_repo? merge_request = source_of_merge_requests.opened - .where(allow_maintainer_to_push: true) + .where(allow_collaboration: true) .find_by(source_branch: branch_name) merge_request&.can_be_merged_by?(user) end if RequestStore.active? - RequestStore.fetch("project-#{id}:branch-#{branch_name}:user-#{user.id}:branch_allows_maintainer_push") do + RequestStore.fetch("project-#{id}:branch-#{branch_name}:user-#{user.id}:branch_allows_collaboration") do check_access.call end else diff --git a/app/models/protected_branch.rb b/app/models/protected_branch.rb index cb361a66591..dff99cfca35 100644 --- a/app/models/protected_branch.rb +++ b/app/models/protected_branch.rb @@ -5,7 +5,7 @@ class ProtectedBranch < ActiveRecord::Base protected_ref_access_levels :merge, :push def self.protected_ref_accessible_to?(ref, user, project:, action:, protected_refs: nil) - # Masters, owners and admins are allowed to create the default branch + # Maintainers, owners and admins are allowed to create the default branch if default_branch_protected? && project.empty_repo? return true if user.admin? || project.team.max_member_access(user.id) > Gitlab::Access::DEVELOPER end diff --git a/app/models/repository.rb b/app/models/repository.rb index 82cf47ba04e..e4202505634 100644 --- a/app/models/repository.rb +++ b/app/models/repository.rb @@ -270,6 +270,16 @@ class Repository end end + def archive_metadata(ref, storage_path, format = "tar.gz", append_sha:) + raw_repository.archive_metadata( + ref, + storage_path, + project.path, + format, + append_sha: append_sha + ) + end + def expire_tags_cache expire_method_caches(%i(tag_names tag_count)) @tags = nil @@ -946,6 +956,10 @@ class Repository blob_data_at(sha, path) end + def lfsconfig_for(sha) + blob_data_at(sha, '.lfsconfig') + end + def fetch_ref(source_repository, source_ref:, target_ref:) raw_repository.fetch_ref(source_repository.raw_repository, source_ref: source_ref, target_ref: target_ref) end diff --git a/app/models/term_agreement.rb b/app/models/term_agreement.rb index 8458a231bbd..c317bd0c90b 100644 --- a/app/models/term_agreement.rb +++ b/app/models/term_agreement.rb @@ -2,5 +2,7 @@ class TermAgreement < ActiveRecord::Base belongs_to :term, class_name: 'ApplicationSetting::Term' belongs_to :user + scope :accepted, -> { where(accepted: true) } + validates :user, :term, presence: true end diff --git a/app/policies/ci/build_policy.rb b/app/policies/ci/build_policy.rb index 8b65758f3e8..1c0cc7425ec 100644 --- a/app/policies/ci/build_policy.rb +++ b/app/policies/ci/build_policy.rb @@ -14,8 +14,8 @@ module Ci @subject.triggered_by?(@user) end - condition(:branch_allows_maintainer_push) do - @subject.project.branch_allows_maintainer_push?(@user, @subject.ref) + condition(:branch_allows_collaboration) do + @subject.project.branch_allows_collaboration?(@user, @subject.ref) end rule { protected_ref }.policy do @@ -25,7 +25,7 @@ module Ci rule { can?(:admin_build) | (can?(:update_build) & owner_of_job) }.enable :erase_build - rule { can?(:public_access) & branch_allows_maintainer_push }.policy do + rule { can?(:public_access) & branch_allows_collaboration }.policy do enable :update_build enable :update_commit_status end diff --git a/app/policies/ci/pipeline_policy.rb b/app/policies/ci/pipeline_policy.rb index 540e4235299..b81329d0625 100644 --- a/app/policies/ci/pipeline_policy.rb +++ b/app/policies/ci/pipeline_policy.rb @@ -4,13 +4,13 @@ module Ci condition(:protected_ref) { ref_protected?(@user, @subject.project, @subject.tag?, @subject.ref) } - condition(:branch_allows_maintainer_push) do - @subject.project.branch_allows_maintainer_push?(@user, @subject.ref) + condition(:branch_allows_collaboration) do + @subject.project.branch_allows_collaboration?(@user, @subject.ref) end rule { protected_ref }.prevent :update_pipeline - rule { can?(:public_access) & branch_allows_maintainer_push }.policy do + rule { can?(:public_access) & branch_allows_collaboration }.policy do enable :update_pipeline end diff --git a/app/policies/project_policy.rb b/app/policies/project_policy.rb index 99a0d7118f2..8ea5435d740 100644 --- a/app/policies/project_policy.rb +++ b/app/policies/project_policy.rb @@ -45,7 +45,7 @@ class ProjectPolicy < BasePolicy desc "User has developer access" condition(:developer) { team_access_level >= Gitlab::Access::DEVELOPER } - desc "User has master access" + desc "User has maintainer access" condition(:master) { team_access_level >= Gitlab::Access::MASTER } desc "Project is public" diff --git a/app/presenters/merge_request_presenter.rb b/app/presenters/merge_request_presenter.rb index ad839d9840a..8d466c33510 100644 --- a/app/presenters/merge_request_presenter.rb +++ b/app/presenters/merge_request_presenter.rb @@ -179,6 +179,25 @@ class MergeRequestPresenter < Gitlab::View::Presenter::Delegated .can_push_to_branch?(source_branch) end + def mergeable_discussions_state + # This avoids calling MergeRequest#mergeable_discussions_state without + # considering the state of the MR first. If a MR isn't mergeable, we can + # safely short-circuit it. + if merge_request.mergeable_state?(skip_ci_check: true, skip_discussions_check: true) + merge_request.mergeable_discussions_state? + else + false + end + end + + def web_url + Gitlab::UrlBuilder.build(merge_request) + end + + def subscribed? + merge_request.subscribed?(current_user, merge_request.target_project) + end + private def cached_can_be_reverted? diff --git a/app/serializers/merge_request_widget_entity.rb b/app/serializers/merge_request_widget_entity.rb index 141070aef45..8260c6c7b84 100644 --- a/app/serializers/merge_request_widget_entity.rb +++ b/app/serializers/merge_request_widget_entity.rb @@ -13,7 +13,7 @@ class MergeRequestWidgetEntity < IssuableEntity expose :squash expose :target_branch expose :target_project_id - expose :allow_maintainer_to_push + expose :allow_collaboration expose :should_be_rebased?, as: :should_be_rebased expose :ff_only_enabled do |merge_request| diff --git a/app/serializers/pipeline_details_entity.rb b/app/serializers/pipeline_details_entity.rb index 130968a44c1..8ba9cac53c4 100644 --- a/app/serializers/pipeline_details_entity.rb +++ b/app/serializers/pipeline_details_entity.rb @@ -1,6 +1,6 @@ class PipelineDetailsEntity < PipelineEntity expose :details do - expose :legacy_stages, as: :stages, using: StageEntity + expose :ordered_stages, as: :stages, using: StageEntity expose :artifacts, using: BuildArtifactEntity expose :manual_actions, using: BuildActionEntity end diff --git a/app/serializers/pipeline_serializer.rb b/app/serializers/pipeline_serializer.rb index 7181f8a6b04..17a022539bb 100644 --- a/app/serializers/pipeline_serializer.rb +++ b/app/serializers/pipeline_serializer.rb @@ -1,14 +1,11 @@ class PipelineSerializer < BaseSerializer include WithPagination - - InvalidResourceError = Class.new(StandardError) - entity PipelineDetailsEntity def represent(resource, opts = {}) if resource.is_a?(ActiveRecord::Relation) - resource = resource.preload([ + :stages, :retryable_builds, :cancelable_statuses, :trigger_requests, @@ -20,10 +17,14 @@ class PipelineSerializer < BaseSerializer end if paginated? - super(@paginator.paginate(resource), opts) - else - super(resource, opts) + resource = paginator.paginate(resource) end + + if opts.delete(:preload) + resource = Gitlab::Ci::Pipeline::Preloader.preload!(resource) + end + + super(resource, opts) end def represent_status(resource) @@ -36,7 +37,7 @@ class PipelineSerializer < BaseSerializer def represent_stages(resource) return {} unless resource.present? - data = represent(resource, { only: [{ details: [:stages] }] }) + data = represent(resource, { only: [{ details: [:stages] }], preload: true }) data.dig(:details, :stages) || [] end end diff --git a/app/services/application_settings/update_service.rb b/app/services/application_settings/update_service.rb index e70445cfb67..7bcb8f49d0d 100644 --- a/app/services/application_settings/update_service.rb +++ b/app/services/application_settings/update_service.rb @@ -1,5 +1,7 @@ module ApplicationSettings class UpdateService < ApplicationSettings::BaseService + attr_reader :params, :application_setting + def execute update_terms(@params.delete(:terms)) diff --git a/app/services/applications/create_service.rb b/app/services/applications/create_service.rb index 35d45f25a71..e67af929954 100644 --- a/app/services/applications/create_service.rb +++ b/app/services/applications/create_service.rb @@ -2,8 +2,7 @@ module Applications class CreateService def initialize(current_user, params) @current_user = current_user - @params = params - @ip_address = @params.delete(:ip_address) + @params = params.except(:ip_address) end def execute(request = nil) diff --git a/app/services/base_service.rb b/app/services/base_service.rb index 6883ba36c71..3519b7c5e7d 100644 --- a/app/services/base_service.rb +++ b/app/services/base_service.rb @@ -3,7 +3,7 @@ class BaseService attr_accessor :project, :current_user, :params - def initialize(project, user, params = {}) + def initialize(project, user = nil, params = {}) @project, @current_user, @params = project, user, params.dup end diff --git a/app/services/concerns/exclusive_lease_guard.rb b/app/services/concerns/exclusive_lease_guard.rb index 30be6accc32..f45436370c1 100644 --- a/app/services/concerns/exclusive_lease_guard.rb +++ b/app/services/concerns/exclusive_lease_guard.rb @@ -47,6 +47,6 @@ module ExclusiveLeaseGuard end def log_error(message, extra_args = {}) - logger.error(message) + Rails.logger.error(message) end end diff --git a/app/services/lfs/unlock_file_service.rb b/app/services/lfs/unlock_file_service.rb index 7eb89339a92..7e3edf21d54 100644 --- a/app/services/lfs/unlock_file_service.rb +++ b/app/services/lfs/unlock_file_service.rb @@ -24,7 +24,7 @@ module Lfs success(lock: lock, http_status: :ok) elsif forced - error(_('You must have master access to force delete a lock'), 403) + error(_('You must have maintainer access to force delete a lock'), 403) else error(_("%{lock_path} is locked by GitLab User %{lock_user_id}") % { lock_path: lock.path, lock_user_id: lock.user_id }, 403) end diff --git a/app/services/merge_requests/base_service.rb b/app/services/merge_requests/base_service.rb index 231ab76fde4..4c420b38258 100644 --- a/app/services/merge_requests/base_service.rb +++ b/app/services/merge_requests/base_service.rb @@ -38,8 +38,8 @@ module MergeRequests def filter_params(merge_request) super - unless merge_request.can_allow_maintainer_to_push?(current_user) - params.delete(:allow_maintainer_to_push) + unless merge_request.can_allow_collaboration?(current_user) + params.delete(:allow_collaboration) end end diff --git a/app/services/pages_service.rb b/app/services/pages_service.rb deleted file mode 100644 index 446eeb34d3b..00000000000 --- a/app/services/pages_service.rb +++ /dev/null @@ -1,15 +0,0 @@ -class PagesService - attr_reader :data - - def initialize(data) - @data = data - end - - def execute - return unless Settings.pages.enabled - return unless data[:build_name] == 'pages' - return unless data[:build_status] == 'success' - - PagesWorker.perform_async(:deploy, data[:build_id]) - end -end diff --git a/app/services/projects/autocomplete_service.rb b/app/services/projects/autocomplete_service.rb index 346971138b1..3e38a8a12d4 100644 --- a/app/services/projects/autocomplete_service.rb +++ b/app/services/projects/autocomplete_service.rb @@ -11,7 +11,7 @@ module Projects order: { due_date: :asc, title: :asc } } - finder_params[:group_ids] = [@project.group.id] if @project.group + finder_params[:group_ids] = @project.group.self_and_ancestors.select(:id) if @project.group MilestonesFinder.new(finder_params).execute.select([:iid, :title]) end diff --git a/app/services/projects/create_service.rb b/app/services/projects/create_service.rb index d16ecdb7b9b..a02a9052fb2 100644 --- a/app/services/projects/create_service.rb +++ b/app/services/projects/create_service.rb @@ -46,6 +46,9 @@ module Projects yield(@project) if block_given? + # If the block added errors, don't try to save the project + return @project if @project.errors.any? + @project.creator = current_user if forked_from_project_id @@ -63,6 +66,7 @@ module Projects message = "Unable to save #{e.record.type}: #{e.record.errors.full_messages.join(", ")} " fail(error: message) rescue => e + @project.errors.add(:base, e.message) if @project fail(error: e.message) end @@ -141,7 +145,6 @@ module Projects Rails.logger.error(log_message) if @project - @project.errors.add(:base, message) @project.mark_import_as_failed(message) if @project.persisted? && @project.import? end diff --git a/app/services/projects/import_service.rb b/app/services/projects/import_service.rb index 00080717600..1781a01cbd4 100644 --- a/app/services/projects/import_service.rb +++ b/app/services/projects/import_service.rb @@ -17,6 +17,8 @@ module Projects def execute add_repository_to_project + download_lfs_objects + import_data success @@ -37,7 +39,7 @@ module Projects # We should skip the repository for a GitHub import or GitLab project import, # because these importers fetch the project repositories for us. - return if has_importer? && importer_class.try(:imports_repository?) + return if importer_imports_repository? if unknown_url? # In this case, we only want to import issues, not a repository. @@ -73,6 +75,27 @@ module Projects end end + def download_lfs_objects + # In this case, we only want to import issues + return if unknown_url? + + # If it has its own repository importer, it has to implements its own lfs import download + return if importer_imports_repository? + + return unless project.lfs_enabled? + + oids_to_download = Projects::LfsPointers::LfsImportService.new(project).execute + download_service = Projects::LfsPointers::LfsDownloadService.new(project) + + oids_to_download.each do |oid, link| + download_service.execute(oid, link) + end + rescue => e + # Right now, to avoid aborting the importing process, we silently fail + # if any exception raises. + Rails.logger.error("The Lfs import process failed. #{e.message}") + end + def import_data return unless has_importer? @@ -98,5 +121,9 @@ module Projects def unknown_url? project.import_url == Project::UNKNOWN_IMPORT_URL end + + def importer_imports_repository? + has_importer? && importer_class.try(:imports_repository?) + end end end diff --git a/app/services/projects/lfs_pointers/lfs_download_link_list_service.rb b/app/services/projects/lfs_pointers/lfs_download_link_list_service.rb new file mode 100644 index 00000000000..d9fb74b090e --- /dev/null +++ b/app/services/projects/lfs_pointers/lfs_download_link_list_service.rb @@ -0,0 +1,93 @@ +# This service lists the download link from a remote source based on the +# oids provided +module Projects + module LfsPointers + class LfsDownloadLinkListService < BaseService + DOWNLOAD_ACTION = 'download'.freeze + + DownloadLinksError = Class.new(StandardError) + DownloadLinkNotFound = Class.new(StandardError) + + attr_reader :remote_uri + + def initialize(project, remote_uri: nil) + super(project) + + @remote_uri = remote_uri + end + + # This method accepts two parameters: + # - oids: hash of oids to query. The structure is { lfs_file_oid => lfs_file_size } + # + # Returns a hash with the structure { lfs_file_oids => download_link } + def execute(oids) + return {} unless project&.lfs_enabled? && remote_uri && oids.present? + + get_download_links(oids) + end + + private + + def get_download_links(oids) + response = Gitlab::HTTP.post(remote_uri, + body: request_body(oids), + headers: headers) + + raise DownloadLinksError, response.message unless response.success? + + parse_response_links(response['objects']) + end + + def parse_response_links(objects_response) + objects_response.each_with_object({}) do |entry, link_list| + begin + oid = entry['oid'] + link = entry.dig('actions', DOWNLOAD_ACTION, 'href') + + raise DownloadLinkNotFound unless link + + link_list[oid] = add_credentials(link) + rescue DownloadLinkNotFound, URI::InvalidURIError + Rails.logger.error("Link for Lfs Object with oid #{oid} not found or invalid.") + end + end + end + + def request_body(oids) + { + operation: DOWNLOAD_ACTION, + objects: oids.map { |oid, size| { oid: oid, size: size } } + }.to_json + end + + def headers + { + 'Accept' => LfsRequest::CONTENT_TYPE, + 'Content-Type' => LfsRequest::CONTENT_TYPE + }.freeze + end + + def add_credentials(link) + uri = URI.parse(link) + + if should_add_credentials?(uri) + uri.user = remote_uri.user + uri.password = remote_uri.password + end + + uri.to_s + end + + # The download link can be a local url or an object storage url + # If the download link has the some host as the import url then + # we add the same credentials because we may need them + def should_add_credentials?(link_uri) + url_credentials? && link_uri.host == remote_uri.host + end + + def url_credentials? + remote_uri.user.present? || remote_uri.password.present? + end + end + end +end diff --git a/app/services/projects/lfs_pointers/lfs_download_service.rb b/app/services/projects/lfs_pointers/lfs_download_service.rb new file mode 100644 index 00000000000..6ea43561d61 --- /dev/null +++ b/app/services/projects/lfs_pointers/lfs_download_service.rb @@ -0,0 +1,58 @@ +# This service downloads and links lfs objects from a remote URL +module Projects + module LfsPointers + class LfsDownloadService < BaseService + def execute(oid, url) + return unless project&.lfs_enabled? && oid.present? && url.present? + + return if LfsObject.exists?(oid: oid) + + sanitized_uri = Gitlab::UrlSanitizer.new(url) + + with_tmp_file(oid) do |file| + size = download_and_save_file(file, sanitized_uri) + lfs_object = LfsObject.new(oid: oid, size: size, file: file) + + project.all_lfs_objects << lfs_object + end + rescue StandardError => e + Rails.logger.error("LFS file with oid #{oid} could't be downloaded from #{sanitized_uri.sanitized_url}: #{e.message}") + end + + private + + def download_and_save_file(file, sanitized_uri) + IO.copy_stream(open(sanitized_uri.sanitized_url, headers(sanitized_uri)), file) + end + + def headers(sanitized_uri) + {}.tap do |headers| + credentials = sanitized_uri.credentials + + if credentials[:user].present? || credentials[:password].present? + # Using authentication headers in the request + headers[:http_basic_authentication] = [credentials[:user], credentials[:password]] + end + end + end + + def with_tmp_file(oid) + create_tmp_storage_dir + + File.open(File.join(tmp_storage_dir, oid), 'w') { |file| yield file } + end + + def create_tmp_storage_dir + FileUtils.makedirs(tmp_storage_dir) unless Dir.exist?(tmp_storage_dir) + end + + def tmp_storage_dir + @tmp_storage_dir ||= File.join(storage_dir, 'tmp', 'download') + end + + def storage_dir + @storage_dir ||= Gitlab.config.lfs.storage_path + end + end + end +end diff --git a/app/services/projects/lfs_pointers/lfs_import_service.rb b/app/services/projects/lfs_pointers/lfs_import_service.rb new file mode 100644 index 00000000000..b6b0dec142f --- /dev/null +++ b/app/services/projects/lfs_pointers/lfs_import_service.rb @@ -0,0 +1,92 @@ +# This service manages the whole worflow of discovering the Lfs files in a +# repository, linking them to the project and downloading (and linking) the non +# existent ones. +module Projects + module LfsPointers + class LfsImportService < BaseService + include Gitlab::Utils::StrongMemoize + + HEAD_REV = 'HEAD'.freeze + LFS_ENDPOINT_PATTERN = /^\t?url\s*=\s*(.+)$/.freeze + LFS_BATCH_API_ENDPOINT = '/info/lfs/objects/batch'.freeze + + LfsImportError = Class.new(StandardError) + + def execute + return {} unless project&.lfs_enabled? + + if external_lfs_endpoint? + # If the endpoint host is different from the import_url it means + # that the repo is using a third party service for storing the LFS files. + # In this case, we have to disable lfs in the project + disable_lfs! + + return {} + end + + get_download_links + rescue LfsDownloadLinkListService::DownloadLinksError => e + raise LfsImportError, "The LFS objects download list couldn't be imported. Error: #{e.message}" + end + + private + + def external_lfs_endpoint? + lfsconfig_endpoint_uri && lfsconfig_endpoint_uri.host != import_uri.host + end + + def disable_lfs! + project.update(lfs_enabled: false) + end + + def get_download_links + existent_lfs = LfsListService.new(project).execute + linked_oids = LfsLinkService.new(project).execute(existent_lfs.keys) + + # Retrieving those oids not linked and which we need to download + not_linked_lfs = existent_lfs.except(*linked_oids) + + LfsDownloadLinkListService.new(project, remote_uri: current_endpoint_uri).execute(not_linked_lfs) + end + + def lfsconfig_endpoint_uri + strong_memoize(:lfsconfig_endpoint_uri) do + # Retrieveing the blob data from the .lfsconfig file + data = project.repository.lfsconfig_for(HEAD_REV) + # Parsing the data to retrieve the url + parsed_data = data&.match(LFS_ENDPOINT_PATTERN) + + if parsed_data + URI.parse(parsed_data[1]).tap do |endpoint| + endpoint.user ||= import_uri.user + endpoint.password ||= import_uri.password + end + end + end + rescue URI::InvalidURIError + raise LfsImportError, 'Invalid URL in .lfsconfig file' + end + + def import_uri + @import_uri ||= URI.parse(project.import_url) + rescue URI::InvalidURIError + raise LfsImportError, 'Invalid project import URL' + end + + def current_endpoint_uri + (lfsconfig_endpoint_uri || default_endpoint_uri) + end + + # The import url must end with '.git' here we ensure it is + def default_endpoint_uri + @default_endpoint_uri ||= begin + import_uri.dup.tap do |uri| + path = uri.path.gsub(%r(/$), '') + path += '.git' unless path.ends_with?('.git') + uri.path = path + LFS_BATCH_API_ENDPOINT + end + end + end + end + end +end diff --git a/app/services/projects/lfs_pointers/lfs_link_service.rb b/app/services/projects/lfs_pointers/lfs_link_service.rb new file mode 100644 index 00000000000..d20bdf86c58 --- /dev/null +++ b/app/services/projects/lfs_pointers/lfs_link_service.rb @@ -0,0 +1,29 @@ +# Given a list of oids, this services links the existent Lfs Objects to the project +module Projects + module LfsPointers + class LfsLinkService < BaseService + # Accept an array of oids to link + # + # Returns a hash with the same structure with oids linked + def execute(oids) + return {} unless project&.lfs_enabled? + + # Search and link existing LFS Object + link_existing_lfs_objects(oids) + end + + private + + def link_existing_lfs_objects(oids) + existent_lfs_objects = LfsObject.where(oid: oids) + + return [] unless existent_lfs_objects.any? + + not_linked_lfs_objects = existent_lfs_objects.where.not(id: project.all_lfs_objects) + project.all_lfs_objects << not_linked_lfs_objects + + existent_lfs_objects.pluck(:oid) + end + end + end +end diff --git a/app/services/projects/lfs_pointers/lfs_list_service.rb b/app/services/projects/lfs_pointers/lfs_list_service.rb new file mode 100644 index 00000000000..b770982cbc0 --- /dev/null +++ b/app/services/projects/lfs_pointers/lfs_list_service.rb @@ -0,0 +1,19 @@ +# This service list all existent Lfs objects in a repository +module Projects + module LfsPointers + class LfsListService < BaseService + REV = 'HEAD'.freeze + + # Retrieve all lfs blob pointers and returns a hash + # with the structure { lfs_file_oid => lfs_file_size } + def execute + return {} unless project&.lfs_enabled? + + Gitlab::Git::LfsChanges.new(project.repository, REV) + .all_pointers + .map! { |blob| [blob.lfs_oid, blob.lfs_size] } + .to_h + end + end + end +end diff --git a/app/services/projects/update_service.rb b/app/services/projects/update_service.rb index 679f4a9cb62..d8250cd8102 100644 --- a/app/services/projects/update_service.rb +++ b/app/services/projects/update_service.rb @@ -17,6 +17,11 @@ module Projects ensure_wiki_exists if enabling_wiki? + yield if block_given? + + # If the block added errors, don't try to save the project + return validation_failed! if project.errors.any? + if project.update_attributes(params.except(:default_branch)) if project.previous_changes.include?('path') project.rename_repo @@ -28,21 +33,25 @@ module Projects success else - model_errors = project.errors.full_messages.to_sentence - error_message = model_errors.presence || 'Project could not be updated!' - - error(error_message) + validation_failed! end end def run_auto_devops_pipeline? - return false if project.repository.gitlab_ci_yml || !project.auto_devops.previous_changes.include?('enabled') + return false if project.repository.gitlab_ci_yml || !project.auto_devops&.previous_changes&.include?('enabled') project.auto_devops.enabled? || (project.auto_devops.enabled.nil? && Gitlab::CurrentSettings.auto_devops_enabled?) end private + def validation_failed! + model_errors = project.errors.full_messages.to_sentence + error_message = model_errors.presence || 'Project could not be updated!' + + error(error_message) + end + def renaming_project_with_container_registry_tags? new_path = params[:path] @@ -53,8 +62,8 @@ module Projects def changing_default_branch? new_branch = params[:default_branch] - project.repository.exists? && - new_branch && new_branch != project.default_branch + new_branch && project.repository.exists? && + new_branch != project.default_branch end def enabling_wiki? diff --git a/app/services/test_hooks/project_service.rb b/app/services/test_hooks/project_service.rb index 01d5d774cd5..65183e84cce 100644 --- a/app/services/test_hooks/project_service.rb +++ b/app/services/test_hooks/project_service.rb @@ -1,11 +1,13 @@ module TestHooks class ProjectService < TestHooks::BaseService - private + attr_writer :project def project @project ||= hook.project end + private + def push_events_data throw(:validation_error, 'Ensure the project has at least one commit.') if project.empty_repo? diff --git a/app/uploaders/object_storage.rb b/app/uploaders/object_storage.rb index 5bdca26a584..5aa1bc7227c 100644 --- a/app/uploaders/object_storage.rb +++ b/app/uploaders/object_storage.rb @@ -10,8 +10,6 @@ module ObjectStorage UnknownStoreError = Class.new(StandardError) ObjectStorageUnavailable = Class.new(StandardError) - DIRECT_UPLOAD_TIMEOUT = 4.hours - DIRECT_UPLOAD_EXPIRE_OFFSET = 15.minutes TMP_UPLOAD_PATH = 'tmp/uploads'.freeze module Store @@ -35,7 +33,7 @@ module ObjectStorage unless current_upload_satisfies?(paths, model) # the upload we already have isn't right, find the correct one - self.upload = uploads.find_by(model: model, path: paths) + self.upload = model&.retrieve_upload(identifier, paths) end super @@ -48,7 +46,7 @@ module ObjectStorage end def upload=(upload) - return unless upload + return if upload.nil? self.object_store = upload.store super @@ -157,9 +155,9 @@ module ObjectStorage model_class.uploader_options.dig(mount_point, :mount_on) || mount_point end - def workhorse_authorize + def workhorse_authorize(has_length:, maximum_size: nil) { - RemoteObject: workhorse_remote_upload_options, + RemoteObject: workhorse_remote_upload_options(has_length: has_length, maximum_size: maximum_size), TempPath: workhorse_local_upload_path }.compact end @@ -168,23 +166,16 @@ module ObjectStorage File.join(self.root, TMP_UPLOAD_PATH) end - def workhorse_remote_upload_options + def workhorse_remote_upload_options(has_length:, maximum_size: nil) return unless self.object_store_enabled? return unless self.direct_upload_enabled? id = [CarrierWave.generate_cache_id, SecureRandom.hex].join('-') upload_path = File.join(TMP_UPLOAD_PATH, id) - connection = ::Fog::Storage.new(self.object_store_credentials) - expire_at = Time.now + DIRECT_UPLOAD_TIMEOUT + DIRECT_UPLOAD_EXPIRE_OFFSET - options = { 'Content-Type' => 'application/octet-stream' } + direct_upload = ObjectStorage::DirectUpload.new(self.object_store_credentials, remote_store_path, upload_path, + has_length: has_length, maximum_size: maximum_size) - { - ID: id, - Timeout: DIRECT_UPLOAD_TIMEOUT, - GetURL: connection.get_object_url(remote_store_path, upload_path, expire_at), - DeleteURL: connection.delete_object_url(remote_store_path, upload_path, expire_at), - StoreURL: connection.put_object_url(remote_store_path, upload_path, expire_at, options) - } + direct_upload.to_hash.merge(ID: id) end end diff --git a/app/views/admin/projects/show.html.haml b/app/views/admin/projects/show.html.haml index 0a22a142858..ccba1c461fc 100644 --- a/app/views/admin/projects/show.html.haml +++ b/app/views/admin/projects/show.html.haml @@ -116,8 +116,8 @@ .card-body = form_for @project, url: transfer_admin_project_path(@project), method: :put do |f| .form-group.row - = f.label :new_namespace_id, "Namespace", class: 'col-form-label col-sm-2' - .col-sm-10 + = f.label :new_namespace_id, "Namespace", class: 'col-form-label col-sm-3' + .col-sm-9 .dropdown = dropdown_toggle('Search for Namespace', { toggle: 'dropdown', field_name: 'new_namespace_id' }, { toggle_class: 'js-namespace-select large' }) .dropdown-menu.dropdown-select @@ -127,7 +127,7 @@ = dropdown_loading .form-group.row - .offset-sm-2.col-sm-10 + .offset-sm-3.col-sm-9 = f.submit 'Transfer', class: 'btn btn-primary' .card.repository-check diff --git a/app/views/admin/runners/index.html.haml b/app/views/admin/runners/index.html.haml index f38aeb151df..8dfd176f1b7 100644 --- a/app/views/admin/runners/index.html.haml +++ b/app/views/admin/runners/index.html.haml @@ -67,7 +67,7 @@ %th Projects %th Jobs %th Tags - %th= link_to 'Last contact', admin_runners_path(params.slice(:search).merge(sort: 'contacted_asc')) + %th= link_to 'Last contact', admin_runners_path(safe_params.slice(:search).merge(sort: 'contacted_asc')) %th - @runners.each do |runner| diff --git a/app/views/admin/users/_user.html.haml b/app/views/admin/users/_user.html.haml index 5e176b61d68..b2163ee85fa 100644 --- a/app/views/admin/users/_user.html.haml +++ b/app/views/admin/users/_user.html.haml @@ -38,7 +38,7 @@ %li.divider - if user.can_be_removed? %li - %button.delete-user-button.btn.btn-danger{ data: { toggle: 'modal', + %button.delete-user-button.btn.text-danger{ data: { toggle: 'modal', target: '#delete-user-modal', delete_user_url: admin_user_path(user), block_user_url: block_admin_user_path(user), @@ -47,7 +47,7 @@ = s_('AdminUsers|Delete user') %li - %button.delete-user-button.btn.btn-danger{ data: { toggle: 'modal', + %button.delete-user-button.btn.text-danger{ data: { toggle: 'modal', target: '#delete-user-modal', delete_user_url: admin_user_path(user, hard_delete: true), block_user_url: block_admin_user_path(user), diff --git a/app/views/groups/_activities.html.haml b/app/views/groups/_activities.html.haml index fd6e7111f38..577c63503a8 100644 --- a/app/views/groups/_activities.html.haml +++ b/app/views/groups/_activities.html.haml @@ -1,4 +1,4 @@ -.nav-block +.nav-block.activities .controls = link_to group_path(@group, rss_url_options), class: 'btn rss-btn has-tooltip' , title: 'Subscribe' do %i.fa.fa-rss diff --git a/app/views/groups/settings/ci_cd/show.html.haml b/app/views/groups/settings/ci_cd/show.html.haml index 383d955d71f..ff2b418e479 100644 --- a/app/views/groups/settings/ci_cd/show.html.haml +++ b/app/views/groups/settings/ci_cd/show.html.haml @@ -7,7 +7,7 @@ .settings-header %h4 = _('Variables') - = link_to icon('question-circle'), help_page_path('ci/variables/README', anchor: 'secret-variables'), target: '_blank', rel: 'noopener noreferrer' + = link_to icon('question-circle'), help_page_path('ci/variables/README', anchor: 'variables'), target: '_blank', rel: 'noopener noreferrer' %button.btn.btn-default.js-settings-toggle{ type: "button" } = expanded ? _('Collapse') : _('Expand') %p.append-bottom-0 diff --git a/app/views/help/_shortcuts.html.haml b/app/views/help/_shortcuts.html.haml index 29db29235c1..c23fe0b5c49 100644 --- a/app/views/help/_shortcuts.html.haml +++ b/app/views/help/_shortcuts.html.haml @@ -18,71 +18,71 @@ %th Global Shortcuts %tr %td.shortcut - .key s + %kbd s %td Focus Search %tr %td.shortcut - .key f + %kbd f %td Focus Filter - if performance_bar_enabled? %tr %td.shortcut - .key p b + %kbd p b %td Show/hide the Performance Bar %tr %td.shortcut - .key ? + %kbd ? %td Show/hide this dialog %tr %td.shortcut - if browser.platform.mac? - .key ⌘ shift p + %kbd ⌘ shift p - else - .key ctrl shift p + %kbd ctrl shift p %td Toggle Markdown preview %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-up %td Edit last comment (when focused on an empty textarea) %tr %td.shortcut - .key shift t + %kbd shift t %td Go to todos %tr %td.shortcut - .key shift a + %kbd shift a %td Go to the activity feed %tr %td.shortcut - .key shift p + %kbd shift p %td Go to projects %tr %td.shortcut - .key shift i + %kbd shift i %td Go to issues %tr %td.shortcut - .key shift m + %kbd shift m %td Go to merge requests %tr %td.shortcut - .key shift g + %kbd shift g %td Go to groups %tr %td.shortcut - .key shift l + %kbd shift l %td Go to milestones %tr %td.shortcut - .key shift s + %kbd shift s %td Go to snippets %tbody @@ -91,21 +91,21 @@ %th Finding Project File %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-up %td Move selection up %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-down %td Move selection down %tr %td.shortcut - .key enter + %kbd enter %td Open Selection %tr %td.shortcut - .key esc + %kbd esc %td Go back .col-lg-4 %table.shortcut-mappings @@ -115,95 +115,95 @@ %th Project %tr %td.shortcut - .key g - .key p + %kbd g + %kbd p %td Go to the project's overview page %tr %td.shortcut - .key g - .key v + %kbd g + %kbd v %td Go to the project's activity feed %tr %td.shortcut - .key g - .key f + %kbd g + %kbd f %td Go to files %tr %td.shortcut - .key g - .key c + %kbd g + %kbd c %td Go to commits %tr %td.shortcut - .key g - .key j + %kbd g + %kbd j %td Go to jobs %tr %td.shortcut - .key g - .key n + %kbd g + %kbd n %td Go to network graph %tr %td.shortcut - .key g - .key d + %kbd g + %kbd d %td Go to repository charts %tr %td.shortcut - .key g - .key i + %kbd g + %kbd i %td Go to issues %tr %td.shortcut - .key g - .key b + %kbd g + %kbd b %td Go to issue boards %tr %td.shortcut - .key g - .key m + %kbd g + %kbd m %td Go to merge requests %tr %td.shortcut - .key g - .key e + %kbd g + %kbd e %td Go to environments %tr %td.shortcut - .key g - .key k + %kbd g + %kbd k %td Go to kubernetes %tr %td.shortcut - .key g - .key s + %kbd g + %kbd s %td Go to snippets %tr %td.shortcut - .key g - .key w + %kbd g + %kbd w %td Go to wiki %tr %td.shortcut - .key t + %kbd t %td Go to finding file %tr %td.shortcut - .key i + %kbd i %td New issue %tbody @@ -212,17 +212,17 @@ %th Project Files browsing %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-up %td Move selection up %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-down %td Move selection down %tr %td.shortcut - .key enter + %kbd enter %td Open Selection %tbody %tr @@ -230,7 +230,7 @@ %th Project File %tr %td.shortcut - .key y + %kbd y %td Go to file permalink %tbody %tr @@ -239,115 +239,115 @@ %tr %td.shortcut - if browser.platform.mac? - .key ⌘ p + %kbd ⌘ p - else - .key ctrl p + %kbd ctrl p %td Go to file .col-lg-4 %table.shortcut-mappings - %tbody.hidden-shortcut.network{ style: 'display:none' } + %tbody.hidden-shortcut{ style: 'display:none' } %tr %th %th Network Graph %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-left \/ - .key h + %kbd h %td Scroll left %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-right \/ - .key l + %kbd l %td Scroll right %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-up \/ - .key k + %kbd k %td Scroll up %tr %td.shortcut - .key + %kbd %i.fa.fa-arrow-down \/ - .key j + %kbd j %td Scroll down %tr %td.shortcut - .key + %kbd shift %i.fa.fa-arrow-up \/ - .key + %kbd shift k %td Scroll to top %tr %td.shortcut - .key + %kbd shift %i.fa.fa-arrow-down \/ - .key + %kbd shift j %td Scroll to bottom - %tbody.hidden-shortcut.issues{ style: 'display:none' } + %tbody.hidden-shortcut{ style: 'display:none' } %tr %th %th Issues %tr %td.shortcut - .key a + %kbd a %td Change assignee %tr %td.shortcut - .key m + %kbd m %td Change milestone %tr %td.shortcut - .key r + %kbd r %td Reply (quoting selected text) %tr %td.shortcut - .key e + %kbd e %td Edit issue %tr %td.shortcut - .key l + %kbd l %td Change Label - %tbody.hidden-shortcut.merge_requests{ style: 'display:none' } + %tbody.hidden-shortcut{ style: 'display:none' } %tr %th %th Merge Requests %tr %td.shortcut - .key a + %kbd a %td Change assignee %tr %td.shortcut - .key m + %kbd m %td Change milestone %tr %td.shortcut - .key r + %kbd r %td Reply (quoting selected text) %tr %td.shortcut - .key e + %kbd e %td Edit merge request %tr %td.shortcut - .key l + %kbd l %td Change Label - %tbody.hidden-shortcut.wiki{ style: 'display:none' } + %tbody.hidden-shortcut{ style: 'display:none' } %tr %th %th Wiki pages %tr %td.shortcut - .key e + %kbd e %td Edit wiki page diff --git a/app/views/layouts/devise.html.haml b/app/views/layouts/devise.html.haml index 82ca7252424..81f35615555 100644 --- a/app/views/layouts/devise.html.haml +++ b/app/views/layouts/devise.html.haml @@ -1,7 +1,7 @@ !!! 5 %html.devise-layout-html{ class: system_message_class } = render "layouts/head" - %body.ui_indigo.login-page.application.navless{ data: { page: body_data_page } } + %body.ui-indigo.login-page.application.navless{ data: { page: body_data_page } } .page-wrap = render "layouts/header/empty" .login-page-broadcast diff --git a/app/views/layouts/devise_empty.html.haml b/app/views/layouts/devise_empty.html.haml index adf90cb7667..52805e0da73 100644 --- a/app/views/layouts/devise_empty.html.haml +++ b/app/views/layouts/devise_empty.html.haml @@ -1,7 +1,7 @@ !!! 5 %html{ lang: "en", class: system_message_class } = render "layouts/head" - %body.ui_indigo.login-page.application.navless + %body.ui-indigo.login-page.application.navless = render "layouts/header/empty" = render "layouts/broadcast" .container.navless-container diff --git a/app/views/profiles/preferences/show.html.haml b/app/views/profiles/preferences/show.html.haml index ce312943154..8f1078bd41d 100644 --- a/app/views/profiles/preferences/show.html.haml +++ b/app/views/profiles/preferences/show.html.haml @@ -4,18 +4,12 @@ = form_for @user, url: profile_preferences_path, remote: true, method: :put, html: { class: 'row prepend-top-default js-preferences-form' } do |f| .col-lg-4.application-theme %h4.prepend-top-0 - s_('Preferences|Navigation theme') + = s_('Preferences|Navigation theme') %p Customize the appearance of the application header and navigation sidebar. .col-lg-8.application-theme - Gitlab::Themes.each do |theme| = label_tag do - .preview{ class: theme.name.downcase } - .preview-row - .quadrant.one - .quadrant.two - .preview-row - .quadrant.three - .quadrant.four + .preview{ class: theme.css_class } = f.radio_button :theme_id, theme.id, checked: Gitlab::Themes.for_user(@user).id == theme.id = theme.name diff --git a/app/views/projects/blob/viewers/_download.html.haml b/app/views/projects/blob/viewers/_download.html.haml index f9b1da05a00..fda4b9c92cd 100644 --- a/app/views/projects/blob/viewers/_download.html.haml +++ b/app/views/projects/blob/viewers/_download.html.haml @@ -1,5 +1,5 @@ .file-content.blob_file.blob-no-preview - .center.render-error.vertical-center + .center.render-error = link_to blob_raw_path do %h1.light = sprite_icon('download') diff --git a/app/views/projects/branches/_branch.html.haml b/app/views/projects/branches/_branch.html.haml index f641d7bc51a..88f9b7dfc9f 100644 --- a/app/views/projects/branches/_branch.html.haml +++ b/app/views/projects/branches/_branch.html.haml @@ -72,7 +72,7 @@ - else %button{ class: "btn btn-remove remove-row js-ajax-loading-spinner has-tooltip disabled", disabled: true, - title: s_('Branches|Only a project master or owner can delete a protected branch') } + title: s_('Branches|Only a project maintainer or owner can delete a protected branch') } = icon("trash-o") - else = link_to project_branch_path(@project, branch.name), diff --git a/app/views/projects/clusters/_advanced_settings.html.haml b/app/views/projects/clusters/_advanced_settings.html.haml index e9bdc54364b..243e8cd9ba0 100644 --- a/app/views/projects/clusters/_advanced_settings.html.haml +++ b/app/views/projects/clusters/_advanced_settings.html.haml @@ -7,8 +7,8 @@ - link_gke = link_to(s_('ClusterIntegration|Google Kubernetes Engine'), @cluster.gke_cluster_url, target: '_blank', rel: 'noopener noreferrer') = s_('ClusterIntegration|Manage your Kubernetes cluster by visiting %{link_gke}').html_safe % { link_gke: link_gke } - .card.form-group - %label.text-danger + .sub-section.form-group + %h4.text-danger = s_('ClusterIntegration|Remove Kubernetes cluster integration') %p = s_("ClusterIntegration|Remove this Kubernetes cluster's configuration from this project. This will not delete your actual Kubernetes cluster.") diff --git a/app/views/projects/jobs/show.html.haml b/app/views/projects/jobs/show.html.haml index ec9a04c0eab..1f33bb3a129 100644 --- a/app/views/projects/jobs/show.html.haml +++ b/app/views/projects/jobs/show.html.haml @@ -86,9 +86,7 @@ %button.js-scroll-down.btn-scroll.btn-transparent.btn-blank{ type: 'button', disabled: true } = custom_icon('scroll_down') - %pre.build-trace#build-trace - %code.bash.js-build-output - .build-loader-animation.js-build-refresh + = render 'shared/builds/build_output' - else = render "empty_states" diff --git a/app/views/projects/pages/_destroy.haml b/app/views/projects/pages/_destroy.haml index 4ada19a1368..9b77c4e3494 100644 --- a/app/views/projects/pages/_destroy.haml +++ b/app/views/projects/pages/_destroy.haml @@ -5,7 +5,7 @@ .errors-holder .card-body %p - Removing the pages will prevent from exposing them to outside world. + Removing pages will prevent them from being exposed to the outside world. .form-actions = link_to 'Remove pages', project_pages_path(@project), data: { confirm: 'Are you sure?'}, method: :delete, class: "btn btn-remove" - else diff --git a/app/views/projects/project_members/index.html.haml b/app/views/projects/project_members/index.html.haml index a56023e98cd..9716322f8a1 100644 --- a/app/views/projects/project_members/index.html.haml +++ b/app/views/projects/project_members/index.html.haml @@ -12,17 +12,17 @@ - else %p Members can be added by project - %i Masters + %i Maintainers or %i Owners .light - if can?(current_user, :admin_project_member, @project) %ul.nav-links.nav.nav-tabs.gitlab-tabs{ role: 'tablist' } - %li.active{ role: 'presentation' } - %a{ href: '#add-member-pane', id: 'add-member-tab', data: { toggle: 'tab' }, role: 'tab' } Add member + %li.nav-tab{ role: 'presentation' } + %a.nav-link.active{ href: '#add-member-pane', id: 'add-member-tab', data: { toggle: 'tab' }, role: 'tab' } Add member - if @project.allowed_to_share_with_group? - %li{ role: 'presentation' } - %a{ href: '#share-with-group-pane', id: 'share-with-group-tab', data: { toggle: 'tab' }, role: 'tab' } Share with group + %li.nav-tab{ role: 'presentation' } + %a.nav-link{ href: '#share-with-group-pane', id: 'share-with-group-tab', data: { toggle: 'tab' }, role: 'tab' } Share with group .tab-content.gitlab-tab-content .tab-pane.active{ id: 'add-member-pane', role: 'tabpanel' } diff --git a/app/views/projects/protected_branches/shared/_index.html.haml b/app/views/projects/protected_branches/shared/_index.html.haml index fd5c1aa342a..846f8858d14 100644 --- a/app/views/projects/protected_branches/shared/_index.html.haml +++ b/app/views/projects/protected_branches/shared/_index.html.haml @@ -12,8 +12,8 @@ %p By default, protected branches are designed to: %ul - %li prevent their creation, if not already created, from everybody except Masters - %li prevent pushes from everybody except Masters + %li prevent their creation, if not already created, from everybody except Maintainers + %li prevent pushes from everybody except Maintainers %li prevent <strong>anyone</strong> from force pushing to the branch %li prevent <strong>anyone</strong> from deleting the branch %p Read more about #{link_to "protected branches", help_page_path("user/project/protected_branches")} and #{link_to "project permissions", help_page_path("user/permissions")}. diff --git a/app/views/projects/protected_tags/shared/_index.html.haml b/app/views/projects/protected_tags/shared/_index.html.haml index c33723d8072..fe2903b456f 100644 --- a/app/views/projects/protected_tags/shared/_index.html.haml +++ b/app/views/projects/protected_tags/shared/_index.html.haml @@ -12,7 +12,7 @@ %p By default, protected tags are designed to: %ul - %li Prevent tag creation by everybody except Masters + %li Prevent tag creation by everybody except Maintainers %li Prevent <strong>anyone</strong> from updating the tag %li Prevent <strong>anyone</strong> from deleting the tag diff --git a/app/views/projects/runners/_group_runners.html.haml b/app/views/projects/runners/_group_runners.html.haml index dfed0553f84..86de71c732b 100644 --- a/app/views/projects/runners/_group_runners.html.haml +++ b/app/views/projects/runners/_group_runners.html.haml @@ -26,9 +26,9 @@ - if can?(current_user, :admin_pipeline, @project.group) - group_link = link_to _('Group CI/CD settings'), group_settings_ci_cd_path(@project.group) - = _('Group masters can register group runners in the %{link}').html_safe % { link: group_link } + = _('Group maintainers can register group runners in the %{link}').html_safe % { link: group_link } - else - = _('Ask your group master to setup a group Runner.') + = _('Ask your group maintainer to setup a group Runner.') - else %h4.underlined-title diff --git a/app/views/projects/settings/ci_cd/show.html.haml b/app/views/projects/settings/ci_cd/show.html.haml index ed17bd4f7dc..ed118d1bcef 100644 --- a/app/views/projects/settings/ci_cd/show.html.haml +++ b/app/views/projects/settings/ci_cd/show.html.haml @@ -43,7 +43,7 @@ .settings-header %h4 = _('Variables') - = link_to icon('question-circle'), help_page_path('ci/variables/README', anchor: 'secret-variables'), target: '_blank', rel: 'noopener noreferrer' + = link_to icon('question-circle'), help_page_path('ci/variables/README', anchor: 'variables'), target: '_blank', rel: 'noopener noreferrer' %button.btn.js-settings-toggle{ type: 'button' } = expanded ? 'Collapse' : 'Expand' %p.append-bottom-0 diff --git a/app/views/projects/tree/_tree_header.html.haml b/app/views/projects/tree/_tree_header.html.haml index 4fc1a284693..9d196075bf1 100644 --- a/app/views/projects/tree/_tree_header.html.haml +++ b/app/views/projects/tree/_tree_header.html.haml @@ -6,7 +6,7 @@ = render 'shared/ref_switcher', destination: 'tree', path: @path, show_create: true - if on_top_of_branch? - - addtotree_toggle_attributes = { href: '#', 'data-toggle': 'dropdown', 'data-target': '.add-to-tree-dropdown' } + - addtotree_toggle_attributes = { href: '#', 'data-toggle': 'dropdown', 'data-target': '.add-to-tree-dropdown', 'data-boundary': 'window' } - else - addtotree_toggle_attributes = { title: _("You can only add files when you are on a branch"), data: { container: 'body' }, class: 'disabled has-tooltip' } diff --git a/app/views/shared/builds/_build_output.html.haml b/app/views/shared/builds/_build_output.html.haml new file mode 100644 index 00000000000..07f1501fadd --- /dev/null +++ b/app/views/shared/builds/_build_output.html.haml @@ -0,0 +1,3 @@ +%pre.build-trace#build-trace + %code.bash.js-build-output + .build-loader-animation.js-build-refresh diff --git a/app/views/shared/issuable/form/_contribution.html.haml b/app/views/shared/issuable/form/_contribution.html.haml index b34549240e0..519b5fae846 100644 --- a/app/views/shared/issuable/form/_contribution.html.haml +++ b/app/views/shared/issuable/form/_contribution.html.haml @@ -12,9 +12,9 @@ = _('Contribution') .col-sm-10 .form-check - = form.check_box :allow_maintainer_to_push, disabled: !issuable.can_allow_maintainer_to_push?(current_user), class: 'form-check-input' - = form.label :allow_maintainer_to_push, class: 'form-check-label' do - = _('Allow edits from maintainers.') - = link_to 'About this feature', help_page_path('user/project/merge_requests/maintainer_access') + = form.check_box :allow_collaboration, disabled: !issuable.can_allow_collaboration?(current_user), class: 'form-check-input' + = form.label :allow_collaboration, class: 'form-check-label' do + = _('Allow commits from members who can merge to the target branch.') + = link_to 'About this feature', help_page_path('user/project/merge_requests/allow_collaboration') .form-text.text-muted - = allow_maintainer_push_unavailable_reason(issuable) + = allow_collaboration_unavailable_reason(issuable) diff --git a/app/views/shared/projects/_edit_information.html.haml b/app/views/shared/projects/_edit_information.html.haml index ec9dc8f62c2..9230e045a81 100644 --- a/app/views/shared/projects/_edit_information.html.haml +++ b/app/views/shared/projects/_edit_information.html.haml @@ -1,6 +1,6 @@ - unless can?(current_user, :push_code, @project) .inline.prepend-left-10 - - if @project.branch_allows_maintainer_push?(current_user, selected_branch) + - if @project.branch_allows_collaboration?(current_user, selected_branch) = commit_in_single_accessible_branch - else = commit_in_fork_help diff --git a/app/views/users/terms/index.html.haml b/app/views/users/terms/index.html.haml index b9f25a71170..33cddf63952 100644 --- a/app/views/users/terms/index.html.haml +++ b/app/views/users/terms/index.html.haml @@ -7,6 +7,10 @@ .float-right = button_to accept_term_path(@term, redirect_params), class: 'btn btn-success prepend-left-8' do = _('Accept terms') + - else + .pull-right + = link_to root_path, class: 'btn btn-success prepend-left-8' do + = _('Continue') - if can?(current_user, :decline_terms, @term) .float-right = button_to decline_term_path(@term, redirect_params), class: 'btn btn-default prepend-left-8' do diff --git a/app/workers/all_queues.yml b/app/workers/all_queues.yml index 93e57512edb..30b6796a7d6 100644 --- a/app/workers/all_queues.yml +++ b/app/workers/all_queues.yml @@ -17,6 +17,7 @@ - cronjob:stuck_ci_jobs - cronjob:stuck_import_jobs - cronjob:stuck_merge_jobs +- cronjob:ci_archive_traces_cron - cronjob:trending_projects - cronjob:issue_due_scheduler @@ -30,12 +31,14 @@ - github_importer:github_import_import_diff_note - github_importer:github_import_import_issue - github_importer:github_import_import_note +- github_importer:github_import_import_lfs_object - github_importer:github_import_import_pull_request - github_importer:github_import_refresh_import_jid - github_importer:github_import_stage_finish_import - github_importer:github_import_stage_import_base_data - github_importer:github_import_stage_import_issues_and_diff_notes - github_importer:github_import_stage_import_notes +- github_importer:github_import_stage_import_lfs_objects - github_importer:github_import_stage_import_pull_requests - github_importer:github_import_stage_import_repository diff --git a/app/workers/ci/archive_traces_cron_worker.rb b/app/workers/ci/archive_traces_cron_worker.rb new file mode 100644 index 00000000000..2ac65f41f4e --- /dev/null +++ b/app/workers/ci/archive_traces_cron_worker.rb @@ -0,0 +1,26 @@ +module Ci + class ArchiveTracesCronWorker + include ApplicationWorker + include CronjobQueue + + def perform + # Archive stale live traces which still resides in redis or database + # This could happen when ArchiveTraceWorker sidekiq jobs were lost by receiving SIGKILL + # More details in https://gitlab.com/gitlab-org/gitlab-ce/issues/36791 + Ci::Build.finished.with_live_trace.find_each(batch_size: 100) do |build| + begin + build.trace.archive! + rescue => e + failed_archive_counter.increment + Rails.logger.error "Failed to archive stale live trace. id: #{build.id} message: #{e.message}" + end + end + end + + private + + def failed_archive_counter + @failed_archive_counter ||= Gitlab::Metrics.counter(:job_trace_archive_failed_total, "Counter of failed attempts of traces archiving") + end + end +end diff --git a/app/workers/git_garbage_collect_worker.rb b/app/workers/git_garbage_collect_worker.rb index be4203bc7ad..f3c9e2b1582 100644 --- a/app/workers/git_garbage_collect_worker.rb +++ b/app/workers/git_garbage_collect_worker.rb @@ -29,7 +29,7 @@ class GitGarbageCollectWorker task = task.to_sym cmd = command(task) - gitaly_migrate(GITALY_MIGRATED_TASKS[task]) do |is_enabled| + gitaly_migrate(GITALY_MIGRATED_TASKS[task], status: Gitlab::GitalyClient::MigrationStatus::OPT_OUT) do |is_enabled| if is_enabled gitaly_call(task, project.repository.raw_repository) else @@ -114,8 +114,8 @@ class GitGarbageCollectWorker %W[git -c repack.writeBitmaps=#{config_value}] end - def gitaly_migrate(method, &block) - Gitlab::GitalyClient.migrate(method, &block) + def gitaly_migrate(method, status: Gitlab::GitalyClient::MigrationStatus::OPT_IN, &block) + Gitlab::GitalyClient.migrate(method, status: status, &block) rescue GRPC::NotFound => e Gitlab::GitLogger.error("#{method} failed:\nRepository not found") raise Gitlab::Git::Repository::NoRepository.new(e) diff --git a/app/workers/gitlab/github_import/advance_stage_worker.rb b/app/workers/gitlab/github_import/advance_stage_worker.rb index 8d708e15a66..be0b6c180b0 100644 --- a/app/workers/gitlab/github_import/advance_stage_worker.rb +++ b/app/workers/gitlab/github_import/advance_stage_worker.rb @@ -21,6 +21,7 @@ module Gitlab STAGES = { issues_and_diff_notes: Stage::ImportIssuesAndDiffNotesWorker, notes: Stage::ImportNotesWorker, + lfs_objects: Stage::ImportLfsObjectsWorker, finish: Stage::FinishImportWorker }.freeze diff --git a/app/workers/gitlab/github_import/import_lfs_object_worker.rb b/app/workers/gitlab/github_import/import_lfs_object_worker.rb new file mode 100644 index 00000000000..520c5cb091a --- /dev/null +++ b/app/workers/gitlab/github_import/import_lfs_object_worker.rb @@ -0,0 +1,25 @@ +# frozen_string_literal: true + +module Gitlab + module GithubImport + class ImportLfsObjectWorker + include ObjectImporter + + def representation_class + Representation::LfsObject + end + + def importer_class + Importer::LfsObjectImporter + end + + def counter_name + :github_importer_imported_lfs_objects + end + + def counter_description + 'The number of imported GitHub Lfs Objects' + end + end + end +end diff --git a/app/workers/gitlab/github_import/stage/import_lfs_objects_worker.rb b/app/workers/gitlab/github_import/stage/import_lfs_objects_worker.rb new file mode 100644 index 00000000000..29257603a9d --- /dev/null +++ b/app/workers/gitlab/github_import/stage/import_lfs_objects_worker.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +module Gitlab + module GithubImport + module Stage + class ImportLfsObjectsWorker + include ApplicationWorker + include GithubImport::Queue + include StageMethods + + def perform(project_id) + return unless (project = find_project(project_id)) + + import(project) + end + + # project - An instance of Project. + def import(project) + waiter = Importer::LfsObjectsImporter + .new(project, nil) + .execute + + AdvanceStageWorker.perform_async( + project.id, + { waiter.key => waiter.jobs_remaining }, + :finish + ) + end + end + end + end +end diff --git a/app/workers/gitlab/github_import/stage/import_notes_worker.rb b/app/workers/gitlab/github_import/stage/import_notes_worker.rb index 5f4678a595f..ccf0013180d 100644 --- a/app/workers/gitlab/github_import/stage/import_notes_worker.rb +++ b/app/workers/gitlab/github_import/stage/import_notes_worker.rb @@ -18,7 +18,7 @@ module Gitlab AdvanceStageWorker.perform_async( project.id, { waiter.key => waiter.jobs_remaining }, - :finish + :lfs_objects ) end end diff --git a/changelogs/unreleased/25045-add-variables-to-post-pipeline-api.yml b/changelogs/unreleased/25045-add-variables-to-post-pipeline-api.yml new file mode 100644 index 00000000000..1e648b75248 --- /dev/null +++ b/changelogs/unreleased/25045-add-variables-to-post-pipeline-api.yml @@ -0,0 +1,5 @@ +--- +title: Add variables to POST api/v4/projects/:id/pipeline +merge_request: 19124 +author: Jacopo Beschi @jacopo-beschi +type: added diff --git a/changelogs/unreleased/36862-subgroup-milestones.yml b/changelogs/unreleased/36862-subgroup-milestones.yml new file mode 100644 index 00000000000..98b9dc41cb1 --- /dev/null +++ b/changelogs/unreleased/36862-subgroup-milestones.yml @@ -0,0 +1,5 @@ +--- +title: Include milestones from parent groups when assigning a milestone to an issue or merge request +merge_request: +author: +type: changed diff --git a/changelogs/unreleased/42751-rename-master-to-maintainer.yml b/changelogs/unreleased/42751-rename-master-to-maintainer.yml new file mode 100644 index 00000000000..d7f499ecd52 --- /dev/null +++ b/changelogs/unreleased/42751-rename-master-to-maintainer.yml @@ -0,0 +1,5 @@ +--- +title: Rename the Master role to Maintainer +merge_request: 19080 +author: +type: changed diff --git a/changelogs/unreleased/42751-rename-mr-maintainer-push.yml b/changelogs/unreleased/42751-rename-mr-maintainer-push.yml new file mode 100644 index 00000000000..aa29f6ed4b7 --- /dev/null +++ b/changelogs/unreleased/42751-rename-mr-maintainer-push.yml @@ -0,0 +1,5 @@ +--- +title: Rephrasing Merge Request's 'allow edits from maintainer' functionality +merge_request: 19061 +author: +type: deprecated diff --git a/changelogs/unreleased/43597-new-navigation-themes.yml b/changelogs/unreleased/43597-new-navigation-themes.yml new file mode 100644 index 00000000000..de703e46b3c --- /dev/null +++ b/changelogs/unreleased/43597-new-navigation-themes.yml @@ -0,0 +1,5 @@ +--- +title: Add additional theme color options +merge_request: +author: +type: changed diff --git a/changelogs/unreleased/45702-fix-hashed-storage-repository-archive.yml b/changelogs/unreleased/45702-fix-hashed-storage-repository-archive.yml new file mode 100644 index 00000000000..0f85ced06a9 --- /dev/null +++ b/changelogs/unreleased/45702-fix-hashed-storage-repository-archive.yml @@ -0,0 +1,5 @@ +--- +title: Fix repository archive generation when hashed storage is enabled +merge_request: 19441 +author: +type: fixed diff --git a/changelogs/unreleased/45821-avatar_api.yml b/changelogs/unreleased/45821-avatar_api.yml new file mode 100644 index 00000000000..e16b28c36a2 --- /dev/null +++ b/changelogs/unreleased/45821-avatar_api.yml @@ -0,0 +1,5 @@ +--- +title: Add Avatar API +merge_request: 19121 +author: Imre Farkas +type: added diff --git a/changelogs/unreleased/46452-nomethoderror-undefined-method-previous_changes-for-nil-nilclass.yml b/changelogs/unreleased/46452-nomethoderror-undefined-method-previous_changes-for-nil-nilclass.yml new file mode 100644 index 00000000000..89dee65f5a8 --- /dev/null +++ b/changelogs/unreleased/46452-nomethoderror-undefined-method-previous_changes-for-nil-nilclass.yml @@ -0,0 +1,5 @@ +--- +title: Check for nil AutoDevOps when saving project CI/CD settings. +merge_request: 19190 +author: +type: fixed diff --git a/changelogs/unreleased/46585-gdpr-terms-acceptance.yml b/changelogs/unreleased/46585-gdpr-terms-acceptance.yml new file mode 100644 index 00000000000..84853846b0e --- /dev/null +++ b/changelogs/unreleased/46585-gdpr-terms-acceptance.yml @@ -0,0 +1,6 @@ +--- +title: Add flash notice if user has already accepted terms and allow users to continue + to root path +merge_request: 19156 +author: +type: changed diff --git a/changelogs/unreleased/46845-update-email_spec-to-2-2-0.yml b/changelogs/unreleased/46845-update-email_spec-to-2-2-0.yml new file mode 100644 index 00000000000..bf501340769 --- /dev/null +++ b/changelogs/unreleased/46845-update-email_spec-to-2-2-0.yml @@ -0,0 +1,5 @@ +--- +title: Update email_spec to 2.2.0 +merge_request: 19164 +author: Takuya Noguchi +type: other diff --git a/changelogs/unreleased/47189-github_import_visibility.yml b/changelogs/unreleased/47189-github_import_visibility.yml new file mode 100644 index 00000000000..a2a727a3227 --- /dev/null +++ b/changelogs/unreleased/47189-github_import_visibility.yml @@ -0,0 +1,6 @@ +--- +title: Use Github repo visibility during import while respecting restricted visibility + levels +merge_request: +author: +type: fixed diff --git a/changelogs/unreleased/add-background-migrations-for-not-archived-traces.yml b/changelogs/unreleased/add-background-migrations-for-not-archived-traces.yml new file mode 100644 index 00000000000..b1b23c477df --- /dev/null +++ b/changelogs/unreleased/add-background-migrations-for-not-archived-traces.yml @@ -0,0 +1,5 @@ +--- +title: Add background migrations for archiving legacy job traces +merge_request: 19194 +author: +type: performance diff --git a/changelogs/unreleased/bootstrap-changelog.yml b/changelogs/unreleased/bootstrap-changelog.yml new file mode 100644 index 00000000000..fd58447769d --- /dev/null +++ b/changelogs/unreleased/bootstrap-changelog.yml @@ -0,0 +1,5 @@ +--- +title: Upgrade GitLab from Bootstrap 3 to 4 +merge_request: +author: +type: other diff --git a/changelogs/unreleased/bvl-graphql-start-34754.yml b/changelogs/unreleased/bvl-graphql-start-34754.yml new file mode 100644 index 00000000000..a31f46d3a61 --- /dev/null +++ b/changelogs/unreleased/bvl-graphql-start-34754.yml @@ -0,0 +1,5 @@ +--- +title: Setup graphql with initial project & merge request query +merge_request: 19008 +author: +type: added diff --git a/changelogs/unreleased/fix-avatars-n-plus-one.yml b/changelogs/unreleased/fix-avatars-n-plus-one.yml new file mode 100644 index 00000000000..c5b42071f2b --- /dev/null +++ b/changelogs/unreleased/fix-avatars-n-plus-one.yml @@ -0,0 +1,5 @@ +--- +title: Fix an N+1 when loading user avatars +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/fj-40401-support-import-lfs-objects.yml b/changelogs/unreleased/fj-40401-support-import-lfs-objects.yml new file mode 100644 index 00000000000..a8abdd943ba --- /dev/null +++ b/changelogs/unreleased/fj-40401-support-import-lfs-objects.yml @@ -0,0 +1,5 @@ +--- +title: Added support for LFS Download in the importing process +merge_request: 18871 +author: +type: fixed diff --git a/changelogs/unreleased/jivl-smarter-system-notes.yml b/changelogs/unreleased/jivl-smarter-system-notes.yml new file mode 100644 index 00000000000..e640981de9a --- /dev/null +++ b/changelogs/unreleased/jivl-smarter-system-notes.yml @@ -0,0 +1,5 @@ +--- +title: Add support for smarter system notes +merge_request: 17164 +author: +type: changed diff --git a/changelogs/unreleased/jprovazn-uploader-migration.yml b/changelogs/unreleased/jprovazn-uploader-migration.yml new file mode 100644 index 00000000000..1db67e9ace2 --- /dev/null +++ b/changelogs/unreleased/jprovazn-uploader-migration.yml @@ -0,0 +1,5 @@ +--- +title: Migrate any remaining jobs from deprecated `object_storage_upload` queue. +merge_request: +author: +type: deprecated diff --git a/changelogs/unreleased/live-trace-v2-persist-data.yml b/changelogs/unreleased/live-trace-v2-persist-data.yml new file mode 100644 index 00000000000..3ac47b04218 --- /dev/null +++ b/changelogs/unreleased/live-trace-v2-persist-data.yml @@ -0,0 +1,5 @@ +--- +title: Add a cronworker to rescue stale live traces +merge_request: 18680 +author: +type: performance diff --git a/changelogs/unreleased/optimise-pages-service-calling.yml b/changelogs/unreleased/optimise-pages-service-calling.yml new file mode 100644 index 00000000000..e017e6b01f1 --- /dev/null +++ b/changelogs/unreleased/optimise-pages-service-calling.yml @@ -0,0 +1,5 @@ +--- +title: Optimise PagesWorker usage +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/optimise-runner-update-cached-info.yml b/changelogs/unreleased/optimise-runner-update-cached-info.yml new file mode 100644 index 00000000000..15fb9bcdf41 --- /dev/null +++ b/changelogs/unreleased/optimise-runner-update-cached-info.yml @@ -0,0 +1,5 @@ +--- +title: Update runner cached informations without performing validations +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/per-project-pipeline-iid.yml b/changelogs/unreleased/per-project-pipeline-iid.yml new file mode 100644 index 00000000000..78a513a9986 --- /dev/null +++ b/changelogs/unreleased/per-project-pipeline-iid.yml @@ -0,0 +1,5 @@ +--- +title: Add per-project pipeline id +merge_request: 18558 +author: +type: added diff --git a/changelogs/unreleased/presigned-multipart-uploads.yml b/changelogs/unreleased/presigned-multipart-uploads.yml new file mode 100644 index 00000000000..52fae6534fd --- /dev/null +++ b/changelogs/unreleased/presigned-multipart-uploads.yml @@ -0,0 +1,5 @@ +--- +title: Support direct_upload with S3 Multipart uploads +merge_request: +author: +type: added diff --git a/changelogs/unreleased/rails5-fix-46236.yml b/changelogs/unreleased/rails5-fix-46236.yml new file mode 100644 index 00000000000..9203b448bed --- /dev/null +++ b/changelogs/unreleased/rails5-fix-46236.yml @@ -0,0 +1,5 @@ +--- +title: Support rails5 in postgres indexes function and fix some migrations +merge_request: 19400 +author: Jasper Maes +type: fixed diff --git a/changelogs/unreleased/rails5-fix-47368.yml b/changelogs/unreleased/rails5-fix-47368.yml new file mode 100644 index 00000000000..81bb1adabff --- /dev/null +++ b/changelogs/unreleased/rails5-fix-47368.yml @@ -0,0 +1,6 @@ +--- +title: 'Rails 5 fix unknown keywords: changes, key_id, project, gl_repository, action, + secret_token, protocol' +merge_request: 19466 +author: Jasper Maes +type: fixed diff --git a/changelogs/unreleased/rails5-fix-47376.yml b/changelogs/unreleased/rails5-fix-47376.yml new file mode 100644 index 00000000000..ac9950e908e --- /dev/null +++ b/changelogs/unreleased/rails5-fix-47376.yml @@ -0,0 +1,5 @@ +--- +title: Rails 5 fix glob spec +merge_request: 19469 +author: Jasper Maes +type: fixed diff --git a/changelogs/unreleased/rd-44364-deprecate-support-for-dsa-keys.yml b/changelogs/unreleased/rd-44364-deprecate-support-for-dsa-keys.yml new file mode 100644 index 00000000000..1a52ffaaf79 --- /dev/null +++ b/changelogs/unreleased/rd-44364-deprecate-support-for-dsa-keys.yml @@ -0,0 +1,5 @@ +--- +title: Add migration to disable the usage of DSA keys +merge_request: 19299 +author: +type: other diff --git a/changelogs/unreleased/remove-unused-query-in-hooks.yml b/changelogs/unreleased/remove-unused-query-in-hooks.yml new file mode 100644 index 00000000000..ef40b2db5a9 --- /dev/null +++ b/changelogs/unreleased/remove-unused-query-in-hooks.yml @@ -0,0 +1,5 @@ +--- +title: Remove unused running_or_pending_build_count +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/sh-add-uncached-query-limiter.yml b/changelogs/unreleased/sh-add-uncached-query-limiter.yml new file mode 100644 index 00000000000..4318338c229 --- /dev/null +++ b/changelogs/unreleased/sh-add-uncached-query-limiter.yml @@ -0,0 +1,5 @@ +--- +title: Remove N+1 query for author in issues API +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/sh-fix-events-nplus-one.yml b/changelogs/unreleased/sh-fix-events-nplus-one.yml new file mode 100644 index 00000000000..e5a974bef30 --- /dev/null +++ b/changelogs/unreleased/sh-fix-events-nplus-one.yml @@ -0,0 +1,5 @@ +--- +title: Eliminate N+1 queries with authors and push_data_payload in Events API +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/sh-fix-pipeline-jobs-nplus-one.yml b/changelogs/unreleased/sh-fix-pipeline-jobs-nplus-one.yml new file mode 100644 index 00000000000..eac00f4fca6 --- /dev/null +++ b/changelogs/unreleased/sh-fix-pipeline-jobs-nplus-one.yml @@ -0,0 +1,5 @@ +--- +title: Eliminate N+1 queries for CI job artifacts in /api/prjoects/:id/pipelines/:pipeline_id/jobs +merge_request: +author: +type: performance diff --git a/changelogs/unreleased/sh-improve-import-status-error.yml b/changelogs/unreleased/sh-improve-import-status-error.yml new file mode 100644 index 00000000000..6523280f9e6 --- /dev/null +++ b/changelogs/unreleased/sh-improve-import-status-error.yml @@ -0,0 +1,5 @@ +--- +title: Show a more helpful error for import status +merge_request: +author: +type: other diff --git a/changelogs/unreleased/sh-log-422-responses.yml b/changelogs/unreleased/sh-log-422-responses.yml new file mode 100644 index 00000000000..c7dfdbb703b --- /dev/null +++ b/changelogs/unreleased/sh-log-422-responses.yml @@ -0,0 +1,6 @@ +--- +title: Log response body to production_json.log when a controller responds with a + 422 status +merge_request: +author: +type: other diff --git a/changelogs/unreleased/update-help-integration-screenshot.yml b/changelogs/unreleased/update-help-integration-screenshot.yml new file mode 100644 index 00000000000..b1f76a346e4 --- /dev/null +++ b/changelogs/unreleased/update-help-integration-screenshot.yml @@ -0,0 +1,5 @@ +--- +title: Update screenshot in Gitlab.com integration documentation +merge_request: 19433 +author: Tuğçe Nur TaÅŸ +type: other diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example index 7eb44b8059e..489dc8840e5 100644 --- a/config/gitlab.yml.example +++ b/config/gitlab.yml.example @@ -78,10 +78,15 @@ production: &base # username_changing_enabled: false # default: true - User can change her username/namespace ## Default theme ID ## 1 - Indigo - ## 2 - Dark - ## 3 - Light - ## 4 - Blue + ## 2 - Light Indigo + ## 3 - Blue + ## 4 - Light Blue ## 5 - Green + ## 6 - Light Green + ## 7 - Red + ## 8 - Light Red + ## 9 - Dark + ## 10 - Light # default_theme: 1 # default: 1 ## Automatic issue closing diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb index a0e3ab0d343..12d09150127 100644 --- a/config/initializers/1_settings.rb +++ b/config/initializers/1_settings.rb @@ -289,6 +289,9 @@ Settings.cron_jobs['repository_archive_cache_worker']['job_class'] = 'Repository Settings.cron_jobs['import_export_project_cleanup_worker'] ||= Settingslogic.new({}) Settings.cron_jobs['import_export_project_cleanup_worker']['cron'] ||= '0 * * * *' Settings.cron_jobs['import_export_project_cleanup_worker']['job_class'] = 'ImportExportProjectCleanupWorker' +Settings.cron_jobs['ci_archive_traces_cron_worker'] ||= Settingslogic.new({}) +Settings.cron_jobs['ci_archive_traces_cron_worker']['cron'] ||= '17 * * * *' +Settings.cron_jobs['ci_archive_traces_cron_worker']['job_class'] = 'Ci::ArchiveTracesCronWorker' Settings.cron_jobs['requests_profiles_worker'] ||= Settingslogic.new({}) Settings.cron_jobs['requests_profiles_worker']['cron'] ||= '0 0 * * *' Settings.cron_jobs['requests_profiles_worker']['job_class'] = 'RequestsProfilesWorker' diff --git a/config/initializers/artifacts_direct_upload_support.rb b/config/initializers/artifacts_direct_upload_support.rb deleted file mode 100644 index d2bc35ea613..00000000000 --- a/config/initializers/artifacts_direct_upload_support.rb +++ /dev/null @@ -1,7 +0,0 @@ -artifacts_object_store = Gitlab.config.artifacts.object_store - -if artifacts_object_store.enabled && - artifacts_object_store.direct_upload && - artifacts_object_store.connection&.provider.to_s != 'Google' - raise "Only 'Google' is supported as a object storage provider when 'direct_upload' of artifacts is used" -end diff --git a/config/initializers/direct_upload_support.rb b/config/initializers/direct_upload_support.rb new file mode 100644 index 00000000000..32fc8c8bc69 --- /dev/null +++ b/config/initializers/direct_upload_support.rb @@ -0,0 +1,19 @@ +class DirectUploadsValidator + SUPPORTED_DIRECT_UPLOAD_PROVIDERS = %w(Google AWS).freeze + + ValidationError = Class.new(StandardError) + + def verify!(object_store) + return unless object_store.enabled + return unless object_store.direct_upload + return if SUPPORTED_DIRECT_UPLOAD_PROVIDERS.include?(object_store.connection&.provider.to_s) + + raise ValidationError, "Only #{SUPPORTED_DIRECT_UPLOAD_PROVIDERS.join(',')} are supported as a object storage provider when 'direct_upload' is used" + end +end + +DirectUploadsValidator.new.tap do |validator| + [Gitlab.config.artifacts, Gitlab.config.uploads, Gitlab.config.lfs].each do |uploader| + validator.verify!(uploader.object_store) + end +end diff --git a/config/initializers/lograge.rb b/config/initializers/lograge.rb index 114c1cb512f..1cf8a24e98c 100644 --- a/config/initializers/lograge.rb +++ b/config/initializers/lograge.rb @@ -27,6 +27,7 @@ unless Sidekiq.server? gitaly_calls = Gitlab::GitalyClient.get_request_count payload[:gitaly_calls] = gitaly_calls if gitaly_calls > 0 + payload[:response] = event.payload[:response] if event.payload[:response] payload end diff --git a/config/initializers/postgresql_opclasses_support.rb b/config/initializers/postgresql_opclasses_support.rb index c2f3023b330..03bda44a630 100644 --- a/config/initializers/postgresql_opclasses_support.rb +++ b/config/initializers/postgresql_opclasses_support.rb @@ -107,8 +107,15 @@ module ActiveRecord result.map do |row| index_name = row[0] - unique = row[1] == 't' + unique = if Gitlab.rails5? + row[1] + else + row[1] == 't' + end indkey = row[2].split(" ") + if Gitlab.rails5? + indkey = indkey.map(&:to_i) + end inddef = row[3] oid = row[4] diff --git a/config/routes/api.rb b/config/routes/api.rb index ce7a7c88900..b1aebf4d606 100644 --- a/config/routes/api.rb +++ b/config/routes/api.rb @@ -1,2 +1,7 @@ +constraints(::Constraints::FeatureConstrainer.new(:graphql)) do + post '/api/graphql', to: 'graphql#execute' + mount GraphiQL::Rails::Engine, at: '/-/graphql-explorer', graphql_path: '/api/graphql' +end + API::API.logger Rails.logger mount API::API => '/' diff --git a/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb b/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb index 375e389e07a..7aa79bf5e02 100644 --- a/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb +++ b/db/migrate/20160226114608_add_trigram_indexes_for_searching.rb @@ -37,7 +37,12 @@ class AddTrigramIndexesForSearching < ActiveRecord::Migration res = execute("SELECT true AS enabled FROM pg_available_extensions WHERE name = 'pg_trgm' AND installed_version IS NOT NULL;") row = res.first - row && row['enabled'] == 't' ? true : false + check = if Gitlab.rails5? + true + else + 't' + end + row && row['enabled'] == check ? true : false end def create_trigrams_extension diff --git a/db/migrate/20161207231620_fixup_environment_name_uniqueness.rb b/db/migrate/20161207231620_fixup_environment_name_uniqueness.rb index f73e4f6c99b..1db8c68626a 100644 --- a/db/migrate/20161207231620_fixup_environment_name_uniqueness.rb +++ b/db/migrate/20161207231620_fixup_environment_name_uniqueness.rb @@ -1,4 +1,5 @@ class FixupEnvironmentNameUniqueness < ActiveRecord::Migration + include Gitlab::Database::ArelMethods include Gitlab::Database::MigrationHelpers DOWNTIME = true @@ -41,7 +42,7 @@ class FixupEnvironmentNameUniqueness < ActiveRecord::Migration conflicts.each do |id, name| update_sql = - Arel::UpdateManager.new(ActiveRecord::Base) + arel_update_manager .table(environments) .set(environments[:name] => name + "-" + id.to_s) .where(environments[:id].eq(id)) diff --git a/db/migrate/20161207231626_add_environment_slug.rb b/db/migrate/20161207231626_add_environment_slug.rb index 83cdd484c4c..162f82a01cb 100644 --- a/db/migrate/20161207231626_add_environment_slug.rb +++ b/db/migrate/20161207231626_add_environment_slug.rb @@ -2,6 +2,7 @@ # for more information on how to write migrations for GitLab. class AddEnvironmentSlug < ActiveRecord::Migration + include Gitlab::Database::ArelMethods include Gitlab::Database::MigrationHelpers DOWNTIME = true @@ -19,7 +20,7 @@ class AddEnvironmentSlug < ActiveRecord::Migration finder = environments.project(:id, :name) connection.exec_query(finder.to_sql).rows.each do |id, name| - updater = Arel::UpdateManager.new(ActiveRecord::Base) + updater = arel_update_manager .table(environments) .set(environments[:slug] => generate_slug(name)) .where(environments[:id].eq(id)) diff --git a/db/migrate/20170622135728_add_unique_constraint_to_ci_variables.rb b/db/migrate/20170622135728_add_unique_constraint_to_ci_variables.rb index 8b2cc40ee59..787022b7bfe 100644 --- a/db/migrate/20170622135728_add_unique_constraint_to_ci_variables.rb +++ b/db/migrate/20170622135728_add_unique_constraint_to_ci_variables.rb @@ -2,12 +2,13 @@ class AddUniqueConstraintToCiVariables < ActiveRecord::Migration include Gitlab::Database::MigrationHelpers DOWNTIME = false + INDEX_NAME = 'index_ci_variables_on_project_id_and_key_and_environment_scope' disable_ddl_transaction! def up unless this_index_exists? - add_concurrent_index(:ci_variables, columns, name: index_name, unique: true) + add_concurrent_index(:ci_variables, columns, name: INDEX_NAME, unique: true) end end @@ -18,21 +19,17 @@ class AddUniqueConstraintToCiVariables < ActiveRecord::Migration add_concurrent_index(:ci_variables, :project_id) end - remove_concurrent_index(:ci_variables, columns, name: index_name) + remove_concurrent_index(:ci_variables, columns, name: INDEX_NAME) end end private def this_index_exists? - index_exists?(:ci_variables, columns, name: index_name) + index_exists?(:ci_variables, columns, name: INDEX_NAME) end def columns @columns ||= [:project_id, :key, :environment_scope] end - - def index_name - 'index_ci_variables_on_project_id_and_key_and_environment_scope' - end end diff --git a/db/migrate/20171106155656_turn_issues_due_date_index_to_partial_index.rb b/db/migrate/20171106155656_turn_issues_due_date_index_to_partial_index.rb index e4bed778695..08784de4043 100644 --- a/db/migrate/20171106155656_turn_issues_due_date_index_to_partial_index.rb +++ b/db/migrate/20171106155656_turn_issues_due_date_index_to_partial_index.rb @@ -20,9 +20,7 @@ class TurnIssuesDueDateIndexToPartialIndex < ActiveRecord::Migration name: NEW_INDEX_NAME ) - # We set the column name to nil as otherwise Rails will ignore the custom - # index name and remove the wrong index. - remove_concurrent_index(:issues, nil, name: OLD_INDEX_NAME) + remove_concurrent_index_by_name(:issues, OLD_INDEX_NAME) end def down @@ -32,6 +30,6 @@ class TurnIssuesDueDateIndexToPartialIndex < ActiveRecord::Migration name: OLD_INDEX_NAME ) - remove_concurrent_index(:issues, nil, name: NEW_INDEX_NAME) + remove_concurrent_index_by_name(:issues, NEW_INDEX_NAME) end end diff --git a/db/migrate/20180201110056_add_foreign_keys_to_todos.rb b/db/migrate/20180201110056_add_foreign_keys_to_todos.rb index b7c40f8c01a..020b0550321 100644 --- a/db/migrate/20180201110056_add_foreign_keys_to_todos.rb +++ b/db/migrate/20180201110056_add_foreign_keys_to_todos.rb @@ -31,7 +31,7 @@ class AddForeignKeysToTodos < ActiveRecord::Migration end def down - remove_foreign_key :todos, :users + remove_foreign_key :todos, column: :user_id remove_foreign_key :todos, column: :author_id remove_foreign_key :todos, :notes end diff --git a/db/migrate/20180424160449_add_pipeline_iid_to_ci_pipelines.rb b/db/migrate/20180424160449_add_pipeline_iid_to_ci_pipelines.rb new file mode 100644 index 00000000000..e8f0c91d612 --- /dev/null +++ b/db/migrate/20180424160449_add_pipeline_iid_to_ci_pipelines.rb @@ -0,0 +1,13 @@ +class AddPipelineIidToCiPipelines < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + def up + add_column :ci_pipelines, :iid, :integer + end + + def down + remove_column :ci_pipelines, :iid, :integer + end +end diff --git a/db/migrate/20180425205249_add_index_constraints_to_pipeline_iid.rb b/db/migrate/20180425205249_add_index_constraints_to_pipeline_iid.rb new file mode 100644 index 00000000000..3fa59b44d5d --- /dev/null +++ b/db/migrate/20180425205249_add_index_constraints_to_pipeline_iid.rb @@ -0,0 +1,15 @@ +class AddIndexConstraintsToPipelineIid < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + disable_ddl_transaction! + + def up + add_concurrent_index :ci_pipelines, [:project_id, :iid], unique: true, where: 'iid IS NOT NULL' + end + + def down + remove_concurrent_index :ci_pipelines, [:project_id, :iid] + end +end diff --git a/db/migrate/20180523042841_rename_merge_requests_allow_maintainer_to_push.rb b/db/migrate/20180523042841_rename_merge_requests_allow_maintainer_to_push.rb new file mode 100644 index 00000000000..975bdfe70f4 --- /dev/null +++ b/db/migrate/20180523042841_rename_merge_requests_allow_maintainer_to_push.rb @@ -0,0 +1,15 @@ +class RenameMergeRequestsAllowMaintainerToPush < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + disable_ddl_transaction! + + def up + rename_column_concurrently :merge_requests, :allow_maintainer_to_push, :allow_collaboration + end + + def down + cleanup_concurrent_column_rename :merge_requests, :allow_collaboration, :allow_maintainer_to_push + end +end diff --git a/db/migrate/20180530135500_add_index_to_stages_position.rb b/db/migrate/20180530135500_add_index_to_stages_position.rb new file mode 100644 index 00000000000..61150f33a25 --- /dev/null +++ b/db/migrate/20180530135500_add_index_to_stages_position.rb @@ -0,0 +1,15 @@ +class AddIndexToStagesPosition < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + disable_ddl_transaction! + + def up + add_concurrent_index :ci_stages, [:pipeline_id, :position] + end + + def down + remove_concurrent_index :ci_stages, [:pipeline_id, :position] + end +end diff --git a/db/migrate/20180531220618_change_default_value_for_dsa_key_restriction.rb b/db/migrate/20180531220618_change_default_value_for_dsa_key_restriction.rb new file mode 100644 index 00000000000..d0dcacc5b66 --- /dev/null +++ b/db/migrate/20180531220618_change_default_value_for_dsa_key_restriction.rb @@ -0,0 +1,16 @@ +class ChangeDefaultValueForDsaKeyRestriction < ActiveRecord::Migration + # Set this constant to true if this migration requires downtime. + DOWNTIME = false + + def up + change_column :application_settings, :dsa_key_restriction, :integer, null: false, + default: -1 + + execute("UPDATE application_settings SET dsa_key_restriction = -1") + end + + def down + change_column :application_settings, :dsa_key_restriction, :integer, null: false, + default: 0 + end +end diff --git a/db/post_migrate/20161109150329_fix_project_records_with_invalid_visibility.rb b/db/post_migrate/20161109150329_fix_project_records_with_invalid_visibility.rb index 69007b8e8ed..f058e85c1ec 100644 --- a/db/post_migrate/20161109150329_fix_project_records_with_invalid_visibility.rb +++ b/db/post_migrate/20161109150329_fix_project_records_with_invalid_visibility.rb @@ -1,4 +1,5 @@ class FixProjectRecordsWithInvalidVisibility < ActiveRecord::Migration + include Gitlab::Database::ArelMethods include Gitlab::Database::MigrationHelpers BATCH_SIZE = 500 @@ -33,7 +34,7 @@ class FixProjectRecordsWithInvalidVisibility < ActiveRecord::Migration end updates.each do |visibility_level, project_ids| - updater = Arel::UpdateManager.new(ActiveRecord::Base) + updater = arel_update_manager .table(projects) .set(projects[:visibility_level] => visibility_level) .where(projects[:id].in(project_ids)) diff --git a/db/post_migrate/20170324160416_migrate_user_activities_to_users_last_activity_on.rb b/db/post_migrate/20170324160416_migrate_user_activities_to_users_last_activity_on.rb index 78413a608f1..392fa00b1ba 100644 --- a/db/post_migrate/20170324160416_migrate_user_activities_to_users_last_activity_on.rb +++ b/db/post_migrate/20170324160416_migrate_user_activities_to_users_last_activity_on.rb @@ -1,5 +1,6 @@ # rubocop:disable Migration/UpdateLargeTable class MigrateUserActivitiesToUsersLastActivityOn < ActiveRecord::Migration + include Gitlab::Database::ArelMethods include Gitlab::Database::MigrationHelpers disable_ddl_transaction! @@ -39,7 +40,7 @@ class MigrateUserActivitiesToUsersLastActivityOn < ActiveRecord::Migration activities = activities(day.at_beginning_of_day, day.at_end_of_day, page: page) update_sql = - Arel::UpdateManager.new(ActiveRecord::Base) + arel_update_manager .table(users_table) .set(users_table[:last_activity_on] => day.to_date) .where(users_table[:username].in(activities.map(&:first))) diff --git a/db/post_migrate/20180523125103_cleanup_merge_requests_allow_maintainer_to_push_rename.rb b/db/post_migrate/20180523125103_cleanup_merge_requests_allow_maintainer_to_push_rename.rb new file mode 100644 index 00000000000..b9ce4600675 --- /dev/null +++ b/db/post_migrate/20180523125103_cleanup_merge_requests_allow_maintainer_to_push_rename.rb @@ -0,0 +1,15 @@ +class CleanupMergeRequestsAllowMaintainerToPushRename < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + disable_ddl_transaction! + + def up + cleanup_concurrent_column_rename :merge_requests, :allow_maintainer_to_push, :allow_collaboration + end + + def down + rename_column_concurrently :merge_requests, :allow_collaboration, :allow_maintainer_to_push + end +end diff --git a/db/post_migrate/20180529152628_schedule_to_archive_legacy_traces.rb b/db/post_migrate/20180529152628_schedule_to_archive_legacy_traces.rb new file mode 100644 index 00000000000..965cd3a8714 --- /dev/null +++ b/db/post_migrate/20180529152628_schedule_to_archive_legacy_traces.rb @@ -0,0 +1,35 @@ +class ScheduleToArchiveLegacyTraces < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + BATCH_SIZE = 5000 + BACKGROUND_MIGRATION_CLASS = 'ArchiveLegacyTraces' + + disable_ddl_transaction! + + class Build < ActiveRecord::Base + include EachBatch + self.table_name = 'ci_builds' + self.inheritance_column = :_type_disabled # Disable STI + + scope :type_build, -> { where(type: 'Ci::Build') } + + scope :finished, -> { where(status: [:success, :failed, :canceled]) } + + scope :without_archived_trace, -> do + where('NOT EXISTS (SELECT 1 FROM ci_job_artifacts WHERE ci_builds.id = ci_job_artifacts.job_id AND ci_job_artifacts.file_type = 3)') + end + end + + def up + queue_background_migration_jobs_by_range_at_intervals( + ::ScheduleToArchiveLegacyTraces::Build.type_build.finished.without_archived_trace, + BACKGROUND_MIGRATION_CLASS, + 5.minutes, + batch_size: BATCH_SIZE) + end + + def down + # noop + end +end diff --git a/db/post_migrate/20180603190921_migrate_object_storage_upload_sidekiq_queue.rb b/db/post_migrate/20180603190921_migrate_object_storage_upload_sidekiq_queue.rb new file mode 100644 index 00000000000..57bee6269b9 --- /dev/null +++ b/db/post_migrate/20180603190921_migrate_object_storage_upload_sidekiq_queue.rb @@ -0,0 +1,16 @@ +class MigrateObjectStorageUploadSidekiqQueue < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + DOWNTIME = false + + disable_ddl_transaction! + + def up + sidekiq_queue_migrate 'object_storage_upload', to: 'object_storage:object_storage_background_move' + end + + def down + # do not migrate any jobs back because we would migrate also + # jobs which were not part of the 'object_storage_upload' + end +end diff --git a/db/schema.rb b/db/schema.rb index 97247387bc7..f6fb1c92f8d 100644 --- a/db/schema.rb +++ b/db/schema.rb @@ -11,7 +11,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema.define(version: 20180529093006) do +ActiveRecord::Schema.define(version: 20180603190921) do # These are extensions that must be enabled in order to support this database enable_extension "plpgsql" @@ -110,7 +110,7 @@ ActiveRecord::Schema.define(version: 20180529093006) do t.text "shared_runners_text_html" t.text "after_sign_up_text_html" t.integer "rsa_key_restriction", default: 0, null: false - t.integer "dsa_key_restriction", default: 0, null: false + t.integer "dsa_key_restriction", default: -1, null: false t.integer "ecdsa_key_restriction", default: 0, null: false t.integer "ed25519_key_restriction", default: 0, null: false t.boolean "housekeeping_enabled", default: true, null: false @@ -451,10 +451,12 @@ ActiveRecord::Schema.define(version: 20180529093006) do t.integer "config_source" t.boolean "protected" t.integer "failure_reason" + t.integer "iid" end add_index "ci_pipelines", ["auto_canceled_by_id"], name: "index_ci_pipelines_on_auto_canceled_by_id", using: :btree add_index "ci_pipelines", ["pipeline_schedule_id"], name: "index_ci_pipelines_on_pipeline_schedule_id", using: :btree + add_index "ci_pipelines", ["project_id", "iid"], name: "index_ci_pipelines_on_project_id_and_iid", unique: true, where: "(iid IS NOT NULL)", using: :btree add_index "ci_pipelines", ["project_id", "ref", "status", "id"], name: "index_ci_pipelines_on_project_id_and_ref_and_status_and_id", using: :btree add_index "ci_pipelines", ["project_id", "sha"], name: "index_ci_pipelines_on_project_id_and_sha", using: :btree add_index "ci_pipelines", ["project_id"], name: "index_ci_pipelines_on_project_id", using: :btree @@ -518,6 +520,7 @@ ActiveRecord::Schema.define(version: 20180529093006) do end add_index "ci_stages", ["pipeline_id", "name"], name: "index_ci_stages_on_pipeline_id_and_name", unique: true, using: :btree + add_index "ci_stages", ["pipeline_id", "position"], name: "index_ci_stages_on_pipeline_id_and_position", using: :btree add_index "ci_stages", ["pipeline_id"], name: "index_ci_stages_on_pipeline_id", using: :btree add_index "ci_stages", ["project_id"], name: "index_ci_stages_on_project_id", using: :btree @@ -1227,7 +1230,7 @@ ActiveRecord::Schema.define(version: 20180529093006) do t.boolean "discussion_locked" t.integer "latest_merge_request_diff_id" t.string "rebase_commit_sha" - t.boolean "allow_maintainer_to_push" + t.boolean "allow_collaboration" t.boolean "squash", default: false, null: false end diff --git a/doc/README.md b/doc/README.md index ff8dd3fab8a..fee920f2012 100644 --- a/doc/README.md +++ b/doc/README.md @@ -162,7 +162,7 @@ configuration. Then customize everything from buildpacks to CI/CD. - [Auto DevOps](topics/autodevops/index.md) - [Deployment of Helm, Ingress, and Prometheus on Kubernetes](user/project/clusters/index.md#installing-applications) -- [Protected secret variables](ci/variables/README.md#protected-secret-variables) +- [Protected variables](ci/variables/README.md#protected-variables) - [Easy creation of Kubernetes clusters on GKE](user/project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) ### Monitor @@ -191,7 +191,7 @@ instant how code changes impact your production environment. - [User account](user/profile/index.md): Manage your account - [Authentication](topics/authentication/index.md): Account security with two-factor authentication, setup your ssh keys and deploy keys for secure access to your projects. - [Profile settings](user/profile/index.md#profile-settings): Manage your profile settings, two factor authentication and more. -- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. +- [User permissions](user/permissions.md): Learn what each role in a project (external/guest/reporter/developer/maintainer/owner) can do. ### Git and GitLab @@ -215,7 +215,7 @@ Learn how to contribute to GitLab: - [Development](development/README.md): All styleguides and explanations how to contribute. - [Legal](legal/README.md): Contributor license agreements. -- [Writing documentation](development/writing_documentation.md): Contributing to GitLab Docs. +- [Writing documentation](development/documentation/index.md): Contributing to GitLab Docs. ## GitLab subscriptions diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 32ad63c3706..e11ed58eb91 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -2,7 +2,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7690) -in GitLab 8.15. Only project masters and owners can access web terminals. +in GitLab 8.15. Only project maintainers and owners can access web terminals. With the introduction of the [Kubernetes integration](../../user/project/clusters/index.md), GitLab gained the ability to store and use credentials for a Kubernetes cluster. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 77fe4d561a1..e59ab5a72e1 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -94,6 +94,7 @@ _The artifacts are stored by default in > Available in [GitLab Premium](https://about.gitlab.com/products/) and [GitLab.com Silver](https://about.gitlab.com/gitlab-com/). > Since version 10.6, available in [GitLab CE](https://about.gitlab.com/products/) +> Since version 11.0, we support direct_upload to S3. If you don't want to use the local disk where GitLab is installed to store the artifacts, you can use an object storage like AWS S3 instead. @@ -108,7 +109,7 @@ For source installations the following settings are nested under `artifacts:` an |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | | `remote_directory` | The bucket name where Artifacts will be stored| | -| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. Currently only `Google` provider is supported | `false` | +| `direct_upload` | Set to true to enable direct upload of Artifacts without the need of local shared storage. Option may be removed once we decide to support only single storage for all files. | `false` | | `background_upload` | Set to false to disable automatic upload. Option may be removed once upload is direct to S3 | `true` | | `proxy_download` | Set to true to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | diff --git a/doc/api/access_requests.md b/doc/api/access_requests.md index 603fa4a8194..4b2014ca843 100644 --- a/doc/api/access_requests.md +++ b/doc/api/access_requests.md @@ -10,7 +10,7 @@ 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/avatar.md b/doc/api/avatar.md new file mode 100644 index 00000000000..7faed893066 --- /dev/null +++ b/doc/api/avatar.md @@ -0,0 +1,33 @@ +# Avatar API + +> [Introduced][ce-19121] in GitLab 11.0 + +## Get a single avatar URL + +Get a single avatar URL for a given email addres. If user with matching public +email address is not found, results from external avatar services are returned. +This endpoint can be accessed without authentication. In case public visibility +is restricted, response will be `403 Forbidden` when unauthenticated. + +``` +GET /avatar?email=admin@example.com +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `email` | string | yes | Public email address of the user | +| `size` | integer | no | Single pixel dimension (since images are squares). Only used for avatar lookups at `Gravatar` or at the configured `Libravatar` server | + +```bash +curl https://gitlab.example.com/api/v4/avatar?email=admin@example.com +``` + +Example response: + +```json +{ + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon" +} +``` + +[ce-19121]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19121 diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md new file mode 100644 index 00000000000..dcd5377284c --- /dev/null +++ b/doc/api/graphql/index.md @@ -0,0 +1,42 @@ +# GraphQL API (Beta) + +> [Introduced][ce-19008] in GitLab 11.0. + +[GraphQL](https://graphql.org/) is a query language for APIs that +allows clients to request exactly the data they need, making it +possible to get all required data in a limited number of requests. + +The GraphQL data (fields) can be described in the form of types, +allowing clients to use [clientside GraphQL +libraries](https://graphql.org/code/#graphql-clients) to consume the +API and avoid manual parsing. + +Since there's no fixed endpoints and datamodel, new abilities can be +added to the API without creating breaking changes. This allows us to +have a versionless API as described in [the GraphQL +documentation](https://graphql.org/learn/best-practices/#versioning). + +## Enabling the GraphQL feature + +The GraphQL API itself is currently in Alpha, and therefore hidden behind a +feature flag. You can enable the feature using the [features api][features-api] on a self-hosted instance. + +For example: + +```shell +curl --data "value=100" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/features/graphql +``` + +## Available queries + +A first iteration of a GraphQL API includes only 2 queries: `project` and +`merge_request` and only returns scalar fields, or fields of the type `Project` +or `MergeRequest`. + +## GraphiQL + +The API can be explored by using the GraphiQL IDE, it is available on your +instance on `gitlab.example.com/-/graphql-explorer`. + +[ce-19008]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19008 +[features-api]: ../features.md diff --git a/doc/api/members.md b/doc/api/members.md index 3234f833eae..1a10aa75ac0 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -8,7 +8,7 @@ The access levels are defined in the `Gitlab::Access` module. Currently, these l 10 => Guest access 20 => Reporter access 30 => Developer access -40 => Master access +40 => Maintainer access 50 => Owner access # Only valid for groups ``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 62f9884e264..9f06e20f803 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -651,7 +651,8 @@ POST /projects/:id/merge_requests | `labels` | string | no | Labels for MR as a comma-separated list | | `milestone_id` | integer | no | The global ID of a milestone | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | | `squash` | boolean | no | Squash commits into a single commit when merging | ```json @@ -709,6 +710,7 @@ POST /projects/:id/merge_requests "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, @@ -741,7 +743,8 @@ PUT /projects/:id/merge_requests/:merge_request_iid | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | | `squash` | boolean | no | Squash commits into a single commit when merging | | `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | -| `allow_maintainer_to_push` | boolean | no | Whether or not a maintainer of the target project can push to the source branch | +| `allow_collaboration` | boolean | no | Allow commits from members who can merge to the target branch | +| `allow_maintainer_to_push` | boolean | no | Deprecated, see allow_collaboration | Must include at least one non-required attribute from above. @@ -799,6 +802,7 @@ Must include at least one non-required attribute from above. "squash": false, "web_url": "http://example.com/example/example/merge_requests/1", "discussion_locked": false, + "allow_collaboration": false, "allow_maintainer_to_push": false, "time_stats": { "time_estimate": 0, diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 1f0dc700640..656bf07bb55 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -54,7 +54,7 @@ Example response: ] ``` -**Note**: `members_count_with_descendants` are presented only for group masters/owners. +**Note**: `members_count_with_descendants` are presented only for group maintainers/owners. ## Search for namespace diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 899f5da6647..ebae68fe389 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -102,6 +102,7 @@ POST /projects/:id/pipeline |------------|---------|----------|---------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `ref` | string | yes | Reference to commit | +| `variables` | array | no | An array containing the variables available in the pipeline, matching the structure [{ 'key' => 'UPLOAD_TO_S3', 'value' => 'true' }] | ``` curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipeline?ref=master" diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 950ead52560..f6813f27dc0 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -8,7 +8,7 @@ The access levels are defined in the `ProtectedRefAccess::ALLOWED_ACCESS_LEVELS` ``` 0 => No access 30 => Developer access -40 => Master access +40 => Maintainer access ``` ## List protected branches @@ -36,13 +36,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] }, @@ -75,13 +75,13 @@ Example response: "push_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ], "merge_access_levels": [ { "access_level": 40, - "access_level_description": "Masters" + "access_level_description": "Maintainers" } ] } @@ -104,8 +104,8 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" 'https://gitl | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `name` | string | yes | The name of the branch or wildcard | -| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, master access level) | -| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, master access level) | +| `push_access_level` | string | no | Access levels allowed to push (defaults: `40`, maintainer access level) | +| `merge_access_level` | string | no | Access levels allowed to merge (defaults: `40`, maintainer access level) | Example response: @@ -115,13 +115,13 @@ Example response: "push_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ], "merge_access_levels": [ { "access_level": 30, - "access_level_description": "Developers + Masters" + "access_level_description": "Developers + Maintainers" } ] } diff --git a/doc/api/services.md b/doc/api/services.md index f23303ef836..aeb48ccc36c 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -1,6 +1,6 @@ # Services API ->**Note:** This API requires an access token with Master or Owner permissions +>**Note:** This API requires an access token with Maintainer or Owner permissions ## Asana diff --git a/doc/api/settings.md b/doc/api/settings.md index 36a0782d8f2..e6b207d8746 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -81,7 +81,7 @@ PUT /application/settings | `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and masters can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but masters can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and maintainers can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and maintainers can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but maintainers can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | | `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | | `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | diff --git a/doc/articles/index.md b/doc/articles/index.md index 9f85533ea94..87ee17bb6de 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -4,7 +4,7 @@ comments: false # Technical articles list (deprecated) -[Technical articles](../development/writing_documentation.md#technical-articles) are +[Technical articles](../development/documentation/index.md#technical-articles) are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 7c0f837ea9c..71f1d69cdf4 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -496,7 +496,7 @@ To configure access for `registry.example.com`, follow these steps: bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ= ``` -1. Create a [secret variable] `DOCKER_AUTH_CONFIG` with the content of the +1. Create a [variable] `DOCKER_AUTH_CONFIG` with the content of the Docker configuration file as the value: ```json @@ -632,7 +632,7 @@ creation. [postgres-hub]: https://hub.docker.com/r/_/postgres/ [mysql-hub]: https://hub.docker.com/r/_/mysql/ [runner-priv-reg]: http://docs.gitlab.com/runner/configuration/advanced-configuration.html#using-a-private-container-registry -[secret variable]: ../variables/README.md#secret-variables +[variable]: ../variables/README.md#variables [entrypoint]: https://docs.docker.com/engine/reference/builder/#entrypoint [cmd]: https://docs.docker.com/engine/reference/builder/#cmd [register]: https://docs.gitlab.com/runner/register/ diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 7f034409580..36fd8affa5b 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -249,7 +249,7 @@ the basis of [Review apps](review_apps/index.md). NOTE: **Note:** The `name` and `url` parameters can use most of the CI/CD variables, including [predefined](variables/README.md#predefined-variables-environment-variables), -[secret](variables/README.md#secret-variables) and +[project/group ones](variables/README.md#variables) and [`.gitlab-ci.yml` variables](yaml/README.md#variables). You however cannot use variables defined under `script` or on the Runner's side. There are also other variables that are unsupported in the context of `environment:name`. You can read more about @@ -593,7 +593,7 @@ version of the app, all without leaving GitLab. >**Note:** Web terminals were added in GitLab 8.15 and are only available to project -masters and owners. +maintainers and owners. If you deploy to your environments with the help of a deployment service (e.g., the [Kubernetes integration][kube]), GitLab can open diff --git a/doc/ci/examples/artifactory_and_gitlab/index.md b/doc/ci/examples/artifactory_and_gitlab/index.md index d931c9a77f4..9657f52159e 100644 --- a/doc/ci/examples/artifactory_and_gitlab/index.md +++ b/doc/ci/examples/artifactory_and_gitlab/index.md @@ -58,7 +58,7 @@ The application is ready to use, but you need some additional steps to deploy it 1. Log in to Artifactory with your user's credentials. 1. From the main screen, click on the `libs-release-local` item in the **Set Me Up** panel. 1. Copy to clipboard the configuration snippet under the **Deploy** paragraph. -1. Change the `url` value in order to have it configurable via secret variables. +1. Change the `url` value in order to have it configurable via variables. 1. Copy the snippet in the `pom.xml` file for your project, just after the `dependencies` section. The snippet should look like this: @@ -98,7 +98,7 @@ parameter in `.gitlab-ci.yml` to use the custom location instead of the default </settings> ``` - Username and password will be replaced by the correct values using secret variables. + Username and password will be replaced by the correct values using variables. ### Configure GitLab CI/CD for `simple-maven-dep` @@ -107,8 +107,8 @@ Now it's time we set up [GitLab CI/CD](https://about.gitlab.com/features/gitlab- GitLab CI/CD uses a file in the root of the repo, named `.gitlab-ci.yml`, to read the definitions for jobs that will be executed by the configured GitLab Runners. You can read more about this file in the [GitLab Documentation](https://docs.gitlab.com/ee/ci/yaml/). -First of all, remember to set up secret variables for your deployment. Navigate to your project's **Settings > CI/CD** page -and add the following secret variables (replace them with your current values, of course): +First of all, remember to set up variables for your deployment. Navigate to your project's **Settings > CI/CD > Variables** page +and add the following ones (replace them with your current values, of course): - **MAVEN_REPO_URL**: `http://artifactory.example.com:8081/artifactory` (your Artifactory URL) - **MAVEN_REPO_USER**: `gitlab` (your Artifactory username) @@ -156,7 +156,7 @@ by running all Maven phases in a sequential order, therefore, executing `mvn tes Both `build` and `test` jobs leverage the `mvn` command to compile the application and to test it as defined in the test suite that is part of the application. -Deploy to Artifactory is done as defined by the secret variables we have just set up. +Deploy to Artifactory is done as defined by the variables we have just set up. The deployment occurs only if we're pushing or merging to `master` branch, so that the development versions are tested but not published. Done! Now you have all the changes in the GitLab repo, and a pipeline has already been started for this commit. In the **Pipelines** tab you can see what's happening. diff --git a/doc/ci/examples/deployment/README.md b/doc/ci/examples/deployment/README.md index 2dcdc2d41ec..bd60d641493 100644 --- a/doc/ci/examples/deployment/README.md +++ b/doc/ci/examples/deployment/README.md @@ -111,7 +111,7 @@ We also use two secure variables: ## Storing API keys Secure Variables can added by going to your project's -**Settings âž” CI / CD âž” Secret variables**. The variables that are defined +**Settings âž” CI / CD âž” Variables**. The variables that are defined in the project settings are sent along with the build script to the Runner. The secure variables are stored out of the repository. Never store secrets in your project's `.gitlab-ci.yml`. It is also important that the secret's value diff --git a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md index 3d21c0cc306..c226b5bfb71 100644 --- a/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md +++ b/doc/ci/examples/devops_and_game_dev_with_gitlab_ci_cd/index.md @@ -406,7 +406,7 @@ and further delves into the principles of GitLab CI/CD than discussed in this ar We need to be able to deploy to AWS with our AWS account credentials, but we certainly don't want to put secrets into source code. Luckily GitLab provides a solution for this -with [Secret Variables](../../../ci/variables/README.md). This can get complicated +with [Variables](../../../ci/variables/README.md). This can get complicated due to [IAM](https://aws.amazon.com/iam/) management. As a best practice, you shouldn't use root security credentials. Proper IAM credential management is beyond the scope of this article, but AWS will remind you that using root credentials is unadvised and against their @@ -428,7 +428,7 @@ fully understand [IAM Best Practices in AWS](http://docs.aws.amazon.com/IAM/late To deploy our build artifacts, we need to install the [AWS CLI](https://aws.amazon.com/cli/) on the Shared Runner. The Shared Runner also needs to be able to authenticate with your AWS account to deploy the artifacts. By convention, AWS CLI will look for `AWS_ACCESS_KEY_ID` -and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the secret variables we +and `AWS_SECRET_ACCESS_KEY`. GitLab's CI gives us a way to pass the variables we set up in the prior section using the `variables` portion of the `deploy` job. At the end, we add directives to ensure deployment `only` happens on pushes to `master`. This way, every single branch still runs through CI, and only merging (or committing directly) to master will diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index 1f9b9d53fc1..39c65399332 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -116,11 +116,11 @@ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys cat ~/.ssh/id_rsa ``` -Now, let's add it to your GitLab project as a [secret variable](../../variables/README.md#secret-variables). -Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +Now, let's add it to your GitLab project as a [variable](../../variables/README.md#variables). +Variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. They can be added per project by navigating to the project's **Settings** > **CI/CD**. - + To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index b16cbc61d14..4e964af97f5 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -258,7 +258,7 @@ on that specific branch: - trigger **manual actions** on existing pipelines - **retry/cancel** existing jobs (using Web UI or Pipelines API) -**Secret variables** marked as **protected** are accessible only to jobs that +**Variables** marked as **protected** are accessible only to jobs that run on protected branches, avoiding untrusted users to get unintended access to sensitive information like deployment credentials and tokens. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 703a7f030ed..8f1ff190804 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -84,7 +84,7 @@ visit the project you want to make the Runner work for in GitLab: ## Registering a group Runner -Creating a group Runner requires Master permissions for the group. To create a +Creating a group Runner requires Maintainer permissions for the group. To create a group Runner visit the group you want to make the Runner work for in GitLab: 1. Go to **Settings > CI/CD** to obtain the token @@ -120,9 +120,9 @@ To lock/unlock a Runner: ## Assigning a Runner to another project -If you are Master on a project where a specific Runner is assigned to, and the +If you are Maintainer on a project where a specific Runner is assigned to, and the Runner is not [locked only to that project](#locking-a-specific-runner-from-being-enabled-for-other-projects), -you can enable the Runner also on any other project where you have Master permissions. +you can enable the Runner also on any other project where you have Maintainer permissions. To enable/disable a Runner in your project: @@ -132,7 +132,7 @@ To enable/disable a Runner in your project: > **Note**: Consider that if you don't lock your specific Runner to a specific project, any -user with Master role in you project can assign your Runner to another arbitrary +user with Maintainer role in you project can assign your Runner to another arbitrary project without requiring your authorization, so use it with caution. An admin can enable/disable a specific Runner for projects: diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 693c8e9ef18..4cb05509e7b 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -25,7 +25,7 @@ with any type of [executor](https://docs.gitlab.com/runner/executors/) ## How it works 1. Create a new SSH key pair locally with [ssh-keygen](http://linux.die.net/man/1/ssh-keygen) -1. Add the private key as a [secret variable](../variables/README.md) to +1. Add the private key as a [variable](../variables/README.md) to your project 1. Run the [ssh-agent](http://linux.die.net/man/1/ssh-agent) during job to load the private key. @@ -49,7 +49,7 @@ to access it. This is where an SSH key pair comes in handy. **Do not** add a passphrase to the SSH key, or the `before_script` will\ prompt for it. -1. Create a new [secret variable](../variables/README.md#secret-variables). +1. Create a new [variable](../variables/README.md#variables). As **Key** enter the name `SSH_PRIVATE_KEY` and in the **Value** field paste the content of your _private_ key that you created earlier. @@ -157,7 +157,7 @@ ssh-keyscan example.com ssh-keyscan 1.2.3.4 ``` -Create a new [secret variable](../variables/README.md#secret-variables) with +Create a new [variable](../variables/README.md#variables) with `SSH_KNOWN_HOSTS` as "Key", and as a "Value" add the output of `ssh-keyscan`. NOTE: **Note:** @@ -165,7 +165,7 @@ If you need to connect to multiple servers, all the server host keys need to be collected in the **Value** of the variable, one key per line. TIP: **Tip:** -By using a secret variable instead of `ssh-keyscan` directly inside +By using a variable instead of `ssh-keyscan` directly inside `.gitlab-ci.yml`, it has the benefit that you don't have to change `.gitlab-ci.yml` if the host domain name changes for some reason. Also, the values are predefined by you, meaning that if the host keys suddenly change, the CI/CD job will fail, diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 47a576fdf5f..c507036aa6a 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -53,7 +53,7 @@ The action is irreversible. it will not trigger a job. - If your project is public, passing the token in plain text is probably not the wisest idea, so you might want to use a - [secret variable](../variables/README.md#secret-variables) for that purpose. + [variable](../variables/README.md#variables) for that purpose. To trigger a job you need to send a `POST` request to GitLab's API endpoint: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index f10423b92cf..1b24bcdbf6f 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -10,17 +10,17 @@ The variables can be overwritten and they take precedence over each other in this order: 1. [Trigger variables][triggers] or [scheduled pipeline variables](../../user/project/pipelines/schedules.md#making-use-of-scheduled-pipeline-variables) (take precedence over all) -1. Project-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) -1. Group-level [secret variables](#secret-variables) or [protected secret variables](#protected-secret-variables) +1. Project-level [variables](#variables) or [protected variables](#protected-variables) +1. Group-level [variables](#variables) or [protected variables](#protected-variables) 1. YAML-defined [job-level variables](../yaml/README.md#variables) 1. YAML-defined [global variables](../yaml/README.md#variables) 1. [Deployment variables](#deployment-variables) 1. [Predefined variables](#predefined-variables-environment-variables) (are the lowest in the chain) -For example, if you define `API_TOKEN=secure` as a secret variable and +For example, if you define `API_TOKEN=secure` as a project variable and `API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value -`secure` as the secret variables are higher in the chain. +`secure` as the project variables are higher in the chain. ## Unsupported variables @@ -72,6 +72,7 @@ future GitLab releases.** | **CI_RUNNER_REVISION** | all | 10.6 | GitLab Runner revision that is executing the current job | | **CI_RUNNER_EXECUTABLE_ARCH** | all | 10.6 | The OS/architecture of the GitLab Runner executable (note that this is not necessarily the same as the environment of the executor) | | **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | +| **CI_PIPELINE_IID** | 11.0 | all | The unique id of the current pipeline scoped to project | | **CI_PIPELINE_TRIGGERED** | all | all | The flag to indicate that job was [triggered] | | **CI_PIPELINE_SOURCE** | 10.0 | all | Indicates how the pipeline was triggered. Possible options are: `push`, `web`, `trigger`, `schedule`, `api`, and `pipeline`. For pipelines created before GitLab 9.5, this will show as `unknown` | | **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the job is run | @@ -164,49 +165,49 @@ script: - 'eval $LS_CMD' # will execute 'ls -al $TMP_DIR' ``` -## Secret variables +## Variables NOTE: **Note:** -Group-level secret variables were added in GitLab 9.4. +Group-level variables were added in GitLab 9.4. CAUTION: **Important:** -Be aware that secret variables are not masked, and their values can be shown +Be aware that variables are not masked, and their values can be shown in the job logs if explicitly asked to do so. If your project is public or internal, you can set the pipelines private from your [project's Pipelines settings](../../user/project/pipelines/settings.md#visibility-of-pipelines). -Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. +Follow the discussion in issue [#13784][ce-13784] for masking the variables. -GitLab CI allows you to define per-project or per-group secret variables -that are set in the pipeline environment. The secret variables are stored out of +GitLab CI allows you to define per-project or per-group variables +that are set in the pipeline environment. The variables are stored out of the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner making them available during a pipeline run. It's the recommended method to use for storing things like passwords, SSH keys and credentials. -Project-level secret variables can be added by going to your project's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Project-level variables can be added by going to your project's +**Settings > CI/CD**, then finding the section called **Variables**. -Likewise, group-level secret variables can be added by going to your group's -**Settings > CI/CD**, then finding the section called **Secret variables**. +Likewise, group-level variables can be added by going to your group's +**Settings > CI/CD**, then finding the section called **Variables**. Any variables of [subgroups] will be inherited recursively. - + Once you set them, they will be available for all subsequent pipelines. You can also -[protect your variables](#protected-secret-variables). +[protect your variables](#protected-variables). -### Protected secret variables +### Protected variables >**Notes:** This feature requires GitLab 9.3 or higher. -Secret variables could be protected. Whenever a secret variable is +Variables could be protected. Whenever a variable is protected, it would only be securely passed to pipelines running on the [protected branches] or [protected tags]. The other pipelines would not get any protected variables. Protected variables can be added by going to your project's **Settings > CI/CD**, then finding the section called -**Secret variables**, and check "Protected". +**Variables**, and check "Protected". Once you set them, they will be available for all subsequent pipelines. @@ -230,7 +231,7 @@ An example project service that defines deployment variables is the CAUTION: **Warning:** Enabling debug tracing can have severe security implications. The -output **will** contain the content of all your secret variables and any other +output **will** contain the content of all your variables and any other secrets! The output **will** be uploaded to the GitLab server and made visible in job traces! @@ -352,6 +353,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ CI_PROJECT_URL=https://example.com/gitlab-examples/ci-debug-trace ++ export CI_PIPELINE_ID=52666 ++ CI_PIPELINE_ID=52666 +++ export CI_PIPELINE_IID=123 +++ CI_PIPELINE_IID=123 ++ export CI_RUNNER_ID=1337 ++ CI_RUNNER_ID=1337 ++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com @@ -416,7 +419,7 @@ job_name: ``` You can also list all environment variables with the `export` command, -but be aware that this will also expose the values of all the secret variables +but be aware that this will also expose the values of all the variables you set, in the job log: ```yaml @@ -439,6 +442,7 @@ export CI_JOB_MANUAL="true" export CI_JOB_TRIGGERED="true" export CI_JOB_TOKEN="abcde-1234ABCD5678ef" export CI_PIPELINE_ID="1000" +export CI_PIPELINE_IID="10" export CI_PROJECT_ID="34" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_NAME="gitlab-ce" @@ -468,7 +472,7 @@ It is possible to use variables expressions with only / except policies in `.gitlab-ci.yml`. By using this approach you can limit what jobs are going to be created within a pipeline after pushing a code to GitLab. -This is particularly useful in combination with secret variables and triggered +This is particularly useful in combination with variables and triggered pipeline variables. ```yaml @@ -546,7 +550,7 @@ Below you can find supported syntax reference: Pattern matching is case-sensitive by default. Use `i` flag modifier, like `/pattern/i` to make a pattern case-insensitive. -[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI variables" [eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 9800784d918..b2b4a26bdda 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -17,7 +17,7 @@ There are basically two places where you can use any defined variables: | Definition | Can be expanded? | Expansion place | Description | |--------------------------------------|-------------------|-----------------|--------------| -| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (secret variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | +| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> | | `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion **doesn't support**: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> | | `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | | `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) | @@ -55,7 +55,7 @@ since the expansion is done in GitLab before any Runner will get the job. ### GitLab Runner internal variable expansion mechanism -- **Supported:** secret variables, `.gitlab-ci.yml` variables, `config.toml` variables, and +- **Supported:** project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules - **Not supported:** variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`) @@ -76,7 +76,7 @@ are using a different variables syntax. **Supported:** - The `script` may use all available variables that are default for the shell (e.g., `$PATH` which - should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (secret variables, + should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (project/group variables, `.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules). - The `script` may also use all variables defined in the lines before. So, for example, if you define a variable `export MY_VARIABLE="test"`: diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 3e77a6f58b7..f946536701e 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -327,7 +327,7 @@ Refs strategy equals to simplified only/except configuration, whereas kubernetes strategy accepts only `active` keyword. `variables` keyword is used to define variables expressions. In other words -you can use predefined variables / secret variables / project / group or +you can use predefined variables / project / group or environment-scoped variables to define an expression GitLab is going to evaluate in order to decide whether a job should be created or not. @@ -1249,7 +1249,7 @@ Runner itself](../variables/README.md#predefined-variables-environment-variables One example would be `CI_COMMIT_REF_NAME` which has the value of the branch or tag name for which project is built. Apart from the variables you can set in `.gitlab-ci.yml`, there are also the so called -[secret variables](../variables/README.md#secret-variables) +[Variables](../variables/README.md#variables) which can be set in GitLab's UI. [Learn more about variables and their priority.][variables] diff --git a/doc/development/README.md b/doc/development/README.md index 898c60e96c0..92d9829192e 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -32,6 +32,8 @@ description: 'Learn how to contribute to GitLab.' - [GitLab utilities](utilities.md) - [API styleguide](api_styleguide.md) Use this styleguide if you are contributing to the API. +- [GrapQL API styleguide](api_graphql_styleguide.md) Use this + styleguide if you are contribution to the [GraphQL API](../api/graphql/index.md) - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers - [Working with Gitaly](gitaly.md) - [Manage feature flags](feature_flags.md) @@ -86,8 +88,8 @@ description: 'Learn how to contribute to GitLab.' ## Documentation guides -- [Writing documentation](writing_documentation.md) -- [Documentation styleguide](doc_styleguide.md) +- [Writing documentation](documentation/index.md) +- [Documentation styleguide](documentation/styleguide.md) - [Markdown](../user/markdown.md) ## Internationalization (i18n) guides diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md new file mode 100644 index 00000000000..f74e4f0bd7e --- /dev/null +++ b/doc/development/api_graphql_styleguide.md @@ -0,0 +1,81 @@ +# GraphQL API + +## Authentication + +Authentication happens through the `GraphqlController`, right now this +uses the same authentication as the Rails application. So the session +can be shared. + +It is also possible to add a `private_token` to the querystring, or +add a `HTTP_PRIVATE_TOKEN` header. + +### Authorization + +Fields can be authorized using the same abilities used in the Rails +app. This can be done using the `authorize` helper: + +```ruby +module Types + class QueryType < BaseObject + graphql_name 'Query' + + field :project, Types::ProjectType, null: true, resolver: Resolvers::ProjectResolver do + authorize :read_project + end + end +``` + +The object found by the resolve call is used for authorization. + +This works for authorizing a single record, for authorizing +collections, we should only load what the currently authenticated user +is allowed to view. Preferably we use our existing finders for that. + +## Types + +When exposing a model through the GraphQL API, we do so by creating a +new type in `app/graphql/types`. + +When exposing properties in a type, make sure to keep the logic inside +the definition as minimal as possible. Instead, consider moving any +logic into a presenter: + +```ruby +class Types::MergeRequestType < BaseObject + present_using MergeRequestPresenter + + name 'MergeRequest' +end +``` + +An existing presenter could be used, but it is also possible to create +a new presenter specifically for GraphQL. + +The presenter is initialized using the object resolved by a field, and +the context. + +## Resolvers + +To find objects to display in a field, we can add resolvers to +`app/graphql/resolvers`. + +Arguments can be defined within the resolver, those arguments will be +made available to the fields using the resolver. + +We already have a `FullPathLoader` that can be included in other +resolvers to quickly find Projects and Namespaces which will have a +lot of dependant objects. + +To limit the amount of queries performed, we can use `BatchLoader`. + +## Testing + +_full stack_ tests for a graphql query or mutation live in +`spec/requests/api/graphql`. + +When adding a query, the `a working graphql query` shared example can +be used to test if the query renders valid results. + +Using the `GraphqlHelpers#all_graphql_fields_for`-helper, a query +including all available fields can be constructed. This makes it easy +to add a test rendering all possible fields for a query. diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 5d595c33915..7d84f8ca86a 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -1,499 +1,3 @@ --- -description: 'Writing styles, markup, formatting, and reusing regular expressions throughout the GitLab Documentation.' +redirect_to: 'documentation/styleguide.md' --- - -# Documentation style guidelines - -The documentation style guide defines the markup structure used in -GitLab documentation. Check the -[documentation guidelines](writing_documentation.md) for general development instructions. - -Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). - -## Text - -- Split up long lines (wrap text), this makes it much easier to review and edit. Only - double line breaks are shown as a full line break in [GitLab markdown][gfm]. - 80-100 characters is a good line length -- Make sure that the documentation is added in the correct - [directory](writing_documentation.md#documentation-directory-structure) and that - there's a link to it somewhere useful -- Do not duplicate information -- Be brief and clear -- Unless there's a logical reason not to, add documents in alphabetical order -- Write in US English -- Use [single spaces][] instead of double spaces -- Jump a line between different markups (e.g., after every paragraph, header, list, etc) -- Capitalize "G" and "L" in GitLab -- Use sentence case for titles, headings, labels, menu items, and buttons. -- Use title case when referring to [features](https://about.gitlab.com/features/) or [products](https://about.gitlab.com/pricing/), and methods. Note that some features are also objects (e.g. "Merge Requests" and "merge requests"). E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. - -## Formatting - -- Use double asterisks (`**`) to mark a word or text in bold (`**bold**`) -- Use undescore (`_`) for text in italics (`_italic_`) -- Jump a line between different markups, for example: - - ```md - ## Header - - Paragraph. - - - List item - - List item - ``` - -### Punctuation - -For punctuation rules, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/). - -### Ordered and unordered lists - -- Use dashes (`-`) for unordered lists instead of asterisks (`*`) -- Use the number one (`1`) for ordered lists -- For punctuation in bullet lists, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/) - -## Headings - -- Add **only one H1** in each document, by adding `#` at the beginning of - it (when using markdown). The `h1` will be the document `<title>`. -- For subheadings, use `##`, `###` and so on -- Avoid putting numbers in headings. Numbers shift, hence documentation anchor - links shift too, which eventually leads to dead links. If you think it is - compelling to add numbers in headings, make sure to at least discuss it with - someone in the Merge Request -- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) - in headers. Whenever possible, they should be plain and short text. -- Avoid adding things that show ephemeral statuses. For example, if a feature is - considered beta or experimental, put this info in a note, not in the heading. -- When introducing a new document, be careful for the headings to be - grammatically and syntactically correct. Mention one or all - of the following GitLab members for a review: `@axil` or `@marcia`. - This is to ensure that no document with wrong heading is going - live without an audit, thus preventing dead links and redirection issues when - corrected -- Leave exactly one newline after a heading - -## Links - -- Use the regular inline link markdown markup `[Text](https://example.com)`. - It's easier to read, review, and maintain. -- If there's a link that repeats several times through the same document, - you can use `[Text][identifier]` and at the bottom of the section or the - document add: `[identifier]: https://example.com`, in which case, we do - encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. -- To link to internal documentation, use relative links, not full URLs. Use `../` to - navigate tp high-level directories, and always add the file name `file.md` at the - end of the link with the `.md` extension, not `.html`. - Example: instead of `[text](../../merge_requests/)`, use - `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, - for anchor links, `[text](../../ci/README.md#examples)`. - Using the markdown extension is necessary for the [`/help`](writing_documentation.md#gitlab-help) - section of GitLab. -- To link from CE to EE-only documentation, use the EE-only doc full URL. -- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). - E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, - write `Read more about [GitLab Issue Boards](LINK)`. - -## Images - -- Place images in a separate directory named `img/` in the same directory where - the `.md` document that you're working on is located. Always prepend their - names with the name of the document that they will be included in. For - example, if there is a document called `twitter.md`, then a valid image name - could be `twitter_login_screen.png`. [**Exception**: images for - [articles](writing_documentation.md#technical-articles) should be - put in a directory called `img` underneath `/articles/article_title/img/`, therefore, - there's no need to prepend the document name to their filenames.] -- Images should have a specific, non-generic name that will differentiate them. -- Keep all file names in lower case. -- Consider using PNG images instead of JPEG. -- Compress all images with <https://tinypng.com/> or similar tool. -- Compress gifs with <https://ezgif.com/optimize> or similar tool. -- Images should be used (only when necessary) to _illustrate_ the description -of a process, not to _replace_ it. - -Inside the document: - -- The Markdown way of using an image inside a document is: - `` -- Always use a proper description for what the image is about. That way, when a - browser fails to show the image, this text will be used as an alternative - description -- If there are consecutive images with little text between them, always add - three dashes (`---`) between the image and the text to create a horizontal - line for better clarity -- If a heading is placed right after an image, always add three dashes (`---`) - between the image and the heading - -## Alert boxes - -Whenever you want to call the attention to a particular sentence, -use the following markup for highlighting. - -_Note that the alert boxes only work for one paragraph only. Multiple paragraphs, -lists, headers, etc will not render correctly._ - -### Note - -```md -NOTE: **Note:** -This is something to note. -``` - -How it renders in docs.gitlab.com: - -NOTE: **Note:** -This is something to note. - -### Tip - -```md -TIP: **Tip:** -This is a tip. -``` - -How it renders in docs.gitlab.com: - -TIP: **Tip:** -This is a tip. - -### Caution - -```md -CAUTION: **Caution:** -This is something to be cautious about. -``` - -How it renders in docs.gitlab.com: - -CAUTION: **Caution:** -This is something to be cautious about. - -### Danger - -```md -DANGER: **Danger:** -This is a breaking change, a bug, or something very important to note. -``` - -How it renders in docs.gitlab.com: - -DANGER: **Danger:** -This is a breaking change, a bug, or something very important to note. - -## Blockquotes - -For highlighting a text within a blue blockquote, use this format: - -```md -> This is a blockquote. -``` - -which renders in docs.gitlab.com to: - -> This is a blockquote. - -If the text spans across multiple lines it's OK to split the line. - -## Specific sections and terms - -To mention and/or reference specific terms in GitLab, please follow the styles -below. - -### GitLab versions and tiers - -- Every piece of documentation that comes with a new feature should declare the - GitLab version that feature got introduced. Right below the heading add a - note: - - ```md - > Introduced in GitLab 8.3. - ``` - -- Whenever possible, every feature should have a link to the MR, issue, or epic that introduced it. - The above note would be then transformed to: - - ```md - > [Introduced][ce-1242] in GitLab 8.3. - ``` - - , where the [link identifier](#links) is named after the repository (CE) and - the MR number. - -- If the feature is only available in GitLab Enterprise Edition, don't forget to mention - the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) - the feature is available in: - - ```md - > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. - ``` - -### Product badges - -When a feature is available in EE-only tiers, add the corresponding tier according to the -feature availability: - -- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` -- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` -- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` -- For GitLab Core and GitLab.com Free: `**[CORE]**` - -To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the -keyword "only": - -- For GitLab Starter: `**[STARTER ONLY]**` -- For GitLab Premium: `**[PREMIUM ONLY]**` -- For GitLab Ultimate: `**[ULTIMATE ONLY]**` -- For GitLab Core: `**[CORE ONLY]**` - -The tier should be ideally added to headers, so that the full badge will be displayed. -But it can be also mentioned from paragraphs, list items, and table cells. For these cases, -the tier mention will be represented by an orange question mark. -E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. - -The absence of tiers' mentions mean that the feature is available in GitLab Core, -GitLab.com Free, and higher tiers. - -#### How it works - -Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), -the special markup `**[STARTER]**` will generate a `span` element to trigger the -badges and tooltips (`<span class="badge-trigger starter">`). When the keyword -"only" is added, the corresponding GitLab.com badge will not be displayed. - -### GitLab Restart - -There are many cases that a restart/reconfigure of GitLab is required. To -avoid duplication, link to the special document that can be found in -[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will -read like: - - ``` - Save the file and [reconfigure GitLab](../administration/restart_gitlab.md) - for the changes to take effect. - ``` - -If the document you are editing resides in a place other than the GitLab CE/EE -`doc/` directory, instead of the relative link, use the full path: -`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. -Replace `reconfigure` with `restart` where appropriate. - -### Installation guide - -**Ruby:** -In [step 2 of the installation guide](../install/installation.md#2-ruby), -we install Ruby from source. Whenever there is a new version that needs to -be updated, remember to change it throughout the codeblock and also replace -the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby -website). - -[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" - -### Configuration documentation for source and Omnibus installations - -GitLab currently officially supports two installation methods: installations -from source and Omnibus packages installations. - -Whenever there is a setting that is configurable for both installation methods, -prefer to document it in the CE docs to avoid duplication. - -Configuration settings include: - -- settings that touch configuration files in `config/` -- NGINX settings and settings in `lib/support/` in general - -When there is a list of steps to perform, usually that entails editing the -configuration file and reconfiguring/restarting GitLab. In such case, follow -the style below as a guide: - -```md -**For Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - external_url "https://gitlab.example.com" - ``` - -1. Save the file and [reconfigure] GitLab for the changes to take effect. - ---- - -**For installations from source** - -1. Edit `config/gitlab.yml`: - - ```yaml - gitlab: - host: "gitlab.example.com" - ``` - -1. Save the file and [restart] GitLab for the changes to take effect. - - -[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure -[restart]: path/to/administration/restart_gitlab.md#installations-from-source -``` - -In this case: - -- before each step list the installation method is declared in bold -- three dashes (`---`) are used to create a horizontal line and separate the - two methods -- the code blocks are indented one or more spaces under the list item to render - correctly -- different highlighting languages are used for each config in the code block -- the [references](#references) guide is used for reconfigure/restart - -### Fake tokens - -There may be times where a token is needed to demonstrate an API call using -cURL or a secret variable used in CI. It is strongly advised not to use real -tokens in documentation even if the probability of a token being exploited is -low. - -You can use the following fake tokens as examples. - -| **Token type** | **Token value** | -| --------------------- | --------------------------------- | -| Private user token | `9koXpg98eAheJpvBs5tK` | -| Personal access token | `n671WNGecHugsdEDPsyo` | -| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | -| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | -| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | -| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | -| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | -| Trigger token | `be20d8dcc028677c931e04f3871a9b` | -| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | -| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | -| Request profile token | `7VgpS4Ax5utVD2esNstz` | - -### API - -Here is a list of must-have items. Use them in the exact order that appears -on this document. Further explanation is given below. - -- Every method must have the REST API request. For example: - - ``` - GET /projects/:id/repository/branches - ``` - -- Every method must have a detailed - [description of the parameters](#method-description). -- Every method must have a cURL example. -- Every method must have a response body (in JSON format). - -#### Method description - -Use the following table headers to describe the methods. Attributes should -always be in code blocks using backticks (``` ` ```). - -``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -``` - -Rendered example: - -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `user` | string | yes | The GitLab username | - -#### cURL commands - -- Use `https://gitlab.example.com/api/v4/` as an endpoint. -- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. -- Always put the request first. `GET` is the default so you don't have to - include it. -- Use double quotes to the URL when it includes additional parameters. -- Prefer to use examples using the personal access token and don't pass data of - username and password. - -| Methods | Description | -| ------- | ----------- | -| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | -| `-X POST` | Use this method when creating new objects | -| `-X PUT` | Use this method when updating existing objects | -| `-X DELETE` | Use this method when removing existing objects | - -#### cURL Examples - -Below is a set of [cURL][] examples that you can use in the API documentation. - -##### Simple cURL command - -Get the details of a group: - -```bash -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org -``` - -##### cURL example with parameters passed in the URL - -Create a new project under the authenticated user's namespace: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" -``` - -##### Post data using cURL's --data - -Instead of using `-X POST` and appending the parameters to the URI, you can use -cURL's `--data` option. The example below will create a new project `foo` under -the authenticated user's namespace. - -```bash -curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" -``` - -##### Post data using JSON content - -> **Note:** In this example we create a new group. Watch carefully the single -and double quotes. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups -``` - -##### Post data using form-data - -Instead of using JSON or urlencode you can use multipart/form-data which -properly handles data encoding: - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys -``` - -The above example is run by and administrator and will add an SSH public key -titled ssh-key to user's account which has an id of 25. - -##### Escape special characters - -Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended -to escape them when possible. In the example below we create a new issue which -contains spaces in its title. Observe how spaces are escaped using the `%20` -ASCII code. - -```bash -curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" -``` - -Use `%2F` for slashes (`/`). - -##### Pass arrays to API calls - -The GitLab API sometimes accepts arrays of strings or integers. For example, to -restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and -`example.net`, you would do something like this: - -```bash -curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings -``` - -[cURL]: http://curl.haxx.se/ "cURL website" -[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html -[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" -[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 -[doc-restart]: ../administration/restart_gitlab.md "GitLab restart documentation" diff --git a/doc/development/img/manual_build_docs.png b/doc/development/documentation/img/manual_build_docs.png Binary files differindex 615facabb5f..615facabb5f 100644 --- a/doc/development/img/manual_build_docs.png +++ b/doc/development/documentation/img/manual_build_docs.png diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md new file mode 100644 index 00000000000..48e1685082a --- /dev/null +++ b/doc/development/documentation/index.md @@ -0,0 +1,558 @@ +--- +description: Learn how to contribute to GitLab Documentation. +--- + +# GitLab Documentation guidelines + + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). + - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). + +## Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one responsible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +Whenever you submit a merge request for the documentation, use the documentation MR description template. + +Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. + +## Documentation structure + +- Overview and use cases: what it is, why it is necessary, why one would use it +- Requirements: what do we need to get started +- Tutorial: how to set it up, how to use it + +Always link a new document from its topic-related index, otherwise, it will +not be included it in the documentation site search. + +_Note: to be extended._ + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: as the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feature or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + +## Markdown and styles + +Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. + +All the docs follow the [documentation style guidelines](styleguide.md). + +## Documentation directory structure + +The documentation is structured based on the GitLab UI structure itself, +separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), +[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). + +In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, +all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. + +The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have +been deprecated and the majority their docs have been moved to their correct location +in small iterations. Please don't create new docs in these folders. + +### Location and naming documents + +The documentation hierarchy can be vastly improved by providing a better layout +and organization of directories. + +Having a structured document layout, we will be able to have meaningful URLs +like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, +you can immediately tell that you are navigating a user related documentation +and is about the project and its merge requests. + +Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), +rather organize content by its subject (e.g. everything related to CI goes together) +and cross-link between any related content. + +The table below shows what kind of documentation goes where. + +| Directory | What belongs here | +| --------- | -------------- | +| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | +| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | +| `doc/api/` | API related documentation. | +| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | +| `doc/legal/` | Legal documents about contributing to GitLab. | +| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | +| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | +| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | + +--- + +**General rules:** + +1. The correct naming and location of a new document, is a combination + of the relative URL of the document in question and the GitLab Map design + that is used for UX purposes ([source][graffle], [image][gitlab-map]). +1. When creating a new document and it has more than one word in its name, + make sure to use underscores instead of spaces or dashes (`-`). For example, + a proper naming would be `import_projects_from_github.md`. The same rule + applies to images. +1. Start a new directory with an `index.md` file. +1. There are four main directories, `user`, `administration`, `api` and `development`. +1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, + `profile/`, `dashboard/` and `admin_area/`. + 1. `doc/user/project/` should contain all project related documentation. + 1. `doc/user/group/` should contain all group related documentation. + 1. `doc/user/profile/` should contain all profile related documentation. + Every page you would navigate under `/profile` should have its own document, + i.e. `account.md`, `applications.md`, `emails.md`, etc. + 1. `doc/user/dashboard/` should contain all dashboard related documentation. + 1. `doc/user/admin_area/` should contain all admin related documentation + describing what can be achieved by accessing GitLab's admin interface + (_not to be confused with `doc/administration` where server access is + required_). + 1. Every category under `/admin/application_settings` should have its + own document located at `doc/user/admin_area/settings/`. For example, + the **Visibility and Access Controls** category should have a document + located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. +1. The `doc/topics/` directory holds topic-related technical content. Create + `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. + General user- and admin- related documentation, should be placed accordingly. + +If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your +merge request. + +### Changing document location + +Changing a document's location is not to be taken lightly. Remember that the +documentation is available to all installations under `help/` and not only to +GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the +Documentation team beforehand. + +If you indeed need to change a document's location, do NOT remove the old +document, but rather replace all of its contents with a new line: + +``` +This document was moved to [another location](path/to/new_doc.md). +``` + +where `path/to/new_doc.md` is the relative path to the root directory `doc/`. + +--- + +For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to +`doc/administration/lfs.md`, then the steps would be: + +1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` +1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: + + ``` + This document was moved to [another location](../../administration/lfs.md). + ``` + +1. Find and replace any occurrences of the old location with the new one. + A quick way to find them is to use `git grep`. First go to the root directory + where you cloned the `gitlab-ce` repository and then do: + + ``` + git grep -n "workflow/lfs/lfs_administration" + git grep -n "lfs/lfs_administration" + ``` + +NOTE: **Note:** +If the document being moved has any Disqus comments on it, there are extra steps +to follow documented just [below](#redirections-for-pages-with-disqus-comments). + +Things to note: + +- Since we also use inline documentation, except for the documentation itself, + the document might also be referenced in the views of GitLab (`app/`) which will + render when visiting `/help`, and sometimes in the testing suite (`spec/`). +- The above `git grep` command will search recursively in the directory you run + it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` + and will print the file and the line where this file is mentioned. + You may ask why the two greps. Since we use relative paths to link to + documentation, sometimes it might be useful to search a path deeper. +- The `*.md` extension is not used when a document is linked to GitLab's + built-in help page, that's why we omit it in `git grep`. +- Use the checklist on the documentation MR description template. + +#### Alternative redirection method + +Alternatively to the method described above, you can simply replace the content +of the old file with a frontmatter containing a redirect link: + +```yaml +--- +redirect_to: '../path/to/file/README.md' +--- +``` + +It supports both full and relative URLs, e.g. `https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. Note that any `*.md` paths will be compiled to `*.html`. + +### Redirections for pages with Disqus comments + +If the documentation page being relocated already has any Disqus comments, +we need to preserve the Disqus thread. + +Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier +is configured to be the page URL. Therefore, when we change the document location, +we need to preserve the old URL as the same Disqus identifier. + +To do that, add to the frontmatter the variable `redirect_from`, +using the old URL as value. For example, let's say I moved the document +available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, +`https://docs.gitlab.com/my-new-location/index.html`. + +Into the **new document** frontmatter add the following: + +```yaml +--- +redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' +--- +``` + +Note: it is necessary to include the file name in the `redirect_from` URL, +even if it's `index.html` or `README.html`. + +## Testing + +We treat documentation as code, thus have implemented some testing. +Currently, the following tests are in place: + +1. `docs lint`: Check that all internal (relative) links work correctly and + that all cURL examples in API docs use the full switches. It's recommended + to [check locally](#previewing-locally) before pushing to GitLab by executing the command + `bundle exec nanoc check internal_links` on your local + [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. +1. [`ee_compat_check`](../automatic_ce_ee_merge.md#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): + When you submit a merge request to GitLab Community Edition (CE), + there is this additional job that runs against Enterprise Edition (EE) + and checks if your changes can apply cleanly to the EE codebase. + If that job fails, read the instructions in the job log for what to do next. + As CE is merged into EE once a day, it's important to avoid merge conflicts. + Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is + essential to avoid them. + +## Branch naming + +If your contribution contains **only** documentation changes, you can speed up +the CI process by following some branch naming conventions. You have three +choices: + +| Branch name | Valid example | +| ----------- | ------------- | +| Starting with `docs/` | `docs/update-api-issues` | +| Starting with `docs-` | `docs-update-api-issues` | +| Ending in `-docs` | `123-update-api-issues-docs` | + +If your branch name matches any of the above, it will run only the docs +tests. If it doesn't, the whole test suite will run (including docs). + +## Merge requests for GitLab documentation + +Before getting started, make sure you read the introductory section +"[contributing to docs](#contributing-to-docs)" above and the +[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) +for GitLab Team members. + +- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) +- Use the correct [branch name](#branch-naming) +- Label the MR `Documentation` +- Assign the correct milestone (see note below) + + +NOTE: **Note:** +If the release version you want to add the documentation to has already been +frozen or released, use the label `Pick into X.Y` to get it merged into +the correct release. Avoid picking into a past release as much as you can, as +it increases the work of the release managers. + +### Cherry-picking from CE to EE + +As we have the `master` branch of CE merged into EE once a day, it's common to +run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) +with the `ee-compat-check` job, and use the following method of creating equivalent +branches for CE and EE. + +Follow this [method for cherry-picking from CE to EE](../automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: + +- Create the [CE branch](#branch-naming) starting with `docs-`, + e.g.: `git checkout -b docs-example` +- Create the EE-equivalent branch ending with `-ee`, e.g., + `git checkout -b docs-example-ee` +- Once all the jobs are passing in CE and EE, and you've addressed the +feedback from your own team, assign the CE MR to a technical writer for review +- When both MRs are ready, the EE merge request will be merged first, and the +CE-equivalent will be merged next. +- Note that the review will occur only in the CE MR, as the EE MR +contains the same commits as the CE MR. +- If you have a few more changes that apply to the EE-version only, you can submit +a couple more commits to the EE branch, but ask the reviewer to review the EE merge request +additionally to the CE MR. If there are many EE-only changes though, start a new MR +to EE only. + +## Previewing the changes live + +To preview your changes to documentation locally, please follow +this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). + +If you want to preview the doc changes of your merge request live, you can use +the manual `review-docs-deploy` job in your merge request. You will need at +least Maintainer permissions to be able to run it and is currently enabled for the +following projects: + +- https://gitlab.com/gitlab-org/gitlab-ce +- https://gitlab.com/gitlab-org/gitlab-ee + +NOTE: **Note:** +You will need to push a branch to those repositories, it doesn't work for forks. + +TIP: **Tip:** +If your branch contains only documentation changes, you can use +[special branch names](#branch-naming) to avoid long running pipelines. + +In the mini pipeline graph, you should see an `>>` icon. Clicking on it will +reveal the `review-docs-deploy` job. Hit the play button for the job to start. + + + +This job will: + +1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) + project named after the scheme: `preview-<branch-slug>` +1. Trigger a cross project pipeline and build the docs site with your changes + +After a few minutes, the Review App will be deployed and you will be able to +preview the changes. The docs URL can be found in two places: + +- In the merge request widget +- In the output of the `review-docs-deploy` job, which also includes the + triggered pipeline so that you can investigate whether something went wrong + +In case the Review App URL returns 404, follow these steps to debug: + +1. **Did you follow the URL from the merge request widget?** If yes, then check if + the link is the same as the one in the job output. It can happen that if the + branch name slug is longer than 35 characters, it is automatically + truncated. That means that the merge request widget will not show the proper + URL due to a limitation of how `environment: url` works, but you can find the + real URL from the output of the `review-docs-deploy` job. +1. **Did you follow the URL from the job output?** If yes, then it means that + either the site is not yet deployed or something went wrong with the remote + pipeline. Give it a few minutes and it should appear online, otherwise you + can check the status of the remote pipeline from the link in the job output. + If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. + +TIP: **Tip:** +Someone that has no merge rights to the CE/EE projects (think of forks from +contributors) will not be able to run the manual job. In that case, you can +ask someone from the GitLab team who has the permissions to do that for you. + +NOTE: **Note:** +Make sure that you always delete the branch of the merge request you were +working on. If you don't, the remote docs branch won't be removed either, +and the server where the Review Apps are hosted will eventually be out of +disk space. + +### Technical aspects + +If you want to know the hot details, here's what's really happening: + +1. You manually run the `review-docs-deploy` job in a CE/EE merge request. +1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) + script with the `deploy` flag, which in turn: + 1. Takes your branch name and applies the following: + - The slug of the branch name is used to avoid special characters since + ultimately this will be used by NGINX. + - The `preview-` prefix is added to avoid conflicts if there's a remote branch + with the same name that you created in the merge request. + - The final branch name is truncated to 42 characters to avoid filesystem + limitations with long branch names (> 63 chars). + 1. The remote branch is then created if it doesn't exist (meaning you can + re-run the manual job as many times as you want and this step will be skipped). + 1. A new cross-project pipeline is triggered in the docs project. + 1. The preview URL is shown both at the job output and in the merge request + widget. You also get the link to the remote pipeline. +1. In the docs project, the pipeline is created and it + [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) + to lower the build time. +1. Once the docs site is built, the HTML files are uploaded as artifacts. +1. A specific Runner tied only to the docs project, runs the Review App job + that downloads the artifacts and uses `rsync` to transfer the files over + to a location where NGINX serves them. + +The following GitLab features are used among others: + +- [Manual actions](../../ci/yaml/README.md#manual-actions) +- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Review Apps](../../ci/review_apps/index.md) +- [Artifacts](../../ci/yaml/README.md#artifacts) +- [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) + +## GitLab `/help` + +Every GitLab instance includes the documentation, which is available from `/help` +(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. + +When you're building a new feature, you may need to link the documentation +from GitLab, the application. This is normally done in files inside the +`app/views/` directory with the help of the `help_page_path` helper method. + +In its simplest form, the HAML code to generate a link to the `/help` page is: + +```haml += link_to 'Help page', help_page_path('user/permissions') +``` + +The `help_page_path` contains the path to the document you want to link to with +the following conventions: + +- it is relative to the `doc/` directory in the GitLab repository +- the `.md` extension must be omitted +- it must not end with a slash (`/`) + +Below are some special cases where should be used depending on the context. +You can combine one or more of the following: + +1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` + method: + + ```haml + = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') + ``` + +1. **Opening links in a new tab.** This should be the default behavior: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' + ``` + +1. **Linking to a circle icon.** Usually used in settings where a long + description cannot be used, like near checkboxes. You can basically use + any font awesome icon, but prefer the `question-circle`: + + ```haml + = link_to icon('question-circle'), help_page_path('user/permissions') + ``` + +1. **Using a button link.** Useful in places where text would be out of context + with the rest of the page layout: + + ```haml + = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' + ``` + +1. **Using links inline of some text.** + + ```haml + Description to #{link_to 'Help page', help_page_path('user/permissions')}. + ``` + +1. **Adding a period at the end of the sentence.** Useful when you don't want + the period to be part of the link: + + ```haml + = succeed '.' do + Learn more in the + = link_to 'Help page', help_page_path('user/permissions') + ``` + +## General Documentation vs Technical Articles + +### General documentation + +General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. + +### Technical Articles + +Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. + +They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. + +A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. + +They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. + +#### Types of Technical Articles + +- **User guides**: technical content to guide regular users from point A to point B +- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B +- **Technical Overviews**: technical content describing features, solutions, and third-party integrations +- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives + +#### Understanding guides, tutorials, and technical overviews + +Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. + +A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. + +- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../../user/project/pages/getting_started_part_four.md)" + +A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. +It does not only describes steps 2 and 3, but also shows you how to accomplish them. + +- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) + +A **technical overview** is a description of what a certain feature is, and what it does, but does not walk +through the process of how to use it systematically. + +- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) + +#### Special format + +Every **Technical Article** contains a frontmatter at the beginning of the doc +with the following information: + +- **Type of article** (user guide, admin guide, technical overview, tutorial) +- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) +- **Author's name** and **GitLab.com handle** +- **Publication date** (ISO format YYYY-MM-DD) + +For example: + + +```yaml +--- +author: John Doe +author_gitlab: johnDoe +level: beginner +article_type: user guide +date: 2017-02-01 +--- +``` + +#### Technical Articles - Writing Method + +Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. + +[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png +[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md new file mode 100644 index 00000000000..e7ffba635c9 --- /dev/null +++ b/doc/development/documentation/styleguide.md @@ -0,0 +1,499 @@ +--- +description: 'Writing styles, markup, formatting, and reusing regular expressions throughout the GitLab Documentation.' +--- + +# Documentation style guidelines + +The documentation style guide defines the markup structure used in +GitLab documentation. Check the +[documentation guidelines](index.md) for general development instructions. + +Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). + +## Text + +- Split up long lines (wrap text), this makes it much easier to review and edit. Only + double line breaks are shown as a full line break in [GitLab markdown][gfm]. + 80-100 characters is a good line length +- Make sure that the documentation is added in the correct + [directory](index.md#documentation-directory-structure) and that + there's a link to it somewhere useful +- Do not duplicate information +- Be brief and clear +- Unless there's a logical reason not to, add documents in alphabetical order +- Write in US English +- Use [single spaces][] instead of double spaces +- Jump a line between different markups (e.g., after every paragraph, header, list, etc) +- Capitalize "G" and "L" in GitLab +- Use sentence case for titles, headings, labels, menu items, and buttons. +- Use title case when referring to [features](https://about.gitlab.com/features/) or [products](https://about.gitlab.com/pricing/), and methods. Note that some features are also objects (e.g. "Merge Requests" and "merge requests"). E.g.: GitLab Runner, Geo, Issue Boards, Git, Prometheus, Continuous Integration. + +## Formatting + +- Use double asterisks (`**`) to mark a word or text in bold (`**bold**`) +- Use undescore (`_`) for text in italics (`_italic_`) +- Jump a line between different markups, for example: + + ```md + ## Header + + Paragraph. + + - List item + - List item + ``` + +### Punctuation + +For punctuation rules, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/). + +### Ordered and unordered lists + +- Use dashes (`-`) for unordered lists instead of asterisks (`*`) +- Use the number one (`1`) for ordered lists +- For punctuation in bullet lists, please refer to the [GitLab UX guide](https://design.gitlab.com/content/punctuation/) + +## Headings + +- Add **only one H1** in each document, by adding `#` at the beginning of + it (when using markdown). The `h1` will be the document `<title>`. +- For subheadings, use `##`, `###` and so on +- Avoid putting numbers in headings. Numbers shift, hence documentation anchor + links shift too, which eventually leads to dead links. If you think it is + compelling to add numbers in headings, make sure to at least discuss it with + someone in the Merge Request +- [Avoid using symbols and special chars](https://gitlab.com/gitlab-com/gitlab-docs/issues/84) + in headers. Whenever possible, they should be plain and short text. +- Avoid adding things that show ephemeral statuses. For example, if a feature is + considered beta or experimental, put this info in a note, not in the heading. +- When introducing a new document, be careful for the headings to be + grammatically and syntactically correct. Mention one or all + of the following GitLab members for a review: `@axil` or `@marcia`. + This is to ensure that no document with wrong heading is going + live without an audit, thus preventing dead links and redirection issues when + corrected +- Leave exactly one newline after a heading + +## Links + +- Use the regular inline link markdown markup `[Text](https://example.com)`. + It's easier to read, review, and maintain. +- If there's a link that repeats several times through the same document, + you can use `[Text][identifier]` and at the bottom of the section or the + document add: `[identifier]: https://example.com`, in which case, we do + encourage you to also add an alternative text: `[identifier]: https://example.com "Alternative text"` that appears when hovering your mouse on a link. +- To link to internal documentation, use relative links, not full URLs. Use `../` to + navigate tp high-level directories, and always add the file name `file.md` at the + end of the link with the `.md` extension, not `.html`. + Example: instead of `[text](../../merge_requests/)`, use + `[text](../../merge_requests/index.md)` or, `[text](../../ci/README.md)`, or, + for anchor links, `[text](../../ci/README.md#examples)`. + Using the markdown extension is necessary for the [`/help`](index.md#gitlab-help) + section of GitLab. +- To link from CE to EE-only documentation, use the EE-only doc full URL. +- Use [meaningful anchor texts](https://www.futurehosting.com/blog/links-should-have-meaningful-anchor-text-heres-why/). + E.g., instead of writing something like `Read more about GitLab Issue Boards [here](LINK)`, + write `Read more about [GitLab Issue Boards](LINK)`. + +## Images + +- Place images in a separate directory named `img/` in the same directory where + the `.md` document that you're working on is located. Always prepend their + names with the name of the document that they will be included in. For + example, if there is a document called `twitter.md`, then a valid image name + could be `twitter_login_screen.png`. [**Exception**: images for + [articles](index.md#technical-articles) should be + put in a directory called `img` underneath `/articles/article_title/img/`, therefore, + there's no need to prepend the document name to their filenames.] +- Images should have a specific, non-generic name that will differentiate them. +- Keep all file names in lower case. +- Consider using PNG images instead of JPEG. +- Compress all images with <https://tinypng.com/> or similar tool. +- Compress gifs with <https://ezgif.com/optimize> or similar tool. +- Images should be used (only when necessary) to _illustrate_ the description +of a process, not to _replace_ it. + +Inside the document: + +- The Markdown way of using an image inside a document is: + `` +- Always use a proper description for what the image is about. That way, when a + browser fails to show the image, this text will be used as an alternative + description +- If there are consecutive images with little text between them, always add + three dashes (`---`) between the image and the text to create a horizontal + line for better clarity +- If a heading is placed right after an image, always add three dashes (`---`) + between the image and the heading + +## Alert boxes + +Whenever you want to call the attention to a particular sentence, +use the following markup for highlighting. + +_Note that the alert boxes only work for one paragraph only. Multiple paragraphs, +lists, headers, etc will not render correctly._ + +### Note + +```md +NOTE: **Note:** +This is something to note. +``` + +How it renders in docs.gitlab.com: + +NOTE: **Note:** +This is something to note. + +### Tip + +```md +TIP: **Tip:** +This is a tip. +``` + +How it renders in docs.gitlab.com: + +TIP: **Tip:** +This is a tip. + +### Caution + +```md +CAUTION: **Caution:** +This is something to be cautious about. +``` + +How it renders in docs.gitlab.com: + +CAUTION: **Caution:** +This is something to be cautious about. + +### Danger + +```md +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. +``` + +How it renders in docs.gitlab.com: + +DANGER: **Danger:** +This is a breaking change, a bug, or something very important to note. + +## Blockquotes + +For highlighting a text within a blue blockquote, use this format: + +```md +> This is a blockquote. +``` + +which renders in docs.gitlab.com to: + +> This is a blockquote. + +If the text spans across multiple lines it's OK to split the line. + +## Specific sections and terms + +To mention and/or reference specific terms in GitLab, please follow the styles +below. + +### GitLab versions and tiers + +- Every piece of documentation that comes with a new feature should declare the + GitLab version that feature got introduced. Right below the heading add a + note: + + ```md + > Introduced in GitLab 8.3. + ``` + +- Whenever possible, every feature should have a link to the MR, issue, or epic that introduced it. + The above note would be then transformed to: + + ```md + > [Introduced][ce-1242] in GitLab 8.3. + ``` + + , where the [link identifier](#links) is named after the repository (CE) and + the MR number. + +- If the feature is only available in GitLab Enterprise Edition, don't forget to mention + the [paid tier](https://about.gitlab.com/handbook/marketing/product-marketing/#tiers) + the feature is available in: + + ```md + > [Introduced][ee-1234] in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. + ``` + +### Product badges + +When a feature is available in EE-only tiers, add the corresponding tier according to the +feature availability: + +- For GitLab Starter and GitLab.com Bronze: `**[STARTER]**` +- For GitLab Premium and GitLab.com Silver: `**[PREMIUM]**` +- For GitLab Ultimate and GitLab.com Gold: `**[ULTIMATE]**` +- For GitLab Core and GitLab.com Free: `**[CORE]**` + +To exclude GitLab.com tiers (when the feature is not available in GitLab.com), add the +keyword "only": + +- For GitLab Starter: `**[STARTER ONLY]**` +- For GitLab Premium: `**[PREMIUM ONLY]**` +- For GitLab Ultimate: `**[ULTIMATE ONLY]**` +- For GitLab Core: `**[CORE ONLY]**` + +The tier should be ideally added to headers, so that the full badge will be displayed. +But it can be also mentioned from paragraphs, list items, and table cells. For these cases, +the tier mention will be represented by an orange question mark. +E.g., `**[STARTER]**` renders **[STARTER]**, `**[STARTER ONLY]**` renders **[STARTER ONLY]**. + +The absence of tiers' mentions mean that the feature is available in GitLab Core, +GitLab.com Free, and higher tiers. + +#### How it works + +Introduced by [!244](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/244), +the special markup `**[STARTER]**` will generate a `span` element to trigger the +badges and tooltips (`<span class="badge-trigger starter">`). When the keyword +"only" is added, the corresponding GitLab.com badge will not be displayed. + +### GitLab Restart + +There are many cases that a restart/reconfigure of GitLab is required. To +avoid duplication, link to the special document that can be found in +[`doc/administration/restart_gitlab.md`][doc-restart]. Usually the text will +read like: + + ``` + Save the file and [reconfigure GitLab](../../administration/restart_gitlab.md) + for the changes to take effect. + ``` + +If the document you are editing resides in a place other than the GitLab CE/EE +`doc/` directory, instead of the relative link, use the full path: +`http://docs.gitlab.com/ce/administration/restart_gitlab.html`. +Replace `reconfigure` with `restart` where appropriate. + +### Installation guide + +**Ruby:** +In [step 2 of the installation guide](../../install/installation.md#2-ruby), +we install Ruby from source. Whenever there is a new version that needs to +be updated, remember to change it throughout the codeblock and also replace +the sha256sum (it can be found in the [downloads page][ruby-dl] of the Ruby +website). + +[ruby-dl]: https://www.ruby-lang.org/en/downloads/ "Ruby download website" + +### Configuration documentation for source and Omnibus installations + +GitLab currently officially supports two installation methods: installations +from source and Omnibus packages installations. + +Whenever there is a setting that is configurable for both installation methods, +prefer to document it in the CE docs to avoid duplication. + +Configuration settings include: + +- settings that touch configuration files in `config/` +- NGINX settings and settings in `lib/support/` in general + +When there is a list of steps to perform, usually that entails editing the +configuration file and reconfiguring/restarting GitLab. In such case, follow +the style below as a guide: + +```md +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + external_url "https://gitlab.example.com" + ``` + +1. Save the file and [reconfigure] GitLab for the changes to take effect. + +--- + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + gitlab: + host: "gitlab.example.com" + ``` + +1. Save the file and [restart] GitLab for the changes to take effect. + + +[reconfigure]: path/to/administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: path/to/administration/restart_gitlab.md#installations-from-source +``` + +In this case: + +- before each step list the installation method is declared in bold +- three dashes (`---`) are used to create a horizontal line and separate the + two methods +- the code blocks are indented one or more spaces under the list item to render + correctly +- different highlighting languages are used for each config in the code block +- the [references](#references) guide is used for reconfigure/restart + +### Fake tokens + +There may be times where a token is needed to demonstrate an API call using +cURL or a variable used in CI. It is strongly advised not to use real +tokens in documentation even if the probability of a token being exploited is +low. + +You can use the following fake tokens as examples. + +| **Token type** | **Token value** | +| --------------------- | --------------------------------- | +| Private user token | `9koXpg98eAheJpvBs5tK` | +| Personal access token | `n671WNGecHugsdEDPsyo` | +| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | +| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | +| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | +| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | +| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | +| Trigger token | `be20d8dcc028677c931e04f3871a9b` | +| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | +| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | +| Request profile token | `7VgpS4Ax5utVD2esNstz` | + +### API + +Here is a list of must-have items. Use them in the exact order that appears +on this document. Further explanation is given below. + +- Every method must have the REST API request. For example: + + ``` + GET /projects/:id/repository/branches + ``` + +- Every method must have a detailed + [description of the parameters](#method-description). +- Every method must have a cURL example. +- Every method must have a response body (in JSON format). + +#### Method description + +Use the following table headers to describe the methods. Attributes should +always be in code blocks using backticks (``` ` ```). + +``` +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +``` + +Rendered example: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user` | string | yes | The GitLab username | + +#### cURL commands + +- Use `https://gitlab.example.com/api/v4/` as an endpoint. +- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. +- Always put the request first. `GET` is the default so you don't have to + include it. +- Use double quotes to the URL when it includes additional parameters. +- Prefer to use examples using the personal access token and don't pass data of + username and password. + +| Methods | Description | +| ------- | ----------- | +| `-H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK"` | Use this method as is, whenever authentication needed | +| `-X POST` | Use this method when creating new objects | +| `-X PUT` | Use this method when updating existing objects | +| `-X DELETE` | Use this method when removing existing objects | + +#### cURL Examples + +Below is a set of [cURL][] examples that you can use in the API documentation. + +##### Simple cURL command + +Get the details of a group: + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/groups/gitlab-org +``` + +##### cURL example with parameters passed in the URL + +Create a new project under the authenticated user's namespace: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects?name=foo" +``` + +##### Post data using cURL's --data + +Instead of using `-X POST` and appending the parameters to the URI, you can use +cURL's `--data` option. The example below will create a new project `foo` under +the authenticated user's namespace. + +```bash +curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" +``` + +##### Post data using JSON content + +> **Note:** In this example we create a new group. Watch carefully the single +and double quotes. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v4/groups +``` + +##### Post data using form-data + +Instead of using JSON or urlencode you can use multipart/form-data which +properly handles data encoding: + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v4/users/25/keys +``` + +The above example is run by and administrator and will add an SSH public key +titled ssh-key to user's account which has an id of 25. + +##### Escape special characters + +Spaces or slashes (`/`) may sometimes result to errors, thus it is recommended +to escape them when possible. In the example below we create a new issue which +contains spaces in its title. Observe how spaces are escaped using the `%20` +ASCII code. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/42/issues?title=Hello%20Dude" +``` + +Use `%2F` for slashes (`/`). + +##### Pass arrays to API calls + +The GitLab API sometimes accepts arrays of strings or integers. For example, to +restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and +`example.net`, you would do something like this: + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v4/application/settings +``` + +[cURL]: http://curl.haxx.se/ "cURL website" +[single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html +[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" +[ce-1242]: https://gitlab.com/gitlab-org/gitlab-ce/issues/1242 +[doc-restart]: ../../administration/restart_gitlab.md "GitLab restart documentation" diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 655d94793dd..48eb6d0a7d6 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -216,12 +216,12 @@ If you want a line or set of lines to be ignored by the linter, you can use `// scss-lint:disable RuleName` ([more info][disabling-linters]): ```scss -// This lint rule is disabled because the class name comes from a gem. -// scss-lint:disable SelectorFormat -.ui_indigo { - background-color: #333; +// This lint rule is disabled because it is supported only in Chrome/Safari +// scss-lint:disable PropertySpelling +body { + text-decoration-skip: ink; } -// scss-lint:enable SelectorFormat +// scss-lint:enable PropertySpelling ``` Make sure a comment is added on the line above the `disable` rule, otherwise the diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 5185d843ccb..9a677bf09b2 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -16,7 +16,7 @@ are very appreciative of the work done by translators and proofreaders! - Dutch - Esperanto - French - - Rémy Coutable - [GitLab](https://gitlab.com/rymai), [Crowdin](https://crowdin.com/profile/rymai) + - Davy Defaud - [GitLab](https://gitlab.com/DevDef), [Crowdin](https://crowdin.com/profile/DevDef) - German - Indonesian - Ahmad Naufal Mukhtar - [GitLab](https://gitlab.com/anaufalm), [Crowdin](https://crowdin.com/profile/anaufalm) diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md index 26d3355e94d..61e5e1afede 100644 --- a/doc/development/query_recorder.md +++ b/doc/development/query_recorder.md @@ -22,6 +22,19 @@ As an example you might create 5 issues in between counts, which would cause the > **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. +## Cached queries + +By default, QueryRecorder will ignore cached queries in the count. However, it may be better to count +all queries to avoid introducing an N+1 query that may be masked by the statement cache. To do this, +pass the `skip_cached` variable to `QueryRecorder` and use the `exceed_all_query_limit` matcher: + +it "avoids N+1 database queries" do + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) { visit_some_page }.count + create_list(:issue, 5) + expect { visit_some_page }.not_to exceed_all_query_limit(control_count) +end +``` + ## Finding the source of the query It may be useful to identify the source of the queries by looking at the call backtrace. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 31addcaf675..fc51b74da1d 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -176,3 +176,20 @@ git push -u origin update-project-templates ``` Now create a merge request and merge that to master. + +## Generate route lists + +To see the full list of API routes, you can run: + +```shell +bundle exec rake grape:path_helpers +``` + +For the Rails controllers, run: + +```shell +bundle exec rake routes +``` + +Since these take some time to create, it's often helpful to save the output to +a file for quick reference. diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 1c41fc7611f..038a4b1e6ea 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,558 +1,3 @@ --- -description: Learn how to contribute to GitLab Documentation. +redirect_to: 'documentation/index.md' --- - -# GitLab Documentation guidelines - - - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - - **[Technical Articles](#technical-articles)**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). - -## Contributing to docs - -Whenever a feature is changed, updated, introduced, or deprecated, the merge -request introducing these changes must be accompanied by the documentation -(either updating existing ones or creating new ones). This is also valid when -changes are introduced to the UI. - -The one responsible for writing the first piece of documentation is the developer who -wrote the code. It's the job of the Product Manager to ensure all features are -shipped with its docs, whether is a small or big change. At the pace GitLab evolves, -this is the only way to keep the docs up-to-date. If you have any questions about it, -ask a Technical Writer. Otherwise, when your content is ready, assign one of -them to review it for you. - -We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything -is documented. - -Whenever you submit a merge request for the documentation, use the documentation MR description template. - -Please check the [documentation workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) before getting started. - -## Documentation structure - -- Overview and use cases: what it is, why it is necessary, why one would use it -- Requirements: what do we need to get started -- Tutorial: how to set it up, how to use it - -Always link a new document from its topic-related index, otherwise, it will -not be included it in the documentation site search. - -_Note: to be extended._ - -### Feature overview and use cases - -Every major feature (regardless if present in GitLab Community or Enterprise editions) -should present, at the beginning of the document, two main sections: **overview** and -**use cases**. Every GitLab EE-only feature should also contain these sections. - -**Overview**: as the name suggests, the goal here is to provide an overview of the feature. -Describe what is it, what it does, why it is important/cool/nice-to-have, -what problem it solves, and what you can do with this feature that you couldn't -do before. - -**Use cases**: provide at least two, ideally three, use cases for every major feature. -You should answer this question: what can you do with this feature/change? Use cases -are examples of how this feature or change can be used in real life. - -Examples: -- CE and EE: [Issues](../user/project/issues/index.md#use-cases) -- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) -- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) - -Note that if you don't have anything to add between the doc title (`<h1>`) and -the header `## Overview`, you can omit the header, but keep the content of the -overview there. - -> **Overview** and **use cases** are required to **every** Enterprise Edition feature, -and for every **major** feature present in Community Edition. - -## Markdown and styles - -Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. - -All the docs follow the [documentation style guidelines](doc_styleguide.md). - -## Documentation directory structure - -The documentation is structured based on the GitLab UI structure itself, -separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), -[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). - -In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, -all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. - -The directories `/workflow/`, `/gitlab-basics/`, `/university/`, and `/articles/` have -been deprecated and the majority their docs have been moved to their correct location -in small iterations. Please don't create new docs in these folders. - -### Location and naming documents - -The documentation hierarchy can be vastly improved by providing a better layout -and organization of directories. - -Having a structured document layout, we will be able to have meaningful URLs -like `docs.gitlab.com/user/project/merge_requests/index.html`. With this pattern, -you can immediately tell that you are navigating a user related documentation -and is about the project and its merge requests. - -Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.), -rather organize content by its subject (e.g. everything related to CI goes together) -and cross-link between any related content. - -The table below shows what kind of documentation goes where. - -| Directory | What belongs here | -| --------- | -------------- | -| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | -| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | -| `doc/api/` | API related documentation. | -| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | -| `doc/legal/` | Legal documents about contributing to GitLab. | -| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | -| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | -| `doc/topics/` | Indexes per Topic (`doc/topics/topic-name/index.md`): all resources for that topic (user and admin documentation, articles, and third-party docs) | - ---- - -**General rules:** - -1. The correct naming and location of a new document, is a combination - of the relative URL of the document in question and the GitLab Map design - that is used for UX purposes ([source][graffle], [image][gitlab-map]). -1. When creating a new document and it has more than one word in its name, - make sure to use underscores instead of spaces or dashes (`-`). For example, - a proper naming would be `import_projects_from_github.md`. The same rule - applies to images. -1. Start a new directory with an `index.md` file. -1. There are four main directories, `user`, `administration`, `api` and `development`. -1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, - `profile/`, `dashboard/` and `admin_area/`. - 1. `doc/user/project/` should contain all project related documentation. - 1. `doc/user/group/` should contain all group related documentation. - 1. `doc/user/profile/` should contain all profile related documentation. - Every page you would navigate under `/profile` should have its own document, - i.e. `account.md`, `applications.md`, `emails.md`, etc. - 1. `doc/user/dashboard/` should contain all dashboard related documentation. - 1. `doc/user/admin_area/` should contain all admin related documentation - describing what can be achieved by accessing GitLab's admin interface - (_not to be confused with `doc/administration` where server access is - required_). - 1. Every category under `/admin/application_settings` should have its - own document located at `doc/user/admin_area/settings/`. For example, - the **Visibility and Access Controls** category should have a document - located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. -1. The `doc/topics/` directory holds topic-related technical content. Create - `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. - General user- and admin- related documentation, should be placed accordingly. - -If you are unsure where a document should live, you can ping `@axil` or `@marcia` in your -merge request. - -### Changing document location - -Changing a document's location is not to be taken lightly. Remember that the -documentation is available to all installations under `help/` and not only to -GitLab.com or http://docs.gitlab.com. Make sure this is discussed with the -Documentation team beforehand. - -If you indeed need to change a document's location, do NOT remove the old -document, but rather replace all of its contents with a new line: - -``` -This document was moved to [another location](path/to/new_doc.md). -``` - -where `path/to/new_doc.md` is the relative path to the root directory `doc/`. - ---- - -For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to -`doc/administration/lfs.md`, then the steps would be: - -1. Copy `doc/workflow/lfs/lfs_administration.md` to `doc/administration/lfs.md` -1. Replace the contents of `doc/workflow/lfs/lfs_administration.md` with: - - ``` - This document was moved to [another location](../../administration/lfs.md). - ``` - -1. Find and replace any occurrences of the old location with the new one. - A quick way to find them is to use `git grep`. First go to the root directory - where you cloned the `gitlab-ce` repository and then do: - - ``` - git grep -n "workflow/lfs/lfs_administration" - git grep -n "lfs/lfs_administration" - ``` - -NOTE: **Note:** -If the document being moved has any Disqus comments on it, there are extra steps -to follow documented just [below](#redirections-for-pages-with-disqus-comments). - -Things to note: - -- Since we also use inline documentation, except for the documentation itself, - the document might also be referenced in the views of GitLab (`app/`) which will - render when visiting `/help`, and sometimes in the testing suite (`spec/`). -- The above `git grep` command will search recursively in the directory you run - it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` - and will print the file and the line where this file is mentioned. - You may ask why the two greps. Since we use relative paths to link to - documentation, sometimes it might be useful to search a path deeper. -- The `*.md` extension is not used when a document is linked to GitLab's - built-in help page, that's why we omit it in `git grep`. -- Use the checklist on the documentation MR description template. - -#### Alternative redirection method - -Alternatively to the method described above, you can simply replace the content -of the old file with a frontmatter containing a redirect link: - -```yaml ---- -redirect_to: '../path/to/file/README.md' ---- -``` - -It supports both full and relative URLs, e.g. `https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. Note that any `*.md` paths will be compiled to `*.html`. - -### Redirections for pages with Disqus comments - -If the documentation page being relocated already has any Disqus comments, -we need to preserve the Disqus thread. - -Disqus uses an identifier per page, and for docs.gitlab.com, the page identifier -is configured to be the page URL. Therefore, when we change the document location, -we need to preserve the old URL as the same Disqus identifier. - -To do that, add to the frontmatter the variable `redirect_from`, -using the old URL as value. For example, let's say I moved the document -available under `https://docs.gitlab.com/my-old-location/README.html` to a new location, -`https://docs.gitlab.com/my-new-location/index.html`. - -Into the **new document** frontmatter add the following: - -```yaml ---- -redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' ---- -``` - -Note: it is necessary to include the file name in the `redirect_from` URL, -even if it's `index.html` or `README.html`. - -## Testing - -We treat documentation as code, thus have implemented some testing. -Currently, the following tests are in place: - -1. `docs lint`: Check that all internal (relative) links work correctly and - that all cURL examples in API docs use the full switches. It's recommended - to [check locally](#previewing-locally) before pushing to GitLab by executing the command - `bundle exec nanoc check internal_links` on your local - [`gitlab-docs`](https://gitlab.com/gitlab-com/gitlab-docs) directory. -1. [`ee_compat_check`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#avoiding-ce-gt-ee-merge-conflicts-beforehand) (runs on CE only): - When you submit a merge request to GitLab Community Edition (CE), - there is this additional job that runs against Enterprise Edition (EE) - and checks if your changes can apply cleanly to the EE codebase. - If that job fails, read the instructions in the job log for what to do next. - As CE is merged into EE once a day, it's important to avoid merge conflicts. - Submitting an EE-equivalent merge request cherry-picking all commits from CE to EE is - essential to avoid them. - -## Branch naming - -If your contribution contains **only** documentation changes, you can speed up -the CI process by following some branch naming conventions. You have three -choices: - -| Branch name | Valid example | -| ----------- | ------------- | -| Starting with `docs/` | `docs/update-api-issues` | -| Starting with `docs-` | `docs-update-api-issues` | -| Ending in `-docs` | `123-update-api-issues-docs` | - -If your branch name matches any of the above, it will run only the docs -tests. If it doesn't, the whole test suite will run (including docs). - -## Merge requests for GitLab documentation - -Before getting started, make sure you read the introductory section -"[contributing to docs](#contributing-to-docs)" above and the -[tech writing workflow](https://about.gitlab.com/handbook/product/technical-writing/workflow/) -for GitLab Team members. - -- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/merge_request_templates/Documentation.md) -- Use the correct [branch name](#branch-naming) -- Label the MR `Documentation` -- Assign the correct milestone (see note below) - - -NOTE: **Note:** -If the release version you want to add the documentation to has already been -frozen or released, use the label `Pick into X.Y` to get it merged into -the correct release. Avoid picking into a past release as much as you can, as -it increases the work of the release managers. - -### Cherry-picking from CE to EE - -As we have the `master` branch of CE merged into EE once a day, it's common to -run into merge conflicts. To avoid them, we [test for merge conflicts against EE](#testing) -with the `ee-compat-check` job, and use the following method of creating equivalent -branches for CE and EE. - -Follow this [method for cherry-picking from CE to EE](automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee), with a few adjustments: - -- Create the [CE branch](#branch-naming) starting with `docs-`, - e.g.: `git checkout -b docs-example` -- Create the EE-equivalent branch ending with `-ee`, e.g., - `git checkout -b docs-example-ee` -- Once all the jobs are passing in CE and EE, and you've addressed the -feedback from your own team, assign the CE MR to a technical writer for review -- When both MRs are ready, the EE merge request will be merged first, and the -CE-equivalent will be merged next. -- Note that the review will occur only in the CE MR, as the EE MR -contains the same commits as the CE MR. -- If you have a few more changes that apply to the EE-version only, you can submit -a couple more commits to the EE branch, but ask the reviewer to review the EE merge request -additionally to the CE MR. If there are many EE-only changes though, start a new MR -to EE only. - -## Previewing the changes live - -To preview your changes to documentation locally, please follow -this [development guide](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md#development). - -If you want to preview the doc changes of your merge request live, you can use -the manual `review-docs-deploy` job in your merge request. You will need at -least Master permissions to be able to run it and is currently enabled for the -following projects: - -- https://gitlab.com/gitlab-org/gitlab-ce -- https://gitlab.com/gitlab-org/gitlab-ee - -NOTE: **Note:** -You will need to push a branch to those repositories, it doesn't work for forks. - -TIP: **Tip:** -If your branch contains only documentation changes, you can use -[special branch names](#branch-naming) to avoid long running pipelines. - -In the mini pipeline graph, you should see an `>>` icon. Clicking on it will -reveal the `review-docs-deploy` job. Hit the play button for the job to start. - - - -This job will: - -1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) - project named after the scheme: `preview-<branch-slug>` -1. Trigger a cross project pipeline and build the docs site with your changes - -After a few minutes, the Review App will be deployed and you will be able to -preview the changes. The docs URL can be found in two places: - -- In the merge request widget -- In the output of the `review-docs-deploy` job, which also includes the - triggered pipeline so that you can investigate whether something went wrong - -In case the Review App URL returns 404, follow these steps to debug: - -1. **Did you follow the URL from the merge request widget?** If yes, then check if - the link is the same as the one in the job output. It can happen that if the - branch name slug is longer than 35 characters, it is automatically - truncated. That means that the merge request widget will not show the proper - URL due to a limitation of how `environment: url` works, but you can find the - real URL from the output of the `review-docs-deploy` job. -1. **Did you follow the URL from the job output?** If yes, then it means that - either the site is not yet deployed or something went wrong with the remote - pipeline. Give it a few minutes and it should appear online, otherwise you - can check the status of the remote pipeline from the link in the job output. - If the pipeline failed or got stuck, drop a line in the `#docs` chat channel. - -TIP: **Tip:** -Someone that has no merge rights to the CE/EE projects (think of forks from -contributors) will not be able to run the manual job. In that case, you can -ask someone from the GitLab team who has the permissions to do that for you. - -NOTE: **Note:** -Make sure that you always delete the branch of the merge request you were -working on. If you don't, the remote docs branch won't be removed either, -and the server where the Review Apps are hosted will eventually be out of -disk space. - -### Technical aspects - -If you want to know the hot details, here's what's really happening: - -1. You manually run the `review-docs-deploy` job in a CE/EE merge request. -1. The job runs the [`scripts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs) - script with the `deploy` flag, which in turn: - 1. Takes your branch name and applies the following: - - The slug of the branch name is used to avoid special characters since - ultimately this will be used by NGINX. - - The `preview-` prefix is added to avoid conflicts if there's a remote branch - with the same name that you created in the merge request. - - The final branch name is truncated to 42 characters to avoid filesystem - limitations with long branch names (> 63 chars). - 1. The remote branch is then created if it doesn't exist (meaning you can - re-run the manual job as many times as you want and this step will be skipped). - 1. A new cross-project pipeline is triggered in the docs project. - 1. The preview URL is shown both at the job output and in the merge request - widget. You also get the link to the remote pipeline. -1. In the docs project, the pipeline is created and it - [skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55) - to lower the build time. -1. Once the docs site is built, the HTML files are uploaded as artifacts. -1. A specific Runner tied only to the docs project, runs the Review App job - that downloads the artifacts and uses `rsync` to transfer the files over - to a location where NGINX serves them. - -The following GitLab features are used among others: - -- [Manual actions](../ci/yaml/README.md#manual-actions) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) -- [Review Apps](../ci/review_apps/index.md) -- [Artifacts](../ci/yaml/README.md#artifacts) -- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) - -## GitLab `/help` - -Every GitLab instance includes the documentation, which is available from `/help` -(`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. - -When you're building a new feature, you may need to link the documentation -from GitLab, the application. This is normally done in files inside the -`app/views/` directory with the help of the `help_page_path` helper method. - -In its simplest form, the HAML code to generate a link to the `/help` page is: - -```haml -= link_to 'Help page', help_page_path('user/permissions') -``` - -The `help_page_path` contains the path to the document you want to link to with -the following conventions: - -- it is relative to the `doc/` directory in the GitLab repository -- the `.md` extension must be omitted -- it must not end with a slash (`/`) - -Below are some special cases where should be used depending on the context. -You can combine one or more of the following: - -1. **Linking to an anchor link.** Use `anchor` as part of the `help_page_path` - method: - - ```haml - = link_to 'Help page', help_page_path('user/permissions', anchor: 'anchor-link') - ``` - -1. **Opening links in a new tab.** This should be the default behavior: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), target: '_blank' - ``` - -1. **Linking to a circle icon.** Usually used in settings where a long - description cannot be used, like near checkboxes. You can basically use - any font awesome icon, but prefer the `question-circle`: - - ```haml - = link_to icon('question-circle'), help_page_path('user/permissions') - ``` - -1. **Using a button link.** Useful in places where text would be out of context - with the rest of the page layout: - - ```haml - = link_to 'Help page', help_page_path('user/permissions'), class: 'btn btn-info' - ``` - -1. **Using links inline of some text.** - - ```haml - Description to #{link_to 'Help page', help_page_path('user/permissions')}. - ``` - -1. **Adding a period at the end of the sentence.** Useful when you don't want - the period to be part of the link: - - ```haml - = succeed '.' do - Learn more in the - = link_to 'Help page', help_page_path('user/permissions') - ``` - -## General Documentation vs Technical Articles - -### General documentation - -General documentation is categorized by _User_, _Admin_, and _Contributor_, and describe what that feature is, what it does, and its available settings. - -### Technical Articles - -Technical articles replace technical content that once lived in the [GitLab Blog](https://about.gitlab.com/blog/), where they got out-of-date and weren't easily found. - -They are topic-related documentation, written with an user-friendly approach and language, aiming to provide the community with guidance on specific processes to achieve certain objectives. - -A technical article guides users and/or admins to achieve certain objectives (within guides and tutorials), or provide an overview of that particular topic or feature (within technical overviews). It can also describe the use, implementation, or integration of third-party tools with GitLab. - -They should be placed in a new directory named `/article-title/index.md` under a topic-related folder, and their images should be placed in `/article-title/img/`. For example, a new article on GitLab Pages should be placed in `doc/user/project/pages/article-title/` and a new article on GitLab CI/CD should be placed in `doc/ci/examples/article-title/`. - -#### Types of Technical Articles - -- **User guides**: technical content to guide regular users from point A to point B -- **Admin guides**: technical content to guide administrators of GitLab instances from point A to point B -- **Technical Overviews**: technical content describing features, solutions, and third-party integrations -- **Tutorials**: technical content provided step-by-step on how to do things, or how to reach very specific objectives - -#### Understanding guides, tutorials, and technical overviews - -Suppose there's a process to go from point A to point B in 5 steps: `(A) 1 > 2 > 3 > 4 > 5 (B)`. - -A **guide** can be understood as a description of certain processes to achieve a particular objective. A guide brings you from A to B describing the characteristics of that process, but not necessarily going over each step. It can mention, for example, steps 2 and 3, but does not necessarily explain how to accomplish them. - -- Live example: "[Static sites and GitLab Pages domains (Part 1)](../user/project/pages/getting_started_part_one.md) to [Creating and Tweaking GitLab CI/CD for GitLab Pages (Part 4)](../user/project/pages/getting_started_part_four.md)" - -A **tutorial** requires a clear **step-by-step** guidance to achieve a singular objective. It brings you from A to B, describing precisely all the necessary steps involved in that process, showing each of the 5 steps to go from A to B. -It does not only describes steps 2 and 3, but also shows you how to accomplish them. - -- Live example (on the blog): [Hosting on GitLab.com with GitLab Pages](https://about.gitlab.com/2016/04/07/gitlab-pages-setup/) - -A **technical overview** is a description of what a certain feature is, and what it does, but does not walk -through the process of how to use it systematically. - -- Live example (on the blog): [GitLab Workflow, an overview](https://about.gitlab.com/2016/10/25/gitlab-workflow-an-overview/) - -#### Special format - -Every **Technical Article** contains a frontmatter at the beginning of the doc -with the following information: - -- **Type of article** (user guide, admin guide, technical overview, tutorial) -- **Knowledge level** expected from the reader to be able to follow through (beginner, intermediate, advanced) -- **Author's name** and **GitLab.com handle** -- **Publication date** (ISO format YYYY-MM-DD) - -For example: - - -```yaml ---- -author: John Doe -author_gitlab: johnDoe -level: beginner -article_type: user guide -date: 2017-02-01 ---- -``` - -#### Technical Articles - Writing Method - -Use the [writing method](https://about.gitlab.com/handbook/product/technical-writing/#writing-method) defined by the Technical Writing team. - -[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png -[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle diff --git a/doc/downgrade_ee_to_ce/README.md b/doc/downgrade_ee_to_ce/README.md index 236408762e3..a187b3cbb07 100644 --- a/doc/downgrade_ee_to_ce/README.md +++ b/doc/downgrade_ee_to_ce/README.md @@ -57,7 +57,7 @@ $ sudo gitlab-rails runner "Service.where(type: ['JenkinsService', 'JenkinsDepre $ bundle exec rails runner "Service.where(type: ['JenkinsService', 'JenkinsDeprecatedService', 'GithubService']).delete_all" production ``` -### Secret variables environment scopes +### Variables environment scopes If you're using this feature and there are variables sharing the same key, but they have different scopes in a project, then you might want to diff --git a/doc/integration/github.md b/doc/integration/github.md index 23bb8ef9303..680712f9e01 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -110,7 +110,7 @@ On the sign in page there should now be a GitHub icon below the regular sign in Click the icon to begin the authentication process. GitHub will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. -### GitHub Enterprise with Self-Signed Certificate +## GitHub Enterprise with self-signed Certificate If you are attempting to import projects from GitHub Enterprise with a self-signed certificate and the imports are failing, you will need to disable SSL verification. diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index eec40a9b8f1..70087576678 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -7,13 +7,11 @@ GitLab.com will generate an application ID and secret key for you to use. 1. Sign in to GitLab.com -1. Navigate to your profile settings. +1. On the upper right corner, click on your avatar and go to your **Settings**. -1. Select "Applications" in the left menu. +1. Select **Applications** in the left menu. -1. Select "New application". - -1. Provide the required details. +1. Provide the required details for **Add new application**. - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Redirect URI: @@ -24,9 +22,9 @@ GitLab.com will generate an application ID and secret key for you to use. The first link is required for the importer and second for the authorization. -1. Select "Submit". +1. Select **Save application**. -1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). +1. You should now see a **Application Id** and **Secret** near the top right of the page (see screenshot). Keep this page open as you continue configuration.  diff --git a/doc/integration/img/gitlab_app.png b/doc/integration/img/gitlab_app.png Binary files differindex b4958581a9b..8d6a4456fc4 100644 --- a/doc/integration/img/gitlab_app.png +++ b/doc/integration/img/gitlab_app.png diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 5faae7ca2d6..95221d8b6b1 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -492,8 +492,8 @@ directory (repositories, uploads). To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` (for Omnibus packages) or `/home/git/gitlab/.secret` (for installations from source). This file contains the database encryption key, -[CI secret variables](../ci/variables/README.md#secret-variables), and -secret variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). +[CI/CD variables](../ci/variables/README.md#variables), and +variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). If you fail to restore this encryption key file along with the application data backup, users with two-factor authentication enabled and GitLab Runners will lose access to your GitLab server. diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md index f8e7fc3fd0e..22756232025 100644 --- a/doc/security/information_exclusivity.md +++ b/doc/security/information_exclusivity.md @@ -2,7 +2,7 @@ Git is a distributed version control system (DVCS). This means that everyone that works with the source code has a local copy of the complete repository. -In GitLab every project member that is not a guest (so reporters, developers and masters) can clone the repository to get a local copy. +In GitLab every project member that is not a guest (so reporters, developers and maintainers) can clone the repository to get a local copy. After obtaining this local copy the user can upload the full repository anywhere, including another project under their control or another server. The consequence is that you can't build access controls that prevent the intentional sharing of source code by users that have access to the source code. This is an inherent feature of a DVCS and all git management systems have this limitation. diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index a573445ab5b..b17b0a4bc4a 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -2,7 +2,7 @@ If you have non-GitLab web services running on your GitLab server or within its local network, these may be vulnerable to exploitation via Webhooks. -With [Webhooks](../user/project/integrations/webhooks.md), you and your project masters and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. +With [Webhooks](../user/project/integrations/webhooks.md), you and your project maintainers and owners can set up URLs to be triggered when specific things happen to projects. Normally, these requests are sent to external web services specifically set up for this purpose, that process the request and its attached data in some appropriate way. Things get hairy, however, when a Webhook is set up with a URL that doesn't point to an external, but to an internal service, that may do something completely unintended when the webhook is triggered and the POST request is sent. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index b71e9bf3000..bab196e7609 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -171,7 +171,7 @@ This is really useful for cloning repositories to your Continuous Integration (CI) server. By using deploy keys, you don't have to set up a dummy user account. -If you are a project master or owner, you can add a deploy key in the +If you are a project maintainer or owner, you can add a deploy key in the project settings under the section 'Repository'. Specify a title for the new deploy key and paste a public SSH key. After this, the machine that uses the corresponding private SSH key has read-only or read-write (if enabled) @@ -196,7 +196,7 @@ This is really useful for integrating repositories to secured, shared Continuous Integration (CI) services or other shared services. GitLab administrators can set up the Global Shared Deploy key in GitLab and add the private key to any shared systems. Individual repositories opt into -exposing their repository using these keys when a project masters (or higher) +exposing their repository using these keys when a project maintainers (or higher) authorizes a Global Shared Deploy key to be used with their project. Global Shared Keys can provide greater security compared to Per-Project Deploy @@ -205,7 +205,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be -the primary way for project masters and owners to identify the correct Global +the primary way for project maintainers and owners to identify the correct Global Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity @@ -213,7 +213,7 @@ of keys - they could be of very narrow usage such as just a specific service or of broader usage for something like "Anywhere you need to give read access to your repository". -Once a GitLab administrator adds the Global Deployment key, project masters +Once a GitLab administrator adds the Global Deployment key, project maintainers and owners can add it in project's **Settings > Repository** section by expanding the **Deploy Key** section and clicking **Enable** next to the appropriate key listed under **Public deploy keys available to any project**. diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index 9ba05c7815b..cce02d218c2 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -138,7 +138,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -158,7 +158,7 @@ Please refer to `group_rename` and `user_rename` for that case. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_team", - "project_access": "Master", + "project_access": "Maintainer", "project_id": 74, "project_name": "StoreCloud", "project_path": "storecloud", @@ -318,7 +318,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_add_to_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", @@ -335,7 +335,7 @@ If the user is blocked via LDAP, `state` will be `ldap_blocked`. "created_at": "2012-07-21T07:30:56Z", "updated_at": "2012-07-21T07:38:22Z", "event_name": "user_remove_from_group", - "group_access": "Master", + "group_access": "Maintainer", "group_id": 78, "group_name": "StoreCloud", "group_path": "storecloud", diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index efec365042a..fec575f263f 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -437,7 +437,7 @@ repo or by specifying a project variable: file in it, Auto DevOps will detect the chart and use it instead of the [default one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). This can be a great way to control exactly how your application is deployed. -- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) +- **Project variable** - Create a [variable](../../ci/variables/README.md#variables) `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. ### Customizing `.gitlab-ci.yml` @@ -497,17 +497,17 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | | `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. | | `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | -| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | +| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `code_quality` job. If the variable is present, the job will not be created. | | `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | | `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | -| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `container_scanning` job. If the variable is present, the job will not be created. | | `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | | `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | | `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | TIP: **Tip:** Set up the replica variables using a -[project variable](../../ci/variables/README.md#secret-variables) +[project variable](../../ci/variables/README.md#variables) and scale your application by just redeploying it! CAUTION: **Caution:** @@ -582,7 +582,7 @@ staging environment and deploy to production manually. For this scenario, the `STAGING_ENABLED` environment variable was introduced. If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to -`1` as a secret variable), then the application will be automatically deployed +`1` as a variable), then the application will be automatically deployed to a `staging` environment, and a `production_manual` job will be created for you when you're ready to manually deploy to production. @@ -595,7 +595,7 @@ A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployment before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to -`1` as a secret variable) then two manual jobs will be created: +`1` as a variable) then two manual jobs will be created: - `canary` which will deploy the application to the canary environment - `production_manual` which is to be used by you when you're ready to manually @@ -611,7 +611,7 @@ This will allow you to first check how the app is behaving, and later manually increasing the rollout up to 100%. If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set -`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the +`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a variable), then instead of the standard `production` job, 4 different [manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph) will be created: diff --git a/doc/update/10.6-to-10.7.md b/doc/update/10.6-to-10.7.md index 4a76ae14d2e..4efbb8c65cf 100644 --- a/doc/update/10.6-to-10.7.md +++ b/doc/update/10.6-to-10.7.md @@ -80,8 +80,8 @@ More information can be found on the [yarn website](https://yarnpkg.com/en/docs/ ### 5. Update Go -NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go -1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. +NOTE: GitLab 9.2 and higher only supports Go 1.9 and dropped support for Go +1.5.x through 1.8.x. Be sure to upgrade your installation if necessary. You can check which version you are running with `go version`. @@ -91,11 +91,11 @@ Download and install Go: # Remove former Go installation folder sudo rm -rf /usr/local/go -curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz -echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ - sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +curl --remote-name --progress https://storage.googleapis.com/golang/go1.9.linux-amd64.tar.gz +echo 'd70eadefce8e160638a9a6db97f7192d8463069ab33138893ad3bf31b0650a79 go1.9.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.9.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ -rm go1.8.3.linux-amd64.tar.gz +rm go1.9.linux-amd64.tar.gz ``` ### 6. Get latest code diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 159109e8954..9b0ff02f227 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -11,7 +11,7 @@ You can leave a comment in the following places: - commit diffs The comment area supports [Markdown] and [quick actions]. One can edit their -own comment at any time, and anyone with [Master access level][permissions] or +own comment at any time, and anyone with [Maintainer access level][permissions] or higher can also edit a comment made by someone else. You could also reply to the notification email in order to reply to a comment, @@ -253,7 +253,7 @@ to newer issues or merge requests. - The people participating in the discussion are trolling, abusive, or otherwise being unproductive. -In these cases, a user with Master permissions or higher in the project can lock (and unlock) +In these cases, a user with Maintainer permissions or higher in the project can lock (and unlock) an issue or a merge request, using the "Lock" section in the sidebar: | Unlock | Lock | diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 30761a66563..e6bf32a2dc5 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -125,7 +125,7 @@ side of your screen. --- -Group owners and masters will be notified of your request and will be able to approve or +Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 02f8ef08117..08849ac1df4 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -145,8 +145,8 @@ permissions. For example, if User0 was first added to group `group-1/group-1-1` with Developer permissions, then they will inherit those permissions in every other subgroup -of `group-1/group-1-1`. To give them Master access to `group-1/group-1-1/group1-1-1`, -you would add them again in that group as Master. Removing them from that group, +of `group-1/group-1-1`. To give them Maintainer access to `group-1/group-1-1/group1-1-1`, +you would add them again in that group as Maintainer. Removing them from that group, the permissions will fallback to those of the ancestor group. ## Mentioning subgroups diff --git a/doc/user/index.md b/doc/user/index.md index a50e5e8fbf8..9be3c6f683c 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -23,7 +23,7 @@ documentation. GitLab is a fully integrated software development platform that enables you and your team to work cohesively, faster, transparently, and effectively, since the discussion of a new idea until taking that idea to production all -all the way through, from within the same platform. +the way through, from within the same platform. Please check this page for an overview on [GitLab's features](https://about.gitlab.com/features/). @@ -110,7 +110,7 @@ personal access tokens, authorized applications, etc. - [Authentication](../topics/authentication/index.md): Read through the authentication methods available in GitLab. - [Permissions](permissions.md): Learn the different set of permissions levels for each -user type (guest, reporter, developer, master, owner). +user type (guest, reporter, developer, maintainer, owner). - [Feature highlight](feature_highlight.md): Learn more about the little blue dots around the app that explain certain features diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 61dd0fbaed1..16c19855136 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -27,7 +27,7 @@ See our [product handbook on permissions](https://about.gitlab.com/handbook/prod The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer |Maintainer| Owner | |---------------------------------------|---------|------------|-------------|----------|--------| | Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | @@ -109,7 +109,7 @@ review, we've created protected branches. Read through the documentation on [protected branches](project/protected_branches.md) to learn more. -Additionally, you can allow or forbid users with Master and/or +Additionally, you can allow or forbid users with Maintainer and/or Developer permissions to push to a protected branch. Read through the documentation on [Allowed to Merge and Allowed to Push settings](project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings) to learn more. @@ -150,7 +150,7 @@ Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a group. -| Action | Guest | Reporter | Developer | Master | Owner | +| Action | Guest | Reporter | Developer | Maintainer | Owner | |-------------------------|-------|----------|-----------|--------|-------| | Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | | Edit group | | | | | ✓ | @@ -200,7 +200,7 @@ GitLab CI/CD permissions rely on the role the user has in GitLab. There are four permission levels in total: - admin -- master +- maintainer - developer - guest/reporter @@ -208,7 +208,7 @@ The admin user can perform any action on GitLab CI/CD in scope of the GitLab instance and project. In addition, all admins can use the admin interface under `/admin/runners`. -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | @@ -230,7 +230,7 @@ Read all about the [new model and its implications][new-mod]. This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer | Master | Admin | +| Action | Guest, Reporter | Developer |Maintainer| Admin | |---------------------------------------------|-----------------|-------------|----------|--------| | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | @@ -276,7 +276,7 @@ only. [^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** -[^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner +[^4]: Not allowed for Guest, Reporter, Developer, Maintainer, or Owner [^5]: Only if the job was triggered by the user [^6]: Only if user is not external one [^7]: Only if user is a member of the project diff --git a/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png Binary files differnew file mode 100644 index 00000000000..9a0559a19d4 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/add_cluster.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png Binary files differnew file mode 100644 index 00000000000..657ab0d9fa9 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_dns.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/create_project.png b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png Binary files differnew file mode 100644 index 00000000000..f3446131419 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/create_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png Binary files differnew file mode 100644 index 00000000000..d6c3b1b3a94 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/deploy_apps.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/environment.png b/doc/user/project/clusters/eks_and_gitlab/img/environment.png Binary files differnew file mode 100644 index 00000000000..77d711ba8f6 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/environment.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/new_project.png b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png Binary files differnew file mode 100644 index 00000000000..d401c4ac2bf --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/new_project.png diff --git a/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png Binary files differnew file mode 100644 index 00000000000..5f9c9815c24 --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/img/pipeline.png diff --git a/doc/user/project/clusters/eks_and_gitlab/index.md b/doc/user/project/clusters/eks_and_gitlab/index.md new file mode 100644 index 00000000000..f50615729dd --- /dev/null +++ b/doc/user/project/clusters/eks_and_gitlab/index.md @@ -0,0 +1,136 @@ +--- +redirect_from: 'https://docs.gitlab.com/ee/user/project/clusters/eks_and_gitlab/index.html' +--- + +# Connecting and deploying to an Amazon EKS cluster + +> **[Article Type](../../../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediate || +> **Author:** [Joshua Lambert](https://gitlab.com/joshlambert) || +> **Publication date:** 2018-06-05 + +## Introduction + +In this tutorial, we will show how easy it is to integrate an [Amazon EKS](https://aws.amazon.com/eks/) cluster with GitLab, and begin deploying applications. + +For an end-to-end walkthrough we will: +1. Start with a new project based on the sample Ruby on Rails template +1. Integrate an EKS cluster +1. Utilize [Auto DevOps](../../../../topics/autodevops/) to build, test, and deploy our application + +You will need: +1. An account on GitLab, like [GitLab.com](https://gitlab.com) +1. An Amazon EKS cluster +1. `kubectl` [installed and configured for access to the EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl) + +If you don't have an Amazon EKS cluster, one can be created by following [the EKS getting started guide](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html). + +## Creating a new project + +On GitLab, create a new project by clicking on the `+` icon in the top navigation bar, and selecting `New project`. + + + +On the new project screen, click on the `Create from template` tab, and select `Use template` for the Ruby on Rails sample project. + +Give the project a name, and then select `Create project`. + + + +## Connecting the EKS cluster + +From the left side bar, hover over `CI/CD` and select `Kubernetes`, then click on `Add Kubernetes cluster`, and finally `Add an existing Kubernetes cluster`. + +A few details from the EKS cluster will be required to connect it to GitLab. + +1. A valid Kubernetes certificate and token are needed to authenticate to the EKS cluster. A pair is created by default, which can be used. Open a shell and use `kubectl` to retrieve them: + * List the secrets with `kubectl get secrets`, and one should named similar to `default-token-xxxxx`. Copy that token name for use below. + * Get the certificate with `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 -D` + * Retrieve the token with `kubectl get secret <secret name> -o jsonpath="{['data']['token']}" | base64 -D`. +1. The API server endpoint is also required, so GitLab can connect to the cluster. This is displayed on the AWS EKS console, when viewing the EKS cluster details. + +You now have all the information needed to connect the EKS cluster: +* Kubernetes cluster name: Provide a name for the cluster to identify it within GitLab. +* Environment scope: Leave this as `*` for now, since we are only connecting a single cluster. +* API URL: Paste in the API server endpoint retrieved above. +* CA Certificate: Paste the certificate data from the earlier step, as-is. +* Paste the token value. Note on some versions of Kubernetes a trailing `%` is output, do not include it. +* Project namespace: This can be left blank to accept the default namespace, based on the project name. + + + +Click on `Add Kubernetes cluster`, the cluster is now connected to GitLab. At this point, [Kubernetes deployment variables](../#deployment-variables) will automatically be available during CI jobs, making it easy to interact with the cluster. + +If you would like to utilize your own CI/CD scripts to deploy to the cluster, you can stop here. + +## Disable Role Based-Access Control (RBAC) + +Presently, Auto DevOps and one-click app installs do not support [Kubernetes role-based access control](https://kubernetes.io/docs/reference/access-authn-authz/rbac/). Support is [being worked on](https://gitlab.com/groups/gitlab-org/-/epics/136), but in the interim RBAC must be disabled to utilize for these features. + +> **Note**: Disabling RBAC means that any application running in the cluster, or user who can authenticate to the cluster, has full API access. This is a [security concern](https://docs.gitlab.com/ee/user/project/clusters/#security-implications), and may not be desirable. + +To effectively disable RBAC, global permissions can be applied granting full access: + +```bash +kubectl create clusterrolebinding permissive-binding \ + --clusterrole=cluster-admin \ + --user=admin \ + --user=kubelet \ + --group=system:serviceaccounts +``` + +## Deploy services to the cluster + +GitLab supports one-click deployment of helpful services to the cluster, many of which support Auto DevOps. Back on the Kubernetes cluster screen in GitLab, a list of applications is now available to deploy. + +First install Helm Tiller, a package manager for Kubernetes. This enables deployment of the other applications. + + + +### Deploying NGINX Ingress (optional) + +Next, if you would like the deployed app to be reachable on the internet, deploy the Ingress. Note that this will also cause an [Elastic Load Balancer](https://aws.amazon.com/documentation/elastic-load-balancing/) to be created, which will incur additional AWS costs. + +Once installed, you may see a `?` for `Ingress IP Address`. This is because the created ELB is available at a DNS name, not an IP address. To get the DNS name, run: `kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. Note, you may see a trailing `%` on some Kubernetes versions, do not include it. + +The Ingress is now available at this address, and will route incoming requests to the proper service based on the DNS name in the request. To support this, a wildcard DNS CNAME record should be created for the desired domain name. For example `*.myekscluster.com` would point to the Ingress hostname obtained earlier. + + + +### Deploying the GitLab Runner (optional) + +If the project is on GitLab.com, free shared runners are available and you do not have to deploy one. If a project specific runner is desired, or there are no shared runners, it is easy to deploy one. + +Simply click on the `Install` button for the GitLab Runner. It is important to note that the runner deployed is set as **privileged**, which means it essentially has root access to the underlying machine. This is required to build docker images, and so is on by default. + +### Deploying Prometheus (optional) + +GitLab is able to monitor applications automatically, utilizing [Prometheus](../../integrations/prometheus.html). Kubernetes container CPU and memory metrics are automatically collected, and response metrics are retrieved from NGINX Ingress as well. + +To enable monitoring, simply install Prometheus into the cluster with the `Install` button. + +## Create a default Storage Class + +Amazon EKS does not have a default Storage Class out of the box, which means requests for persistent volumes will not be automatically fulfilled. As part of Auto DevOps, the deployed Postgres instance requests persistent storage, and without a default storage class it will fail to start. + +If a default Storage Class does not already exist and is desired, follow Amazon's [short guide](https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html) to create one. + +Alternatively, disable Postgres by setting the project variable [`POSTGRES_ENABLED`](../../../../topics/autodevops/#environment-variables) to `false`. + +## Deploy the app to EKS + +With RBAC disabled and services deployed, [Auto DevOps](https://docs.gitlab.com/ee/topics/autodevops/) can now be leveraged to build, test, and deploy the app. To enable, click on `Settings` in the left sidebar, then `CI/CD`. You will see a section for `Auto DevOps`, expand it. Click on the radio button to `Enable Auto DevOps`. + +If a wildcard DNS entry was created resolving to the Load Balancer, enter it in the `domain` field. Otherwise, the deployed app will not be externally available outside of the cluster. To save, click `Save changes`. + + + +A new pipeline will automatically be created, which will begin to build, test, and deploy the app. + +After the pipeline has finished, your app will be running in EKS and available to users. Click on `CI/CD` tab in the left navigation bar, and choose `Environments`. + + + +You will see a list of the environments and their deploy status, as well as options to browse to the app, view monitoring metrics, and even access a shell on the running pod. + +To learn more about Auto DevOps, review our [documentation](../../../../topics/autodevops/). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 65cdece8d3d..1efbe20b84f 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -19,7 +19,7 @@ or provide the credentials to an [existing Kubernetes cluster](#adding-an-existi ## Adding and creating a new GKE cluster via GitLab NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. Before proceeding, make sure the following requirements are met: @@ -30,7 +30,7 @@ Before proceeding, make sure the following requirements are met: clusters on GKE. That would mean that a [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) must be set up and that you have to have permissions to access it. -- You must have Master [permissions] in order to be able to access the +- You must have Maintainer [permissions] in order to be able to access the **Kubernetes** page. - You must have [Cloud Billing API](https://cloud.google.com/billing/) enabled - You must have [Resource Manager @@ -66,7 +66,7 @@ enable the Cluster integration. ## Adding an existing Kubernetes cluster NOTE: **Note:** -You need Master [permissions] and above to access the Kubernetes page. +You need Maintainer [permissions] and above to access the Kubernetes page. To add an existing Kubernetes cluster to your project: @@ -201,6 +201,11 @@ Otherwise, you can list the IP addresses of all load balancers: kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} ' ``` +> **Note**: Some Kubernetes clusters return a hostname instead, like [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run: +> ```bash +> kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}"`. +> ``` + The output is the external IP address of your cluster. This information can then be used to set up DNS entries and forwarding rules that allow external access to your deployed applications. @@ -233,7 +238,7 @@ When adding more than one Kubernetes clusters to your project, you need to differentiate them with an environment scope. The environment scope associates clusters and [environments](../../../ci/environments.md) in an 1:1 relationship similar to how the -[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-secret-variables) +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-variables) work. The default environment scope is `*`, which means all jobs, regardless of their @@ -325,7 +330,7 @@ To disable the Kubernetes cluster integration, follow the same procedure. ## Removing the Kubernetes cluster integration NOTE: **Note:** -You need Master [permissions] and above to remove a Kubernetes cluster integration. +You need Maintainer [permissions] and above to remove a Kubernetes cluster integration. NOTE: **Note:** When you remove a cluster, you only remove its relation to GitLab, not the @@ -382,7 +387,7 @@ you will need the Kubernetes project integration enabled. ### Web terminals NOTE: **Note:** -Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +Introduced in GitLab 8.15. You must be the project owner or have `maintainer` permissions to use terminals. Support is limited to the first container in the first pod of your environment. diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 9c5e3509046..03302b3815d 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -143,6 +143,24 @@ docker login registry.example.com -u <your_username> -p <your_access_token> for errors (e.g. `/var/log/gitlab/gitlab-rails/production.log`). You may be able to find clues there. +#### Enable the registry debug server + +The optional debug server can be enabled by setting the registry debug address +in your `gitlab.rb` configuration. + +```ruby +registry['debug_addr'] = "localhost:5001" +``` + +After adding the setting, [reconfigure] GitLab to apply the change. + +Use curl to request debug output from the debug server: + +```bash +curl localhost:5001/debug/health +curl localhost:5001/debug/vars +``` + ### Advanced Troubleshooting >**NOTE:** The following section is only recommended for experts. @@ -275,3 +293,4 @@ Once the right permissions were set, the error will go away. [docker-docs]: https://docs.docker.com/engine/userguide/intro/ [pat]: ../profile/personal_access_tokens.md [pdt]: ../project/deploy_tokens/index.md +[reconfigure]: ../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
\ No newline at end of file diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index c09d5aeba8e..0b9b49f326f 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -5,7 +5,7 @@ Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. Please note, that the expiration of deploy tokens happens on the date you define, -at midnight UTC and that they can be only managed by [masters](https://docs.gitlab.com/ee/user/permissions.html). +at midnight UTC and that they can be only managed by [maintainers](https://docs.gitlab.com/ee/user/permissions.html). ## Creating a Deploy Token diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 8c639bd5343..cad85881c4d 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -1,154 +1,143 @@ # Import your project from GitHub to GitLab -Import your projects from GitHub to GitLab with minimal effort. +Using the importer, you can import your GitHub repositories to GitLab.com or to +your self-hosted GitLab instance. ## Overview ->**Note:** -If you are an administrator you can enable the [GitHub integration][gh-import] -in your GitLab instance sitewide. This configuration is optional, users will -still be able to import their GitHub repositories with a -[personal access token][gh-token]. - ->**Note:** -Administrators of a GitLab instance (Community or Enterprise Edition) can also -use the [GitHub rake task][gh-rake] to import projects from GitHub without the -constrains of a Sidekiq worker. - -- At its current state, GitHub importer can import: - - the repository description (GitLab 7.7+) - - the Git repository data (GitLab 7.7+) - - the issues (GitLab 7.7+) - - the pull requests (GitLab 8.4+) - - the wiki pages (GitLab 8.4+) - - the milestones (GitLab 8.7+) - - the labels (GitLab 8.7+) - - the release note descriptions (GitLab 8.12+) - - the pull request review comments (GitLab 10.2+) - - the regular issue and pull request comments -- References to pull requests and issues are preserved (GitLab 8.7+) -- Repository public access is retained. If a repository is private in GitHub - it will be created as private in GitLab as well. +NOTE: **Note:** +While these instructions will always work for users on GitLab.com, if you are an +administrator of a self-hosted GitLab instance, you will need to enable the +[GitHub integration][gh-import] in order for users to follow the preferred +import method described on this page. If this is not enabled, users can alternatively import their +GitHub repositories using a [personal access token](#using-a-github-token) from GitHub, +but this method will not be able to associate all user activity (such as issues and pull requests) +with matching GitLab users. As an administrator of a self-hosted GitLab instance, you can also use +the [GitHub rake task](../../../administration/raketasks/github_import.md) to import projects from +GitHub without the constraints of a Sidekiq worker. + +The following aspects of a project are imported: + * Repository description (GitLab.com & 7.7+) + * Git repository data (GitLab.com & 7.7+) + * Issues (GitLab.com & 7.7+) + * Pull requests (GitLab.com & 8.4+) + * Wiki pages (GitLab.com & 8.4+) + * Milestones (GitLab.com & 8.7+) + * Labels (GitLab.com & 8.7+) + * Release note descriptions (GitLab.com & 8.12+) + * Pull request review comments (GitLab.com & 10.2+) + * Regular issue and pull request comments + +References to pull requests and issues are preserved (GitLab.com & 8.7+), and +each imported repository defaults to `private` but [can be made public](../settings/index.md#sharing-and-permissions), as needed. ## How it works -When issues/pull requests are being imported, the GitHub importer tries to find -the GitHub author/assignee in GitLab's database using the GitHub ID. For this -to work, the GitHub author/assignee should have signed in beforehand in GitLab -and **associated their GitHub account**. If the user is not -found in GitLab's database, the project creator (most of the times the current -user that started the import process) is set as the author, but a reference on -the issue about the original GitHub author is kept. +When issues and pull requests are being imported, the importer attempts to find their GitHub authors and +assignees in the database of the GitLab instance (note that pull requests are called "merge requests" in GitLab). -The importer will create any new namespaces (groups) if they don't exist or in -the case the namespace is taken, the repository will be imported under the user's -namespace that started the import process. +For this association to succeed, prior to the import, each GitHub author and assignee in the repository must +have either previously logged in to a GitLab account using the GitHub icon **or** have a GitHub account with +a [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) that +matches their GitLab account's email address. -The importer will also import branches on forks of projects related to open pull -requests. These branches will be imported with a naming scheme similar to -GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepancy -in branches compared to the GitHub Repository. +If a user referenced in the project is not found in GitLab's database, the project creator (typically the user +that initiated the import process) is set as the author/assignee, but a note on the issue mentioning the original +GitHub author is added. -For a more technical description and an overview of the architecture you can -refer to [Working with the GitHub importer][gh-import-dev-docs]. +The importer creates any new namespaces (groups) if they do not exist, or, if the namespace is taken, the +repository is imported under the namespace of the user who initiated the import process. The namespace/repository +name can also be edited, with the proper permissions. -## Importing your GitHub repositories +The importer will also import branches on forks of projects related to open pull requests. These branches will be +imported with a naming scheme similar to `GH-SHA-username/pull-request-number/fork-name/branch`. This may lead to +a discrepancy in branches compared to those of the GitHub repository. -The importer page is visible when you create a new project. +For additional technical details, you can refer to the +[GitHub Importer](../../../development/github_importer.md "Working with the GitHub importer") +developer documentation. - +## Import your GitHub repository into GitLab -Click on the **GitHub** link and the import authorization process will start. -There are two ways to authorize access to your GitHub repositories: +### Using the GitHub integration -1. [Using the GitHub integration][gh-integration] (if it's enabled by your - GitLab administrator). This is the preferred way as it's possible to - preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works) - section. -1. [Using a personal access token][gh-token] provided by GitHub. +Before you begin, ensure that any GitHub users who you want to map to GitLab users have either: - +1. A GitLab account that has logged in using the GitHub icon +\- or - +2. A GitLab account with an email address that matches the [public email address](https://help.github.com/articles/setting-your-commit-email-address-on-github/) of the GitHub user -### Authorize access to your repositories using the GitHub integration +User-matching attempts occur in that order, and if a user is not identified either way, the activity is associated with +the user account that is performing the import. -If the [GitHub integration][gh-import] is enabled by your GitLab administrator, -you can use it instead of the personal access token. +NOTE: **Note:** +If you are using a self-hosted GitLab instance, this process requires that you have configured the +[GitHub integration][gh-import]. -1. First you may want to connect your GitHub account to GitLab in order for - the username mapping to be correct. -1. Once you connect GitHub, click the **List your GitHub repositories** button - and you will be redirected to GitHub for permission to access your projects. -1. After accepting, you'll be automatically redirected to the importer. +1. From the top navigation bar, click **+** and select **New project**. +2. Select the **Import project** tab and then select **GitHub**. +3. Select the first button to **List your GitHub repositories**. You are redirected to a page on github.com to authorize the GitLab application. +4. Click **Authorize gitlabhq**. You are redirected back to GitLab's Import page and all of your GitHub repositories are listed. +5. Continue on to [selecting which repositories to import](#selecting-which-repositories-to-import). -You can now go on and [select which repositories to import](#select-which-repositories-to-import). +### Using a GitHub token -### Authorize access to your repositories using a personal access token +NOTE: **Note:** +For a proper author/assignee mapping for issues and pull requests, the [GitHub integration method (above)](#using-the-github-integration) +should be used instead of the personal access token. If you are using GitLab.com or a self-hosted GitLab instance with the GitHub +integration enabled, that should be the preferred method to import your repositories. Read more in the [How it works](#how-it-works) section. ->**Note:** -For a proper author/assignee mapping for issues and pull requests, the -[GitHub integration][gh-integration] should be used instead of the -[personal access token][gh-token]. If the GitHub integration is enabled by your -GitLab administrator, it should be the preferred method to import your repositories. -Read more in the [How it works](#how-it-works) section. +If you are not using the GitHub integration, you can still perform an authorization with GitHub to grant GitLab access your repositories: -If you are not using the GitHub integration, you can still perform a one-off -authorization with GitHub to grant GitLab access your repositories: +1. Go to https://github.com/settings/tokens/new +2. Enter a token description. +3. Select the repo scope. +4. Click **Generate token**. +5. Copy the token hash. +6. Go back to GitLab and provide the token to the GitHub importer. +7. Hit the **List Your GitHub Repositories** button and wait while GitLab reads your repositories' information. + Once done, you'll be taken to the importer page to select the repositories to import. -1. Go to <https://github.com/settings/tokens/new>. -1. Enter a token description. -1. Check the `repo` scope. -1. Click **Generate token**. -1. Copy the token hash. -1. Go back to GitLab and provide the token to the GitHub importer. -1. Hit the **List Your GitHub Repositories** button and wait while GitLab reads - your repositories' information. Once done, you'll be taken to the importer - page to select the repositories to import. +### Selecting which repositories to import -### Select which repositories to import +After you have authorized access to your GitHub repositories, you are redirected to the GitHub importer page and +your GitHub repositories are listed. -After you've authorized access to your GitHub repositories, you will be -redirected to the GitHub importer page. +1. By default, the proposed repository namespaces match the names as they exist in GitHub, but based on your permissions, + you can choose to edit these names before you proceed to import any of them. +2. Select the **Import** button next to any number of repositories, or select **Import all repositories**. +3. The **Status** column shows the import status of each repository. You can choose to leave the page open and it will + update in realtime or you can return to it later. +4. Once a repository has been imported, click its GitLab path to open its GitLab URL. -From there, you can see the import statuses of your GitHub repositories. +## Mirroring and pipeline status sharing -- Those that are being imported will show a _started_ status, -- those already successfully imported will be green with a _done_ status, -- whereas those that are not yet imported will have an **Import** button on the - right side of the table. +Depending your GitLab tier, [project mirroring](../../../workflow/repository_mirroring.md) can be set up to keep +your imported project in sync with its GitHub copy. -If you want, you can import all your GitHub projects in one go by hitting -**Import all projects** in the upper left corner. +Additionally, you can configure GitLab to send pipeline status updates back GitHub with the +[GitHub Project Integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). **[PREMIUM]** - +If you import your project using [CI/CD for external repo](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/), then both +of the above are automatically configured. **[PREMIUM]** ---- +## Improving the speed of imports on self-hosted instances -You can also choose a different name for the project and a different namespace, -if you have the privileges to do so. +NOTE: **Note:** +Admin access to the GitLab server is required. -## Making the import process go faster - -For large projects it may take a while to import all data. To reduce the time -necessary you can increase the number of Sidekiq workers that process the -following queues: +For large projects it may take a while to import all data. To reduce the time necessary, you can increase the number of +Sidekiq workers that process the following queues: * `github_importer` * `github_importer_advance_stage` -For an optimal experience we recommend having at least 4 Sidekiq processes (each -running a number of threads equal to the number of CPU cores) that _only_ -process these queues. We also recommend that these processes run on separate -servers. For 4 servers with 8 cores this means you can import up to 32 objects -(e.g. issues) in parallel. +For an optimal experience, it's recommended having at least 4 Sidekiq processes (each running a number of threads equal +to the number of CPU cores) that *only* process these queues. It's also recommended that these processes run on separate +servers. For 4 servers with 8 cores this means you can import up to 32 objects (e.g., issues) in parallel. -Reducing the time spent in cloning a repository can be done by increasing -network throughput, CPU capacity, and disk performance (e.g. by using high -performance SSDs) of the disks that store the Git repositories (for your GitLab -instance). Increasing the number of Sidekiq workers will _not_ reduce the time -spent cloning repositories. +Reducing the time spent in cloning a repository can be done by increasing network throughput, CPU capacity, and disk +performance (e.g., by using high performance SSDs) of the disks that store the Git repositories (for your GitLab instance). +Increasing the number of Sidekiq workers will *not* reduce the time spent cloning repositories. [gh-import]: ../../../integration/github.md "GitHub integration" -[gh-rake]: ../../../administration/raketasks/github_import.md "GitHub rake task" -[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration -[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token -[gh-import-dev-docs]: ../../../development/github_importer.md "Working with the GitHub importer" diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index e384ed57de9..363e994f36d 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -2,7 +2,7 @@ You can find the available integrations under your project's **Settings âž” Integrations** page. You need to have at least -[master permission][permissions] on the project. +[maintainer permission][permissions] on the project. ## Project services diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 0bf1f396f9d..8eada25234f 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -71,10 +71,10 @@ least [Reporter access][permissions]. However, a guest user can also create confidential issues, but can only view the ones that they created themselves. Confidential issues are also hidden in search results for unprivileged users. -For example, here's what a user with Master and Guest access sees in the +For example, here's what a user with Maintainer and Guest access sees in the project's search results respectively. -| Master access | Guest access | +| Maintainer access | Guest access | | :-----------: | :----------: | |  |  | diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 43713855e26..2c2e8e2d556 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -4,7 +4,7 @@ You can manage the groups and users and their access levels in all of your projects. You can also personalize the access level you give each user, per-project. -You should have `master` or `owner` [permissions](../../permissions.md) to add +You should have Maintainer or Owner [permissions](../../permissions.md) to add or import a new user to your project. To view, edit, add, and remove project's members, go to your @@ -43,7 +43,7 @@ level to the project. You can import another project's users in your own project by hitting the **Import members** button on the upper right corner of the **Members** menu. -In the dropdown menu, you can see only the projects you are Master on. +In the dropdown menu, you can see only the projects you are Maintainer on.  @@ -99,7 +99,7 @@ side of your screen. --- -Project owners & masters will be notified of your request and will be able to approve or +Project owners & maintainers will be notified of your request and will be able to approve or decline it on the members page.  diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 5d819998dd9..611ff0e6bfb 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -42,7 +42,7 @@ Admins are able to share projects with any group in the system. ## Maximum access level -In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Master' or 'Owner') will only have 'Developer' access to 'Project Acme'. +In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Maintainer' or 'Owner') will only have 'Developer' access to 'Project Acme'. ## Share project with group lock diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md new file mode 100644 index 00000000000..859ac92ef89 --- /dev/null +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -0,0 +1,20 @@ +# Allow collaboration on merge requests across forks + +> [Introduced][ce-17395] in GitLab 10.6. + +This feature is available for merge requests across forked projects that are +publicly accessible. It makes it easier for members of projects to +collaborate on merge requests across forks. + +When enabled for a merge request, members with merge access to the target +branch of the project will be granted write permissions to the source branch +of the merge request. + +The feature can only be enabled by users who already have push access to the +source project, and only lasts while the merge request is open. + +Enable this functionality while creating or editing a merge request: + + + +[ce-17395]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17395 diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md index 59b3fe7242c..79444ee5682 100644 --- a/doc/user/project/merge_requests/authorization_for_merge_requests.md +++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md @@ -9,7 +9,7 @@ There are two main ways to have a merge request flow with GitLab: With the protected branch flow everybody works within the same GitLab project. -The project maintainers get Master access and the regular developers get +The project maintainers get Maintainer access and the regular developers get Developer access. The maintainers mark the authoritative branches as 'Protected'. @@ -18,7 +18,7 @@ The developers push feature branches to the project and create merge requests to have their feature branches reviewed and merged into one of the protected branches. -By default, only users with Master access can merge changes into a protected +By default, only users with Maintainer access can merge changes into a protected branch. **Advantages** @@ -32,7 +32,7 @@ branch. ## Forking workflow -With the forking workflow the maintainers get Master access and the regular +With the forking workflow the maintainers get Maintainer access and the regular developers get Reporter access to the authoritative repository, which prohibits them from pushing any changes to it. diff --git a/doc/user/project/merge_requests/img/allow_collaboration.png b/doc/user/project/merge_requests/img/allow_collaboration.png Binary files differnew file mode 100644 index 00000000000..75596e7d9ad --- /dev/null +++ b/doc/user/project/merge_requests/img/allow_collaboration.png diff --git a/doc/user/project/merge_requests/img/allow_maintainer_push.png b/doc/user/project/merge_requests/img/allow_maintainer_push.png Binary files differdeleted file mode 100644 index 91cc399f4ff..00000000000 --- a/doc/user/project/merge_requests/img/allow_maintainer_push.png +++ /dev/null diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index b75bcacc9d7..5e2e0c3d171 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -28,7 +28,7 @@ With GitLab merge requests, you can: - Enable [fast-forward merge requests](#fast-forward-merge-requests) - Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch - [Create new merge requests by email](#create-new-merge-requests-by-email) -- Allow maintainers of the target project to push directly to the fork by [allowing edits from maintainers](maintainer_access.md) +- [Allow collaboration](allow_collaboration.md) so members of the target project can push directly to the fork - [Squash and merge](squash_and_merge.md) for a cleaner commit history With **[GitLab Enterprise Edition][ee]**, you can also: @@ -85,7 +85,7 @@ request is merged. This option is also visible in an existing merge request next to the merge request button and can be selected/deselected before merging. It's only visible -to users with [Master permissions](../../permissions.md) in the source project. +to users with [Maintainer permissions](../../permissions.md) in the source project. If the user viewing the merge request does not have the correct permissions to remove the source branch and the source branch is set for removal, the merge diff --git a/doc/user/project/merge_requests/maintainer_access.md b/doc/user/project/merge_requests/maintainer_access.md index 89f71e16a50..d59afecd375 100644 --- a/doc/user/project/merge_requests/maintainer_access.md +++ b/doc/user/project/merge_requests/maintainer_access.md @@ -1,20 +1 @@ -# Allow maintainer pushes for merge requests across forks - -> [Introduced][ce-17395] in GitLab 10.6. - -This feature is available for merge requests across forked projects that are -publicly accessible. It makes it easier for maintainers of projects to -collaborate on merge requests across forks. - -When enabled for a merge request, members with merge access to the target -branch of the project will be granted write permissions to the source branch -of the merge request. - -The feature can only be enabled by users who already have push access to the -source project, and only lasts while the merge request is open. - -Enable this functionality while creating a merge request: - - - -[ce-17395]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17395 +This document was moved to [another location](allow_collaboration.md). diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 862e4a3b466..205f0283107 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -42,7 +42,7 @@ to secure them. Your files live in a project [repository](../repository/index.md) on GitLab. [GitLab CI](../../../ci/README.md) picks up those files and makes them available at, typically, -`http://<username>.gitlab.io/<projectname>`. Please read through the docs on +`https://<username>.gitlab.io/<projectname>`. Please read through the docs on [GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain) for more info. ## Explore GitLab Pages diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 0cbb0c878c2..3bf63a22963 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -10,8 +10,8 @@ created protected branches. By default, a protected branch does four simple things: - it prevents its creation, if not already created, from everybody except users - with Master permission -- it prevents pushes from everybody except users with Master permission + with Maintainer permission +- it prevents pushes from everybody except users with Maintainer permission - it prevents **anyone** from force pushing to the branch - it prevents **anyone** from deleting the branch @@ -24,7 +24,7 @@ See the [Changelog](#changelog) section for changes over time. ## Configuring protected branches -To protect a branch, you need to have at least Master permission level. Note +To protect a branch, you need to have at least Maintainer permission level. Note that the `master` branch is protected by default. 1. Navigate to your project's **Settings âž” Repository** @@ -45,19 +45,19 @@ that the `master` branch is protected by default. Since GitLab 8.11, we added another layer of branch protection which provides more granular management of protected branches. The "Developers can push" option was replaced by an "Allowed to push" setting which can be set to -allow/prohibit Masters and/or Developers to push to a protected branch. +allow/prohibit Maintainers and/or Developers to push to a protected branch. Using the "Allowed to push" and "Allowed to merge" settings, you can control the actions that different roles can perform with the protected branch. For example, you could set "Allowed to push" to "No one", and "Allowed to merge" -to "Developers + Masters", to require _everyone_ to submit a merge request for +to "Developers + Maintainers", to require _everyone_ to submit a merge request for changes going into the protected branch. This is compatible with workflows like the [GitLab workflow](../../workflow/gitlab_flow.md). However, there are workflows where that is not needed, and only protecting from force pushes and branch removal is useful. For those workflows, you can allow everyone with write access to push to a protected branch by setting -"Allowed to push" to "Developers + Masters". +"Allowed to push" to "Developers + Maintainers". You can set the "Allowed to push" and "Allowed to merge" options while creating a protected branch or afterwards by selecting the option you want from the @@ -66,7 +66,7 @@ dropdown list in the "Already protected" area.  If you don't choose any of those options while creating a protected branch, -they are set to "Masters" by default. +they are set to "Maintainers" by default. ## Wildcard protected branches @@ -101,7 +101,7 @@ all matching branches: From time to time, it may be required to delete or clean up branches that are protected. -User with [Master permissions][perm] and up can manually delete protected +User with [Maintainer permissions][perm] and up can manually delete protected branches via GitLab's web interface: 1. Visit **Repository > Branches** diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 0cb7aefdb2f..a5eaf2e9835 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -8,12 +8,12 @@ This feature evolved out of [Protected Branches](protected_branches.md) ## Overview -Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Master permission will be prevented from creating tags. +Protected tags will prevent anyone from updating or deleting the tag, as and will prevent creation of matching tags based on the permissions you have selected. By default, anyone without Maintainer permission will be prevented from creating tags. ## Configuring protected tags -To protect a tag, you need to have at least Master permission level. +To protect a tag, you need to have at least Maintainer permission level. 1. Navigate to the project's Settings -> Repository page diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index b8bac01959e..9034a9b5179 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -19,7 +19,7 @@ > - The exports are stored in a temporary [shared directory][tmp] and are deleted > every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has -> master or admin access to the group where the exported project lives. An admin +> maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index c9d2f8dc32d..212e271ce6f 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -1,7 +1,7 @@ # Project settings NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to access a project +Only project Maintainers and Admin users have the [permissions] to access a project settings. You can adjust your [project](../index.md) settings by navigating @@ -74,7 +74,7 @@ To archive a project: #### Renaming a repository NOTE: **Note:** -Only project Masters and Admin users have the [permissions] to rename a +Only project Maintainers and Admin users have the [permissions] to rename a repository. Not to be confused with a project's name where it can also be changed from the [general project settings](#general-project-settings). @@ -98,7 +98,7 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Master** [permissions] to that group +1. you have at least **Maintainer** [permissions] to that group 1. you are an **Owner** of the project. Similarly, if you are an owner of a group, you can transfer any of its projects diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 23b67310d25..a7313082fac 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -131,7 +131,7 @@ There is room for more feedback and after the assigned person feels comfortable If the assigned person does not feel comfortable they can close the merge request without merging. In GitLab it is common to protect the long-lived branches (e.g. the master branch) so that normal developers [can't modify these protected branches](http://docs.gitlab.com/ce/permissions/permissions.html). -So if you want to merge it into a protected branch you assign it to someone with master authorizations. +So if you want to merge it into a protected branch you assign it to someone with maintainer authorizations. ## Issue tracking with GitLab flow diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 0d592a6d43e..ae161e43233 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -144,7 +144,7 @@ git lfs unlock --id=123 ``` If for some reason you need to unlock a file that was not locked by you, -you can use the `--force` flag as long as you have a `master` access on +you can use the `--force` flag as long as you have a `maintainer` access on the project: ```bash diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md index dbe63144e38..aaddbe4fbf5 100644 --- a/doc/workflow/repository_mirroring.md +++ b/doc/workflow/repository_mirroring.md @@ -1,6 +1,6 @@ # Repository mirroring -Repository Mirroring is a way to mirror repositories from external sources. +Repository mirroring is a way to mirror repositories from external sources. It can be used to mirror all branches, tags, and commits that you have in your repository. @@ -34,13 +34,200 @@ A few things/limitations to consider: - The Git LFS objects will not be synced. You'll need to push/pull them manually. -## Use-case +## Use cases +- You migrated to GitLab but still need to keep your project in another source. + In that case, you can simply set it up to mirror to GitLab (pull) and all the + essential history of commits, tags and branches will be available in your + GitLab instance. - You have old projects in another source that you don't use actively anymore, but don't want to remove for archiving purposes. In that case, you can create a push mirror so that your active GitLab repository can push its changes to the old location. +## Pulling from a remote repository **[STARTER]** + +>[Introduced][ee-51] in GitLab Enterprise Edition 8.2. + +You can set up a repository to automatically have its branches, tags, and commits +updated from an upstream repository. This is useful when a repository you're +interested in is located on a different server, and you want to be able to +browse its content and its activity using the familiar GitLab interface. + +When creating a new project, you can enable repository mirroring when you choose +to import the repository from "Any repo by URL". Enter the full URL of the Git +repository to pull from and click on the **Mirror repository** checkbox. + + + +For an existing project, you can set up mirror pulling by visiting your project's +**Settings âž” Repository** and searching for the "Pull from a remote repository" +section. Check the "Mirror repository" box and hit **Save changes** at the bottom. +You have a few options to choose from one being the user who will be the author +of all events in the activity feed that are the result of an update. This user +needs to have at least [master access][perms] to the project. Another option is +whether you want to trigger builds for mirror updates. + + + +Since the repository on GitLab functions as a mirror of the upstream repository, +you are advised not to push commits directly to the repository on GitLab. +Instead, any commits should be pushed to the upstream repository, and will end +up in the GitLab repository automatically within a certain period of time +or when a [forced update](#forcing-an-update) is initiated. + +If you do manually update a branch in the GitLab repository, the branch will +become diverged from upstream, and GitLab will no longer automatically update +this branch to prevent any changes from being lost. + + + +### Trigger update using API **[STARTER]** + +>[Introduced][ee-3453] in GitLab Enterprise Edition 10.3. + +Pull mirroring uses polling to detect new branches and commits added upstream, +often many minutes afterwards. If you notify GitLab by [API][pull-api], updates +will be pulled immediately. + +Read the [Pull Mirror Trigger API docs][pull-api]. + +### Pull only protected branches **[STARTER]** + +>[Introduced][ee-3326] in GitLab Enterprise Edition 10.3. + +You can choose to only pull the protected branches from your remote repository to GitLab. + +To use this option go to your project's repository settings page under pull mirror. + +### Overwrite diverged branches **[STARTER]** + +>[Introduced][ee-4559] in GitLab Enterprise Edition 10.6. + +You can choose to always update your local branch with the remote version even +if your local version has diverged from the remote. + +To use this option go to your project's repository settings page under pull mirror. + +### Hard failure **[STARTER]** + +>[Introduced][ee-3117] in GitLab Enterprise Edition 10.2. + +Once a mirror gets retried 14 times in a row, it will get marked as hard failed, +this will become visible in either the project main dashboard or in the +pull mirror settings page. + + + + + +When a project is hard failed, it will no longer get picked up for mirroring. +A user can resume the project mirroring again by either [forcing an update](#forcing-an-update) +or by changing the import URL in repository settings. + +### SSH authentication **[STARTER]** + +> [Introduced][ee-2551] in GitLab Starter 9.5 + +If you're mirroring over SSH (i.e., an `ssh://` URL), you can authenticate using +password-based authentication, just as over HTTPS, but you can also use public +key authentication. This is often more secure than password authentication, +especially when the source repository supports [Deploy Keys][deploy-key]. + +To get started, navigate to **Settings âž” Repository âž” Pull from a remote repository**, +enable mirroring (if not already enabled) and enter an `ssh://` URL. + +> **NOTE**: SCP-style URLs, e.g., `git@example.com:group/project.git`, are not +supported at this time. + +Entering the URL adds two features to the page - `Fingerprints` and +`SSH public key authentication`: + + + +SSH authentication is mutual. You have to prove to the server that you're +allowed to access the repository, but the server also has to prove to *you* that +it's who it claims to be. You provide your credentials as a password or public +key. The server that the source repository resides on provides its credentials +as a "host key", the fingerprint of which needs to be verified manually. + +Press the `Detect host keys` button. GitLab will fetch the host keys from the +server, and display the fingerprints to you: + + + +You now need to verify that the fingerprints are those you expect. GitLab.com +and other code hosting sites publish their fingerprints in the open for you +to check: + +* [AWS CodeCommit](http://docs.aws.amazon.com/codecommit/latest/userguide/regions.html#regions-fingerprints) +* [Bitbucket](https://confluence.atlassian.com/bitbucket/use-the-ssh-protocol-with-bitbucket-cloud-221449711.html#UsetheSSHprotocolwithBitbucketCloud-KnownhostorBitbucket%27spublickeyfingerprints) +* [GitHub](https://help.github.com/articles/github-s-ssh-key-fingerprints/) +* [GitLab.com](https://about.gitlab.com/gitlab-com/settings/#ssh-host-keys-fingerprints) +* [Launchpad](https://help.launchpad.net/SSHFingerprints) +* [Savannah](http://savannah.gnu.org/maintenance/SshAccess/) +* [SourceForge](https://sourceforge.net/p/forge/documentation/SSH%20Key%20Fingerprints/) + +Other providers will vary. If you're running on-premises GitLab, or otherwise +have access to the source server, you can securely gather the key fingerprints: + +``` +$ cat /etc/ssh/ssh_host*pub | ssh-keygen -E md5 -l -f - +256 MD5:f4:28:9f:23:99:15:21:1b:bf:ed:1f:8e:a0:76:b2:9d root@example.com (ECDSA) +256 MD5:e6:eb:45:8a:3c:59:35:5f:e9:5b:80:12:be:7e:22:73 root@example.com (ED25519) +2048 MD5:3f:72:be:3d:62:03:5c:62:83:e8:6e:14:34:3a:85:1d root@example.com (RSA) +``` + +(You may need to exclude `-E md5` for some older versions of SSH). + +If you're an SSH expert and already have a `known_hosts` file you'd like to use +unaltered, then you can skip these steps. Just press the "Show advanced" button +and paste in the file contents: + + + +Once you've **carefully verified** that all the fingerprints match your trusted +source, you can press `Save changes`. This will record the host keys, along with +the person who verified them (you!) and the date: + + + +When pulling changes from the source repository, GitLab will now check that at +least one of the stored host keys matches before connecting. This can prevent +malicious code from being injected into your mirror, or your password being +stolen! + +To use SSH public key authentication, you'll also need to choose that option +from the authentication methods dropdown. GitLab will generate a 4096-bit RSA +key and display the public component of that key to you: + + + +You then need to add the public SSH key to the source repository configuration. +If the source is hosted on GitLab, you should add it as a [Deploy Key][deploy-key]. +Other sources may require you to add the key to your user's `authorized_keys` +file - just paste the entire `ssh-rsa AAA.... user@host` block into the file on +its own line and save it. + +Once the public key is set up on the source repository, press `Save changes` and your +mirror will begin working. + +If you need to change the key at any time, you can press the `Regenerate key` +button to do so. You'll have to update the source repository with the new key +to keep the mirror running. + +### How it works + +Once you activate the pull mirroring feature, the mirror will be inserted into +a queue. A scheduler will start every minute and schedule a fixed amount of +mirrors for update, based on the configured maximum capacity. + +If the mirror successfully updates it will be enqueued once again with a small +backoff period. + +If the mirror fails (eg: branch diverged from upstream), the project's backoff +period will be penalized each time it fails up to a maximum amount of time. + ## Pushing to a remote repository **[STARTER]** >[Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/249) in @@ -83,7 +270,7 @@ To use this option go to your project's repository settings page under push mirr To set up a mirror from GitLab to GitHub, you need to follow these steps: 1. Create a [GitHub personal access token](https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/) with the "public_repo" box checked: - +  1. Fill in the "Git repository URL" with the personal access token replacing the password `https://GitHubUsername:GitHubPersonalAccessToken@github.com/group/project.git`: @@ -94,7 +281,7 @@ To set up a mirror from GitLab to GitHub, you need to follow these steps: 1. And either wait or trigger the "Update Now" button:  - + ## Forcing an update While mirrors are scheduled to update automatically, you can always force an update @@ -105,7 +292,60 @@ by using the **Update now** button which is exposed in various places: - in the tags page - in the **Mirror repository** settings page +## Bidirectional mirroring + +CAUTION: **Warning:** +There is no bidirectional support without conflicts. If you +configure a repository to pull and push to a second remote, there is no +guarantee that it will update correctly on both remotes. If you configure +a repository for bidirectional mirroring, you should consider when conflicts +occur who and how they will be resolved. + +Rewriting any mirrored commit on either remote will cause conflicts and +mirroring to fail. This can be prevented by [only pulling protected branches]( +#pull-only-protected-branches) and [only pushing protected branches]( +#push-only-protected-branches). You should protect the branches you wish to +mirror on both remotes to prevent conflicts caused by rewriting history. + +Bidirectional mirroring also creates a race condition where commits to the same +branch in close proximity will cause conflicts. The race condition can be +mitigated by reducing the mirroring delay by using a Push event webhook to +trigger an immediate pull to GitLab. Push mirroring from GitLab is rate limited +to once per minute when only push mirroring protected branches. + +It may be possible to implement a locking mechanism using the server-side +`pre-receive` hook to prevent the race condition. Read about [configuring +custom Git hooks][hooks] on the GitLab server. + +### Mirroring with Perforce via GitFusion + +CAUTION: **Warning:** +Bidirectional mirroring should not be used as a permanent +configuration. There is no bidirectional mirroring without conflicts. +Refer to [Migrating from Perforce Helix][perforce] for alternative migration +approaches. + +GitFusion provides a Git interface to Perforce which can be used by GitLab to +bidirectionally mirror projects with GitLab. This may be useful in some +situations when migrating from Perforce to GitLab where overlapping Perforce +workspaces cannot be migrated simultaneously to GitLab. + +If using mirroring with Perforce you should only mirror protected branches. +Perforce will reject any pushes that rewrite history. It is recommended that +only the fewest number of branches are mirrored due to the performance +limitations of GitFusion. + +[ee-51]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51 +[ee-2551]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2551 +[ee-3117]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3117 +[ee-3326]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3326 [ee-3350]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3350 +[ee-3453]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3453 +[ee-4559]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4559 [ce-18715]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18715 [perms]: ../user/permissions.md - +[hooks]: ../administration/custom_hooks.md +[deploy-key]: ../ssh/README.md#deploy-keys +[webhook]: ../user/project/integrations/webhooks.md#push-events +[pull-api]: ../api/projects.md#start-the-pull-mirroring-process-for-a-project +[perforce]: ../user/project/import/perforce.md diff --git a/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png Binary files differnew file mode 100644 index 00000000000..333648942f8 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_detect_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png Binary files differnew file mode 100644 index 00000000000..45c9bce0889 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_diverged_branch.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png Binary files differnew file mode 100644 index 00000000000..99d429a1802 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_main.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png Binary files differnew file mode 100644 index 00000000000..0ab07afa3cc --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_hard_failed_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_new_project.png b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png Binary files differnew file mode 100644 index 00000000000..43bf304838f --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_new_project.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png Binary files differnew file mode 100644 index 00000000000..5da5a7436bb --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_advanced_host_keys.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png Binary files differnew file mode 100644 index 00000000000..4b9085302a1 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png Binary files differnew file mode 100644 index 00000000000..8c2efdafa43 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_pull_settings_for_ssh.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png Binary files differnew file mode 100644 index 00000000000..93f3a532a0e --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_host_keys_verified.png diff --git a/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png Binary files differnew file mode 100644 index 00000000000..6997ad511d9 --- /dev/null +++ b/doc/workflow/repository_mirroring/repository_mirroring_ssh_public_key_authentication.png diff --git a/doc_styleguide.md b/doc_styleguide.md deleted file mode 100644 index 05ff46323ac..00000000000 --- a/doc_styleguide.md +++ /dev/null @@ -1,3 +0,0 @@ -# Documentation styleguide - -Moved to [development/doc_styleguide](doc/development/doc_styleguide.md). diff --git a/lib/api/api.rb b/lib/api/api.rb index 7ea575a9661..e2ad3c5f4e3 100644 --- a/lib/api/api.rb +++ b/lib/api/api.rb @@ -83,6 +83,7 @@ module API # Keep in alphabetical order mount ::API::AccessRequests mount ::API::Applications + mount ::API::Avatar mount ::API::AwardEmoji mount ::API::Badges mount ::API::Boards diff --git a/lib/api/avatar.rb b/lib/api/avatar.rb new file mode 100644 index 00000000000..70219bc8ea0 --- /dev/null +++ b/lib/api/avatar.rb @@ -0,0 +1,21 @@ +module API + class Avatar < Grape::API + resource :avatar do + desc 'Return avatar url for a user' do + success Entities::Avatar + end + params do + requires :email, type: String, desc: 'Public email address of the user' + optional :size, type: Integer, desc: 'Single pixel dimension for Gravatar images' + end + get do + forbidden!('Unauthorized access') unless can?(current_user, :read_users_list) + + user = User.find_by_public_email(params[:email]) + user ||= User.new(email: params[:email]) + + present user, with: Entities::Avatar, size: params[:size] + end + end + end +end diff --git a/lib/api/entities.rb b/lib/api/entities.rb index c4537036a3a..22afcb9edf2 100644 --- a/lib/api/entities.rb +++ b/lib/api/entities.rb @@ -559,7 +559,9 @@ module API expose :discussion_locked expose :should_remove_source_branch?, as: :should_remove_source_branch expose :force_remove_source_branch?, as: :force_remove_source_branch - expose :allow_maintainer_to_push, if: -> (merge_request, _) { merge_request.for_fork? } + expose :allow_collaboration, if: -> (merge_request, _) { merge_request.for_fork? } + # Deprecated + expose :allow_collaboration, as: :allow_maintainer_to_push, if: -> (merge_request, _) { merge_request.for_fork? } expose :web_url do |merge_request, options| Gitlab::UrlBuilder.build(merge_request) @@ -692,6 +694,12 @@ module API expose :notes, using: Entities::Note end + class Avatar < Grape::Entity + expose :avatar_url do |avatarable, options| + avatarable.avatar_url(only_path: false, size: options[:size]) + end + end + class AwardEmoji < Grape::Entity expose :id expose :name diff --git a/lib/api/events.rb b/lib/api/events.rb index b0713ff1d54..fc4ba5a3188 100644 --- a/lib/api/events.rb +++ b/lib/api/events.rb @@ -17,6 +17,7 @@ module API def present_events(events) events = events.reorder(created_at: params[:sort]) + .with_associations present paginate(events), with: Entities::Event end diff --git a/lib/api/issues.rb b/lib/api/issues.rb index b64f465ce56..25185d6edc8 100644 --- a/lib/api/issues.rb +++ b/lib/api/issues.rb @@ -16,7 +16,7 @@ module API args[:scope] = args[:scope].underscore if args[:scope] issues = IssuesFinder.new(current_user, args).execute - .preload(:assignees, :labels, :notes, :timelogs, :project) + .preload(:assignees, :labels, :notes, :timelogs, :project, :author) issues.reorder(args[:order_by] => args[:sort]) end diff --git a/lib/api/jobs.rb b/lib/api/jobs.rb index 54d1acbd412..e95b0dd5267 100644 --- a/lib/api/jobs.rb +++ b/lib/api/jobs.rb @@ -54,6 +54,7 @@ module API pipeline = user_project.pipelines.find(params[:pipeline_id]) builds = pipeline.builds builds = filter_builds(builds, params[:scope]) + builds = builds.preload(:job_artifacts_archive) present paginate(builds), with: Entities::Job end diff --git a/lib/api/merge_requests.rb b/lib/api/merge_requests.rb index 278d53427f0..af7d2471b34 100644 --- a/lib/api/merge_requests.rb +++ b/lib/api/merge_requests.rb @@ -162,7 +162,8 @@ module API optional :milestone_id, type: Integer, desc: 'The ID of a milestone to assign the merge request' optional :labels, type: String, desc: 'Comma-separated list of label names' optional :remove_source_branch, type: Boolean, desc: 'Remove source branch when merging' - optional :allow_maintainer_to_push, type: Boolean, desc: 'Whether a maintainer of the target project can push to the source project' + optional :allow_collaboration, type: Boolean, desc: 'Allow commits from members who can merge to the target branch' + optional :allow_maintainer_to_push, type: Boolean, as: :allow_collaboration, desc: '[deprecated] See allow_collaboration' optional :squash, type: Grape::API::Boolean, desc: 'When true, the commits will be squashed into a single commit on merge' use :optional_params_ee diff --git a/lib/api/pipelines.rb b/lib/api/pipelines.rb index 735591fedd5..8374a57edfa 100644 --- a/lib/api/pipelines.rb +++ b/lib/api/pipelines.rb @@ -41,15 +41,20 @@ module API end params do requires :ref, type: String, desc: 'Reference' + optional :variables, Array, desc: 'Array of variables available in the pipeline' end post ':id/pipeline' do Gitlab::QueryLimiting.whitelist('https://gitlab.com/gitlab-org/gitlab-ce/issues/42124') authorize! :create_pipeline, user_project + pipeline_params = declared_params(include_missing: false) + .merge(variables_attributes: params[:variables]) + .except(:variables) + new_pipeline = Ci::CreatePipelineService.new(user_project, current_user, - declared_params(include_missing: false)) + pipeline_params) .execute(:api, ignore_skip_ci: true, save_on_errors: false) if new_pipeline.persisted? diff --git a/lib/api/protected_branches.rb b/lib/api/protected_branches.rb index aa7cab4a741..a30eb46c220 100644 --- a/lib/api/protected_branches.rb +++ b/lib/api/protected_branches.rb @@ -41,10 +41,10 @@ module API requires :name, type: String, desc: 'The name of the protected branch' optional :push_access_level, type: Integer, values: ProtectedRefAccess::ALLOWED_ACCESS_LEVELS, - desc: 'Access levels allowed to push (defaults: `40`, master access level)' + desc: 'Access levels allowed to push (defaults: `40`, maintainer access level)' optional :merge_access_level, type: Integer, values: ProtectedRefAccess::ALLOWED_ACCESS_LEVELS, - desc: 'Access levels allowed to merge (defaults: `40`, master access level)' + desc: 'Access levels allowed to merge (defaults: `40`, maintainer access level)' end post ':id/protected_branches' do protected_branch = user_project.protected_branches.find_by(name: params[:name]) diff --git a/lib/api/runner.rb b/lib/api/runner.rb index e9886c76870..db502697a19 100644 --- a/lib/api/runner.rb +++ b/lib/api/runner.rb @@ -205,7 +205,7 @@ module API status 200 content_type Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE - JobArtifactUploader.workhorse_authorize + JobArtifactUploader.workhorse_authorize(has_length: false, maximum_size: max_artifacts_size) end desc 'Upload artifacts for job' do diff --git a/lib/backup.rb b/lib/backup.rb new file mode 100644 index 00000000000..e2c62af23ae --- /dev/null +++ b/lib/backup.rb @@ -0,0 +1,3 @@ +module Backup + Error = Class.new(StandardError) +end diff --git a/lib/backup/database.rb b/lib/backup/database.rb index 1608f7ad02d..086ca5986bd 100644 --- a/lib/backup/database.rb +++ b/lib/backup/database.rb @@ -44,7 +44,7 @@ module Backup end report_success(success) - abort 'Backup failed' unless success + raise Backup::Error, 'Backup failed' unless success end def restore @@ -72,7 +72,7 @@ module Backup end report_success(success) - abort 'Restore failed' unless success + abort Backup::Error, 'Restore failed' unless success end protected diff --git a/lib/backup/files.rb b/lib/backup/files.rb index 9895db9e451..d769a3ee7b0 100644 --- a/lib/backup/files.rb +++ b/lib/backup/files.rb @@ -26,7 +26,7 @@ module Backup unless status.zero? puts output - abort 'Backup failed' + raise Backup::Error, 'Backup failed' end run_pipeline!([%W(tar --exclude=lost+found -C #{@backup_files_dir} -cf - .), %w(gzip -c -1)], out: [backup_tarball, 'w', 0600]) @@ -39,7 +39,11 @@ module Backup def restore backup_existing_files_dir - run_pipeline!([%w(gzip -cd), %W(tar --unlink-first --recursive-unlink -C #{app_files_dir} -xf -)], in: backup_tarball) + run_pipeline!([%w(gzip -cd), %W(#{tar} --unlink-first --recursive-unlink -C #{app_files_dir} -xf -)], in: backup_tarball) + end + + def tar + system(*%w[gtar --version], out: '/dev/null') ? 'gtar' : 'tar' end def backup_existing_files_dir @@ -61,7 +65,7 @@ module Backup def run_pipeline!(cmd_list, options = {}) status_list = Open3.pipeline(*cmd_list, options) - abort 'Backup failed' unless status_list.compact.all?(&:success?) + raise Backup::Error, 'Backup failed' unless status_list.compact.all?(&:success?) end end end diff --git a/lib/backup/manager.rb b/lib/backup/manager.rb index a8da0c7edef..a3641505196 100644 --- a/lib/backup/manager.rb +++ b/lib/backup/manager.rb @@ -27,7 +27,7 @@ module Backup progress.puts "done".color(:green) else puts "creating archive #{tar_file} failed".color(:red) - abort 'Backup failed' + raise Backup::Error, 'Backup failed' end upload @@ -52,7 +52,7 @@ module Backup progress.puts "done".color(:green) else puts "uploading backup to #{remote_directory} failed".color(:red) - abort 'Backup failed' + raise Backup::Error, 'Backup failed' end end @@ -66,7 +66,7 @@ module Backup progress.puts "done".color(:green) else puts "deleting tmp directory '#{dir}' failed".color(:red) - abort 'Backup failed' + raise Backup::Error, 'Backup failed' end end end diff --git a/lib/backup/repository.rb b/lib/backup/repository.rb index 84670d6582e..1b1c83d9fb3 100644 --- a/lib/backup/repository.rb +++ b/lib/backup/repository.rb @@ -17,7 +17,10 @@ module Backup Project.find_each(batch_size: 1000) do |project| progress.print " * #{display_repo_path(project)} ... " - path_to_project_repo = path_to_repo(project) + + path_to_project_repo = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + path_to_repo(project) + end path_to_project_bundle = path_to_bundle(project) # Create namespace dir or hashed path if missing @@ -51,7 +54,9 @@ module Backup end wiki = ProjectWiki.new(project) - path_to_wiki_repo = path_to_repo(wiki) + path_to_wiki_repo = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + path_to_repo(wiki) + end path_to_wiki_bundle = path_to_bundle(wiki) if File.exist?(path_to_wiki_repo) @@ -111,7 +116,9 @@ module Backup # TODO: Need to find a way to do this for gitaly # Gitaly migration issue: https://gitlab.com/gitlab-org/gitaly/issues/1195 in_path(path_to_tars(project)) do |dir| - path_to_project_repo = path_to_repo(project) + path_to_project_repo = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + path_to_repo(project) + end cmd = %W(tar -xf #{path_to_tars(project, dir)} -C #{path_to_project_repo} #{dir}) output, status = Gitlab::Popen.popen(cmd) diff --git a/lib/constraints/feature_constrainer.rb b/lib/constraints/feature_constrainer.rb new file mode 100644 index 00000000000..05d48b0f25a --- /dev/null +++ b/lib/constraints/feature_constrainer.rb @@ -0,0 +1,13 @@ +module Constraints + class FeatureConstrainer + attr_reader :feature + + def initialize(feature) + @feature = feature + end + + def matches?(_request) + Feature.enabled?(feature) + end + end +end diff --git a/lib/gitlab/access.rb b/lib/gitlab/access.rb index 7127948cf00..87e377de4d3 100644 --- a/lib/gitlab/access.rb +++ b/lib/gitlab/access.rb @@ -29,10 +29,10 @@ module Gitlab def options { - "Guest" => GUEST, - "Reporter" => REPORTER, - "Developer" => DEVELOPER, - "Master" => MASTER + "Guest" => GUEST, + "Reporter" => REPORTER, + "Developer" => DEVELOPER, + "Maintainer" => MASTER } end @@ -57,10 +57,10 @@ module Gitlab def protection_options { - "Not protected: Both developers and masters can push new commits, force push, or delete the branch." => PROTECTION_NONE, - "Protected against pushes: Developers cannot push new commits, but are allowed to accept merge requests to the branch. Masters can push to the branch." => PROTECTION_DEV_CAN_MERGE, - "Partially protected: Both developers and masters can push new commits, but cannot force push or delete the branch." => PROTECTION_DEV_CAN_PUSH, - "Fully protected: Developers cannot push new commits, but masters can. No-one can force push or delete the branch." => PROTECTION_FULL + "Not protected: Both developers and maintainers can push new commits, force push, or delete the branch." => PROTECTION_NONE, + "Protected against pushes: Developers cannot push new commits, but are allowed to accept merge requests to the branch. Maintainers can push to the branch." => PROTECTION_DEV_CAN_MERGE, + "Partially protected: Both developers and maintainers can push new commits, but cannot force push or delete the branch." => PROTECTION_DEV_CAN_PUSH, + "Fully protected: Developers cannot push new commits, but maintainers can. No-one can force push or delete the branch." => PROTECTION_FULL } end diff --git a/lib/gitlab/auth.rb b/lib/gitlab/auth.rb index 0f7a7b0ce8d..7de66539848 100644 --- a/lib/gitlab/auth.rb +++ b/lib/gitlab/auth.rb @@ -240,7 +240,7 @@ module Gitlab return unless login == 'gitlab-ci-token' return unless password - build = ::Ci::Build.running.find_by_token(password) + build = find_build_by_token(password) return unless build return unless build.project.builds_enabled? @@ -301,6 +301,12 @@ module Gitlab REGISTRY_SCOPES end + + private + + def find_build_by_token(token) + ::Ci::Build.running.find_by_token(token) + end end end end diff --git a/lib/gitlab/background_migration/archive_legacy_traces.rb b/lib/gitlab/background_migration/archive_legacy_traces.rb new file mode 100644 index 00000000000..5a4e5b2c471 --- /dev/null +++ b/lib/gitlab/background_migration/archive_legacy_traces.rb @@ -0,0 +1,24 @@ +# frozen_string_literal: true +# rubocop:disable Metrics/AbcSize +# rubocop:disable Style/Documentation + +module Gitlab + module BackgroundMigration + class ArchiveLegacyTraces + def perform(start_id, stop_id) + # This background migration directly refers to ::Ci::Build model which is defined in application code. + # In general, migration code should be isolated as much as possible in order to be idempotent. + # However, `archive!` method is too complicated to be replicated by coping its subsequent code. + # So we chose a way to use ::Ci::Build directly and we don't change the `archive!` method until 11.1 + ::Ci::Build.finished.without_archived_trace + .where(id: start_id..stop_id).find_each do |build| + begin + build.trace.archive! + rescue => e + Rails.logger.error "Failed to archive live trace. id: #{build.id} message: #{e.message}" + end + end + end + end + end +end diff --git a/lib/gitlab/checks/change_access.rb b/lib/gitlab/checks/change_access.rb index 51ba09aa129..f76a6fb5f17 100644 --- a/lib/gitlab/checks/change_access.rb +++ b/lib/gitlab/checks/change_access.rb @@ -5,7 +5,7 @@ module Gitlab push_code: 'You are not allowed to push code to this project.', delete_default_branch: 'The default branch of a project cannot be deleted.', force_push_protected_branch: 'You are not allowed to force push code to a protected branch on this project.', - non_master_delete_protected_branch: 'You are not allowed to delete protected branches from this project. Only a project master or owner can delete a protected branch.', + non_master_delete_protected_branch: 'You are not allowed to delete protected branches from this project. Only a project maintainer or owner can delete a protected branch.', non_web_delete_protected_branch: 'You can only delete protected branches using the web interface.', merge_protected_branch: 'You are not allowed to merge code into protected branches on this project.', push_protected_branch: 'You are not allowed to push code to protected branches on this project.', diff --git a/lib/gitlab/ci/pipeline/chain/populate.rb b/lib/gitlab/ci/pipeline/chain/populate.rb index 69b8a8fc68f..f34c11ca3c2 100644 --- a/lib/gitlab/ci/pipeline/chain/populate.rb +++ b/lib/gitlab/ci/pipeline/chain/populate.rb @@ -8,6 +8,9 @@ module Gitlab PopulateError = Class.new(StandardError) def perform! + # Allocate next IID. This operation must be outside of transactions of pipeline creations. + pipeline.ensure_project_iid! + ## # Populate pipeline with block argument of CreatePipelineService#execute. # diff --git a/lib/gitlab/ci/pipeline/preloader.rb b/lib/gitlab/ci/pipeline/preloader.rb index e7a2e5511cf..db0a1ea4dab 100644 --- a/lib/gitlab/ci/pipeline/preloader.rb +++ b/lib/gitlab/ci/pipeline/preloader.rb @@ -5,23 +5,47 @@ module Gitlab module Pipeline # Class for preloading data associated with pipelines such as commit # authors. - module Preloader - def self.preload(pipelines) - # This ensures that all the pipeline commits are eager loaded before we - # start using them. + class Preloader + def self.preload!(pipelines) + ## + # This preloads all commits at once, because `Ci::Pipeline#commit` is + # using a lazy batch loading, what results in only one batched Gitaly + # call. + # pipelines.each(&:commit) pipelines.each do |pipeline| - # This preloads the author of every commit. We're using "lazy_author" - # here since "author" immediately loads the data on the first call. - pipeline.commit.try(:lazy_author) - - # This preloads the number of warnings for every pipeline, ensuring - # that Ci::Pipeline#has_warnings? doesn't execute any additional - # queries. - pipeline.number_of_warnings + self.new(pipeline).tap do |preloader| + preloader.preload_commit_authors + preloader.preload_pipeline_warnings + preloader.preload_stages_warnings + end end end + + def initialize(pipeline) + @pipeline = pipeline + end + + def preload_commit_authors + # This also preloads the author of every commit. We're using "lazy_author" + # here since "author" immediately loads the data on the first call. + @pipeline.commit.try(:lazy_author) + end + + def preload_pipeline_warnings + # This preloads the number of warnings for every pipeline, ensuring + # that Ci::Pipeline#has_warnings? doesn't execute any additional + # queries. + @pipeline.number_of_warnings + end + + def preload_stages_warnings + # This preloads the number of warnings for every stage, ensuring + # that Ci::Stage#has_warnings? doesn't execute any additional + # queries. + @pipeline.stages.each { |stage| stage.number_of_warnings } + end end end end diff --git a/lib/gitlab/ci/status/stage/common.rb b/lib/gitlab/ci/status/stage/common.rb index bc99d925347..f60a7662075 100644 --- a/lib/gitlab/ci/status/stage/common.rb +++ b/lib/gitlab/ci/status/stage/common.rb @@ -8,7 +8,9 @@ module Gitlab end def details_path - project_pipeline_path(subject.project, subject.pipeline, anchor: subject.name) + project_pipeline_path(subject.pipeline.project, + subject.pipeline, + anchor: subject.name) end def has_action? diff --git a/lib/gitlab/ci/trace.rb b/lib/gitlab/ci/trace.rb index fe15fabc2e8..a52d71225bb 100644 --- a/lib/gitlab/ci/trace.rb +++ b/lib/gitlab/ci/trace.rb @@ -1,6 +1,10 @@ module Gitlab module Ci class Trace + include ExclusiveLeaseGuard + + LEASE_TIMEOUT = 1.hour + ArchiveError = Class.new(StandardError) attr_reader :job @@ -105,6 +109,14 @@ module Gitlab end def archive! + try_obtain_lease do + unsafe_archive! + end + end + + private + + def unsafe_archive! raise ArchiveError, 'Already archived' if trace_artifact raise ArchiveError, 'Job is not finished yet' unless job.complete? @@ -126,8 +138,6 @@ module Gitlab end end - private - def archive_stream!(stream) clone_file!(stream, JobArtifactUploader.workhorse_upload_path) do |clone_path| create_build_trace!(job, clone_path) @@ -206,6 +216,16 @@ module Gitlab def trace_artifact job.job_artifacts_trace end + + # For ExclusiveLeaseGuard concern + def lease_key + @lease_key ||= "trace:archive:#{job.id}" + end + + # For ExclusiveLeaseGuard concern + def lease_timeout + LEASE_TIMEOUT + end end end end diff --git a/lib/gitlab/current_settings.rb b/lib/gitlab/current_settings.rb index 6cf7aa1bf0d..3cf35f499cd 100644 --- a/lib/gitlab/current_settings.rb +++ b/lib/gitlab/current_settings.rb @@ -24,7 +24,22 @@ module Gitlab private def ensure_application_settings! + cached_application_settings || uncached_application_settings + end + + def cached_application_settings return in_memory_application_settings if ENV['IN_MEMORY_APPLICATION_SETTINGS'] == 'true' + + begin + ::ApplicationSetting.cached + rescue + # In case Redis isn't running + # or the Redis UNIX socket file is not available + # or the DB is not running (we use migrations in the cache key) + end + end + + def uncached_application_settings return fake_application_settings unless connect_to_db? current_settings = ::ApplicationSetting.current diff --git a/lib/gitlab/git/commit.rb b/lib/gitlab/git/commit.rb index 89f761dd515..c9806cdb85f 100644 --- a/lib/gitlab/git/commit.rb +++ b/lib/gitlab/git/commit.rb @@ -60,6 +60,9 @@ module Gitlab # Some weird thing? return nil unless commit_id.is_a?(String) + # This saves us an RPC round trip. + return nil if commit_id.include?(':') + commit = repo.gitaly_migrate(:find_commit) do |is_enabled| if is_enabled repo.gitaly_commit_client.find_commit(commit_id) diff --git a/lib/gitlab/git/repository.rb b/lib/gitlab/git/repository.rb index 4cbf20bfe76..93f9adaf1f1 100644 --- a/lib/gitlab/git/repository.rb +++ b/lib/gitlab/git/repository.rb @@ -109,7 +109,7 @@ module Gitlab end def ==(other) - path == other.path + [storage, relative_path] == [other.storage, other.relative_path] end def path @@ -395,12 +395,12 @@ module Gitlab nil end - def archive_metadata(ref, storage_path, format = "tar.gz", append_sha:) + def archive_metadata(ref, storage_path, project_path, format = "tar.gz", append_sha:) ref ||= root_ref commit = Gitlab::Git::Commit.find(self, ref) return {} if commit.nil? - prefix = archive_prefix(ref, commit.id, append_sha: append_sha) + prefix = archive_prefix(ref, commit.id, project_path, append_sha: append_sha) { 'ArchivePrefix' => prefix, @@ -412,16 +412,12 @@ module Gitlab # This is both the filename of the archive (missing the extension) and the # name of the top-level member of the archive under which all files go - # - # FIXME: The generated prefix is incorrect for projects with hashed - # storage enabled - def archive_prefix(ref, sha, append_sha:) + def archive_prefix(ref, sha, project_path, append_sha:) append_sha = (ref != sha) if append_sha.nil? - project_name = self.name.chomp('.git') formatted_ref = ref.tr('/', '-') - prefix_segments = [project_name, formatted_ref] + prefix_segments = [project_path, formatted_ref] prefix_segments << sha if append_sha prefix_segments.join('-') @@ -1397,6 +1393,11 @@ module Gitlab def write_config(full_path:) return unless full_path.present? + # This guard avoids Gitaly log/error spam + unless exists? + raise NoRepository, 'repository does not exist' + end + gitaly_migrate(:write_config) do |is_enabled| if is_enabled gitaly_repository_client.write_config(full_path: full_path) @@ -1542,7 +1543,7 @@ module Gitlab end end - def rev_list(including: [], excluding: [], objects: false, &block) + def rev_list(including: [], excluding: [], options: [], objects: false, &block) args = ['rev-list'] args.push(*rev_list_param(including)) @@ -1555,6 +1556,10 @@ module Gitlab args.push('--objects') if objects + if options.any? + args.push(*options) + end + run_git!(args, lazy_block: block) end diff --git a/lib/gitlab/git/rev_list.rb b/lib/gitlab/git/rev_list.rb index 38c3a55f96f..79544ccf13d 100644 --- a/lib/gitlab/git/rev_list.rb +++ b/lib/gitlab/git/rev_list.rb @@ -38,7 +38,10 @@ module Gitlab end def all_objects(require_path: nil, &lazy_block) - get_objects(including: :all, require_path: require_path, &lazy_block) + get_objects(including: :all, + options: ["--filter=blob:limit=#{Gitlab::Git::Blob::LFS_POINTER_MAX_SIZE}"], + require_path: require_path, + &lazy_block) end # This methods returns an array of missed references @@ -54,8 +57,8 @@ module Gitlab repository.rev_list(args).split("\n") end - def get_objects(including: [], excluding: [], require_path: nil) - opts = { including: including, excluding: excluding, objects: true } + def get_objects(including: [], excluding: [], options: [], require_path: nil) + opts = { including: including, excluding: excluding, options: options, objects: true } repository.rev_list(opts) do |lazy_output| objects = objects_from_output(lazy_output, require_path: require_path) diff --git a/lib/gitlab/gitaly_client.rb b/lib/gitlab/gitaly_client.rb index 550294916a4..36e9adf27da 100644 --- a/lib/gitlab/gitaly_client.rb +++ b/lib/gitlab/gitaly_client.rb @@ -191,6 +191,8 @@ module Gitlab metadata['call_site'] = feature.to_s if feature metadata['gitaly-servers'] = address_metadata(remote_storage) if remote_storage + metadata.merge!(server_feature_flags) + result = { metadata: metadata } # nil timeout indicates that we should use the default @@ -209,6 +211,14 @@ module Gitlab result end + SERVER_FEATURE_FLAGS = %w[gogit_findcommit].freeze + + def self.server_feature_flags + SERVER_FEATURE_FLAGS.map do |f| + ["gitaly-feature-#{f.tr('_', '-')}", feature_enabled?(f).to_s] + end.to_h + end + def self.token(storage) params = Gitlab.config.repositories.storages[storage] raise "storage not found: #{storage.inspect}" if params.nil? @@ -243,6 +253,10 @@ module Gitlab else false end + rescue => ex + # During application startup feature lookups in SQL can fail + Rails.logger.warn "exception while checking Gitaly feature status for #{feature_name}: #{ex}" + false end # opt_into_all_features? returns true when the current environment diff --git a/lib/gitlab/github_import/importer/lfs_object_importer.rb b/lib/gitlab/github_import/importer/lfs_object_importer.rb new file mode 100644 index 00000000000..a88c17aaf82 --- /dev/null +++ b/lib/gitlab/github_import/importer/lfs_object_importer.rb @@ -0,0 +1,24 @@ +# frozen_string_literal: true + +module Gitlab + module GithubImport + module Importer + class LfsObjectImporter + attr_reader :lfs_object, :project + + # lfs_object - An instance of `Gitlab::GithubImport::Representation::LfsObject`. + # project - An instance of `Project`. + def initialize(lfs_object, project, _) + @lfs_object = lfs_object + @project = project + end + + def execute + Projects::LfsPointers::LfsDownloadService + .new(project) + .execute(lfs_object.oid, lfs_object.download_link) + end + end + end + end +end diff --git a/lib/gitlab/github_import/importer/lfs_objects_importer.rb b/lib/gitlab/github_import/importer/lfs_objects_importer.rb new file mode 100644 index 00000000000..6046e30d4ef --- /dev/null +++ b/lib/gitlab/github_import/importer/lfs_objects_importer.rb @@ -0,0 +1,37 @@ +# frozen_string_literal: true + +module Gitlab + module GithubImport + module Importer + class LfsObjectsImporter + include ParallelScheduling + + def importer_class + LfsObjectImporter + end + + def representation_class + Representation::LfsObject + end + + def sidekiq_worker_class + ImportLfsObjectWorker + end + + def collection_method + :lfs_objects + end + + def each_object_to_import + lfs_objects = Projects::LfsPointers::LfsImportService.new(project).execute + + lfs_objects.each do |object| + yield object + end + rescue StandardError => e + Rails.logger.error("The Lfs import process failed. #{e.message}") + end + end + end + end +end diff --git a/lib/gitlab/github_import/representation/lfs_object.rb b/lib/gitlab/github_import/representation/lfs_object.rb new file mode 100644 index 00000000000..debe0fa0baf --- /dev/null +++ b/lib/gitlab/github_import/representation/lfs_object.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: true + +module Gitlab + module GithubImport + module Representation + class LfsObject + include ToHash + include ExposeAttribute + + attr_reader :attributes + + expose_attribute :oid, :download_link + + # Builds a lfs_object + def self.from_api_response(lfs_object) + new({ oid: lfs_object[0], download_link: lfs_object[1] }) + end + + # Builds a new lfs_object using a Hash that was built from a JSON payload. + def self.from_json_hash(raw_hash) + new(Representation.symbolize_hash(raw_hash)) + end + + # attributes - A Hash containing the raw lfs_object details. The keys of this + # Hash must be Symbols. + def initialize(attributes) + @attributes = attributes + end + end + end + end +end diff --git a/lib/gitlab/github_import/sequential_importer.rb b/lib/gitlab/github_import/sequential_importer.rb index 4f7324536a0..3cad919b4eb 100644 --- a/lib/gitlab/github_import/sequential_importer.rb +++ b/lib/gitlab/github_import/sequential_importer.rb @@ -19,7 +19,8 @@ module Gitlab Importer::PullRequestsImporter, Importer::IssuesImporter, Importer::DiffNotesImporter, - Importer::NotesImporter + Importer::NotesImporter, + Importer::LfsObjectsImporter ].freeze # project - The project to import the data into. diff --git a/lib/gitlab/gpg.rb b/lib/gitlab/gpg.rb index 413872d7e08..a4263369269 100644 --- a/lib/gitlab/gpg.rb +++ b/lib/gitlab/gpg.rb @@ -54,7 +54,11 @@ module Gitlab fingerprints = CurrentKeyChain.fingerprints_from_key(key) GPGME::Key.find(:public, fingerprints).flat_map do |raw_key| - raw_key.uids.map { |uid| { name: uid.name, email: uid.email.downcase } } + raw_key.uids.each_with_object([]) do |uid, arr| + name = uid.name.force_encoding('UTF-8') + email = uid.email.force_encoding('UTF-8') + arr << { name: name, email: email.downcase } if name.valid_encoding? && email.valid_encoding? + end end end end diff --git a/lib/gitlab/graphql.rb b/lib/gitlab/graphql.rb new file mode 100644 index 00000000000..04a89432230 --- /dev/null +++ b/lib/gitlab/graphql.rb @@ -0,0 +1,5 @@ +module Gitlab + module Graphql + StandardGraphqlError = Class.new(StandardError) + end +end diff --git a/lib/gitlab/graphql/authorize.rb b/lib/gitlab/graphql/authorize.rb new file mode 100644 index 00000000000..04f25c53e49 --- /dev/null +++ b/lib/gitlab/graphql/authorize.rb @@ -0,0 +1,21 @@ +module Gitlab + module Graphql + # Allow fields to declare permissions their objects must have. The field + # will be set to nil unless all required permissions are present. + module Authorize + extend ActiveSupport::Concern + + def self.use(schema_definition) + schema_definition.instrument(:field, Instrumentation.new) + end + + def required_permissions + @required_permissions ||= [] + end + + def authorize(*permissions) + required_permissions.concat(permissions) + end + end + end +end diff --git a/lib/gitlab/graphql/authorize/instrumentation.rb b/lib/gitlab/graphql/authorize/instrumentation.rb new file mode 100644 index 00000000000..6cb8e617f62 --- /dev/null +++ b/lib/gitlab/graphql/authorize/instrumentation.rb @@ -0,0 +1,45 @@ +module Gitlab + module Graphql + module Authorize + class Instrumentation + # Replace the resolver for the field with one that will only return the + # resolved object if the permissions check is successful. + # + # Collections are not supported. Apply permissions checks for those at the + # database level instead, to avoid loading superfluous data from the DB + def instrument(_type, field) + field_definition = field.metadata[:type_class] + return field unless field_definition.respond_to?(:required_permissions) + return field if field_definition.required_permissions.empty? + + old_resolver = field.resolve_proc + + new_resolver = -> (obj, args, ctx) do + resolved_obj = old_resolver.call(obj, args, ctx) + checker = build_checker(ctx[:current_user], field_definition.required_permissions) + + if resolved_obj.respond_to?(:then) + resolved_obj.then(&checker) + else + checker.call(resolved_obj) + end + end + + field.redefine do + resolve(new_resolver) + end + end + + private + + def build_checker(current_user, abilities) + proc do |obj| + # Load the elements if they weren't loaded by BatchLoader yet + obj = obj.sync if obj.respond_to?(:sync) + obj if abilities.all? { |ability| Ability.allowed?(current_user, ability, obj) } + end + end + end + end + end +end diff --git a/lib/gitlab/graphql/present.rb b/lib/gitlab/graphql/present.rb new file mode 100644 index 00000000000..2c7b64f1be9 --- /dev/null +++ b/lib/gitlab/graphql/present.rb @@ -0,0 +1,20 @@ +module Gitlab + module Graphql + module Present + extend ActiveSupport::Concern + prepended do + def self.present_using(kls) + @presenter_class = kls + end + + def self.presenter_class + @presenter_class + end + end + + def self.use(schema_definition) + schema_definition.instrument(:field, Instrumentation.new) + end + end + end +end diff --git a/lib/gitlab/graphql/present/instrumentation.rb b/lib/gitlab/graphql/present/instrumentation.rb new file mode 100644 index 00000000000..1688262974b --- /dev/null +++ b/lib/gitlab/graphql/present/instrumentation.rb @@ -0,0 +1,25 @@ +module Gitlab + module Graphql + module Present + class Instrumentation + def instrument(type, field) + presented_in = field.metadata[:type_class].owner + return field unless presented_in.respond_to?(:presenter_class) + return field unless presented_in.presenter_class + + old_resolver = field.resolve_proc + + resolve_with_presenter = -> (presented_type, args, context) do + object = presented_type.object + presenter = presented_in.presenter_class.new(object, **context.to_h) + old_resolver.call(presenter, args, context) + end + + field.redefine do + resolve(resolve_with_presenter) + end + end + end + end + end +end diff --git a/lib/gitlab/graphql/variables.rb b/lib/gitlab/graphql/variables.rb new file mode 100644 index 00000000000..ffbaf65b512 --- /dev/null +++ b/lib/gitlab/graphql/variables.rb @@ -0,0 +1,37 @@ +module Gitlab + module Graphql + class Variables + Invalid = Class.new(Gitlab::Graphql::StandardGraphqlError) + + def initialize(param) + @param = param + end + + def to_h + ensure_hash(@param) + end + + private + + # Handle form data, JSON body, or a blank value + def ensure_hash(ambiguous_param) + case ambiguous_param + when String + if ambiguous_param.present? + ensure_hash(JSON.parse(ambiguous_param)) + else + {} + end + when Hash, ActionController::Parameters + ambiguous_param + when nil + {} + else + raise Invalid, "Unexpected parameter: #{ambiguous_param}" + end + rescue JSON::ParserError => e + raise Invalid.new(e) + end + end + end +end diff --git a/lib/gitlab/import_export/repo_saver.rb b/lib/gitlab/import_export/repo_saver.rb index 695462c7dd2..0c224bd1971 100644 --- a/lib/gitlab/import_export/repo_saver.rb +++ b/lib/gitlab/import_export/repo_saver.rb @@ -26,10 +26,6 @@ module Gitlab @shared.error(e) false end - - def path_to_repo - @project.repository.path_to_repo - end end end end diff --git a/lib/gitlab/import_export/wiki_repo_saver.rb b/lib/gitlab/import_export/wiki_repo_saver.rb index 5fa2e101e29..2fd62c0fc7b 100644 --- a/lib/gitlab/import_export/wiki_repo_saver.rb +++ b/lib/gitlab/import_export/wiki_repo_saver.rb @@ -22,12 +22,8 @@ module Gitlab "project.wiki.bundle" end - def path_to_repo - @wiki.repository.path_to_repo - end - def wiki_repository_exists? - File.exist?(@wiki.repository.path_to_repo) && !@wiki.repository.empty? + @wiki.repository.exists? && !@wiki.repository.empty? end end end diff --git a/lib/gitlab/legacy_github_import/project_creator.rb b/lib/gitlab/legacy_github_import/project_creator.rb index 3ce245a8050..5e96eb16754 100644 --- a/lib/gitlab/legacy_github_import/project_creator.rb +++ b/lib/gitlab/legacy_github_import/project_creator.rb @@ -35,7 +35,10 @@ module Gitlab end def visibility_level - repo.private ? Gitlab::VisibilityLevel::PRIVATE : Gitlab::CurrentSettings.default_project_visibility + visibility_level = repo.private ? Gitlab::VisibilityLevel::PRIVATE : Gitlab::VisibilityLevel::PUBLIC + visibility_level = Gitlab::CurrentSettings.default_project_visibility if Gitlab::CurrentSettings.restricted_visibility_levels.include?(visibility_level) + + visibility_level end # diff --git a/lib/gitlab/task_helpers.rb b/lib/gitlab/task_helpers.rb index 42be301fd9b..723e655c150 100644 --- a/lib/gitlab/task_helpers.rb +++ b/lib/gitlab/task_helpers.rb @@ -128,10 +128,12 @@ module Gitlab end def all_repos - Gitlab.config.repositories.storages.each_value do |repository_storage| - IO.popen(%W(find #{repository_storage.legacy_disk_path} -mindepth 2 -type d -name *.git)) do |find| - find.each_line do |path| - yield path.chomp + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + Gitlab.config.repositories.storages.each_value do |repository_storage| + IO.popen(%W(find #{repository_storage.legacy_disk_path} -mindepth 2 -type d -name *.git)) do |find| + find.each_line do |path| + yield path.chomp + end end end end diff --git a/lib/gitlab/themes.rb b/lib/gitlab/themes.rb index d43eff5ba4a..9277b57f46f 100644 --- a/lib/gitlab/themes.rb +++ b/lib/gitlab/themes.rb @@ -12,11 +12,16 @@ module Gitlab # All available Themes THEMES = [ - Theme.new(1, 'Indigo', 'ui_indigo'), - Theme.new(2, 'Dark', 'ui_dark'), - Theme.new(3, 'Light', 'ui_light'), - Theme.new(4, 'Blue', 'ui_blue'), - Theme.new(5, 'Green', 'ui_green') + Theme.new(1, 'Indigo', 'ui-indigo'), + Theme.new(2, 'Light Indigo', 'ui-light-indigo'), + Theme.new(3, 'Blue', 'ui-blue'), + Theme.new(4, 'Light Blue', 'ui-light-blue'), + Theme.new(5, 'Green', 'ui-green'), + Theme.new(6, 'Light Green', 'ui-light-green'), + Theme.new(7, 'Red', 'ui-red'), + Theme.new(8, 'Light Red', 'ui-light-red'), + Theme.new(9, 'Dark', 'ui-dark'), + Theme.new(10, 'Light', 'ui-light') ].freeze # Convenience method to get a space-separated String of all the theme diff --git a/lib/gitlab/user_access.rb b/lib/gitlab/user_access.rb index 8cf5d636743..27560abfb96 100644 --- a/lib/gitlab/user_access.rb +++ b/lib/gitlab/user_access.rb @@ -65,7 +65,7 @@ module Gitlab return false unless can_access_git? return false unless project - return false if !user.can?(:push_code, project) && !project.branch_allows_maintainer_push?(user, ref) + return false if !user.can?(:push_code, project) && !project.branch_allows_collaboration?(user, ref) if protected?(ProtectedBranch, project, ref) protected_branch_accessible_to?(ref, action: :push) diff --git a/lib/gitlab/utils/override.rb b/lib/gitlab/utils/override.rb index 8bf6bcb1fe2..7b2a62fed48 100644 --- a/lib/gitlab/utils/override.rb +++ b/lib/gitlab/utils/override.rb @@ -87,18 +87,28 @@ module Gitlab end def included(base = nil) - return super if base.nil? # Rails concern, ignoring it + super + + queue_verification(base) + end + alias_method :prepended, :included + + def extended(mod) super + queue_verification(mod.singleton_class) + end + + def queue_verification(base) + return unless ENV['STATIC_VERIFICATION'] + if base.is_a?(Class) # We could check for Class in `override` # This could be `nil` if `override` was never called Override.extensions[self]&.add_class(base) end end - alias_method :prepended, :included - def self.extensions @extensions ||= {} end diff --git a/lib/object_storage/direct_upload.rb b/lib/object_storage/direct_upload.rb new file mode 100644 index 00000000000..61a69e7ffe4 --- /dev/null +++ b/lib/object_storage/direct_upload.rb @@ -0,0 +1,166 @@ +module ObjectStorage + # + # The DirectUpload c;ass generates a set of presigned URLs + # that can be used to upload data to object storage from untrusted component: Workhorse, Runner? + # + # For Google it assumes that the platform supports variable Content-Length. + # + # For AWS it initiates Multipart Upload and presignes a set of part uploads. + # Class calculates the best part size to be able to upload up to asked maximum size. + # The number of generated parts will never go above 100, + # but we will always try to reduce amount of generated parts. + # The part size is rounded-up to 5MB. + # + class DirectUpload + include Gitlab::Utils::StrongMemoize + + TIMEOUT = 4.hours + EXPIRE_OFFSET = 15.minutes + + MAXIMUM_MULTIPART_PARTS = 100 + MINIMUM_MULTIPART_SIZE = 5.megabytes + + attr_reader :credentials, :bucket_name, :object_name + attr_reader :has_length, :maximum_size + + def initialize(credentials, bucket_name, object_name, has_length:, maximum_size: nil) + unless has_length + raise ArgumentError, 'maximum_size has to be specified if length is unknown' unless maximum_size + end + + @credentials = credentials + @bucket_name = bucket_name + @object_name = object_name + @has_length = has_length + @maximum_size = maximum_size + end + + def to_hash + { + Timeout: TIMEOUT, + GetURL: get_url, + StoreURL: store_url, + DeleteURL: delete_url, + MultipartUpload: multipart_upload_hash + }.compact + end + + def multipart_upload_hash + return unless requires_multipart_upload? + + { + PartSize: rounded_multipart_part_size, + PartURLs: multipart_part_urls, + CompleteURL: multipart_complete_url, + AbortURL: multipart_abort_url + } + end + + def provider + credentials[:provider].to_s + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html + def get_url + connection.get_object_url(bucket_name, object_name, expire_at) + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectDELETE.html + def delete_url + connection.delete_object_url(bucket_name, object_name, expire_at) + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUT.html + def store_url + connection.put_object_url(bucket_name, object_name, expire_at, upload_options) + end + + def multipart_part_urls + Array.new(number_of_multipart_parts) do |part_index| + multipart_part_upload_url(part_index + 1) + end + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html + def multipart_part_upload_url(part_number) + connection.signed_url({ + method: 'PUT', + bucket_name: bucket_name, + object_name: object_name, + query: { uploadId: upload_id, partNumber: part_number }, + headers: upload_options + }, expire_at) + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html + def multipart_complete_url + connection.signed_url({ + method: 'POST', + bucket_name: bucket_name, + object_name: object_name, + query: { uploadId: upload_id }, + headers: { 'Content-Type' => 'application/xml' } + }, expire_at) + end + + # Implements https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadAbort.html + def multipart_abort_url + connection.signed_url({ + method: 'DELETE', + bucket_name: bucket_name, + object_name: object_name, + query: { uploadId: upload_id } + }, expire_at) + end + + private + + def rounded_multipart_part_size + # round multipart_part_size up to minimum_mulitpart_size + (multipart_part_size + MINIMUM_MULTIPART_SIZE - 1) / MINIMUM_MULTIPART_SIZE * MINIMUM_MULTIPART_SIZE + end + + def multipart_part_size + maximum_size / number_of_multipart_parts + end + + def number_of_multipart_parts + [ + # round maximum_size up to minimum_mulitpart_size + (maximum_size + MINIMUM_MULTIPART_SIZE - 1) / MINIMUM_MULTIPART_SIZE, + MAXIMUM_MULTIPART_PARTS + ].min + end + + def aws? + provider == 'AWS' + end + + def requires_multipart_upload? + aws? && !has_length + end + + def upload_id + return unless requires_multipart_upload? + + strong_memoize(:upload_id) do + new_upload = connection.initiate_multipart_upload(bucket_name, object_name) + new_upload.body["UploadId"] + end + end + + def expire_at + strong_memoize(:expire_at) do + Time.now + TIMEOUT + EXPIRE_OFFSET + end + end + + def upload_options + { 'Content-Type' => 'application/octet-stream' } + end + + def connection + @connection ||= ::Fog::Storage.new(credentials) + end + end +end diff --git a/lib/tasks/gitlab/cleanup.rake b/lib/tasks/gitlab/cleanup.rake index d6d15285489..52ae1330d7f 100644 --- a/lib/tasks/gitlab/cleanup.rake +++ b/lib/tasks/gitlab/cleanup.rake @@ -12,7 +12,7 @@ namespace :gitlab do namespaces = Namespace.pluck(:path) namespaces << HASHED_REPOSITORY_NAME # add so that it will be ignored Gitlab.config.repositories.storages.each do |name, repository_storage| - git_base_path = repository_storage.legacy_disk_path + git_base_path = Gitlab::GitalyClient::StorageSettings.allow_disk_access { repository_storage.legacy_disk_path } all_dirs = Dir.glob(git_base_path + '/*') puts git_base_path.color(:yellow) @@ -54,7 +54,8 @@ namespace :gitlab do move_suffix = "+orphaned+#{Time.now.to_i}" Gitlab.config.repositories.storages.each do |name, repository_storage| - repo_root = repository_storage.legacy_disk_path + repo_root = Gitlab::GitalyClient::StorageSettings.allow_disk_access { repository_storage.legacy_disk_path } + # Look for global repos (legacy, depth 1) and normal repos (depth 2) IO.popen(%W(find #{repo_root} -mindepth 1 -maxdepth 2 -name *.git)) do |find| find.each_line do |path| diff --git a/lib/tasks/gitlab/traces.rake b/lib/tasks/gitlab/traces.rake index fd2a4f2d11a..ddcca69711f 100644 --- a/lib/tasks/gitlab/traces.rake +++ b/lib/tasks/gitlab/traces.rake @@ -8,9 +8,7 @@ namespace :gitlab do logger = Logger.new(STDOUT) logger.info('Archiving legacy traces') - Ci::Build.finished - .where('NOT EXISTS (?)', - Ci::JobArtifact.select(1).trace.where('ci_builds.id = ci_job_artifacts.job_id')) + Ci::Build.finished.without_archived_trace .order(id: :asc) .find_in_batches(batch_size: 1000) do |jobs| job_ids = jobs.map { |job| [job.id] } diff --git a/locale/gitlab.pot b/locale/gitlab.pot index 035a2275d9f..f63f2a89aa9 100644 --- a/locale/gitlab.pot +++ b/locale/gitlab.pot @@ -331,7 +331,7 @@ msgstr "" msgid "All features are enabled for blank projects, from templates, or when importing, but you can disable them afterward in the project settings." msgstr "" -msgid "Allow edits from maintainers." +msgid "Allow commits from members who can merge to the target branch." msgstr "" msgid "Allow rendering of PlantUML diagrams in Asciidoc documents." @@ -439,7 +439,7 @@ msgstr "" msgid "Artifacts" msgstr "" -msgid "Ask your group master to setup a group Runner." +msgid "Ask your group maintainer to setup a group Runner." msgstr "" msgid "Assign custom color like #FF0000" @@ -696,7 +696,7 @@ msgstr "" msgid "Branches|Once you confirm and press %{delete_protected_branch}, it cannot be undone or recovered." msgstr "" -msgid "Branches|Only a project master or owner can delete a protected branch" +msgid "Branches|Only a project maintainer or owner can delete a protected branch" msgstr "" msgid "Branches|Overview" @@ -2237,7 +2237,7 @@ msgstr "" msgid "Group Runners" msgstr "" -msgid "Group masters can register group runners in the %{link}" +msgid "Group maintainers can register group runners in the %{link}" msgstr "" msgid "GroupSettings|Prevent sharing a project within %{group} with other groups" @@ -4764,7 +4764,7 @@ msgstr "" msgid "You have reached your project limit" msgstr "" -msgid "You must have master access to force delete a lock" +msgid "You must have maintainer access to force delete a lock" msgstr "" msgid "You must sign in to star a project" @@ -4894,7 +4894,7 @@ msgstr "" msgid "mrWidget|%{metricsLinkStart} Memory %{metricsLinkEnd} usage is %{emphasisStart} unchanged %{emphasisEnd} at %{memoryFrom}MB" msgstr "" -msgid "mrWidget|Allows edits from maintainers" +msgid "mrWidget|Allows commits from members who can merge to the target branch" msgstr "" msgid "mrWidget|Cancel automatic merge" diff --git a/qa/Dockerfile b/qa/Dockerfile index 77cee9c5461..abf2184e1e2 100644 --- a/qa/Dockerfile +++ b/qa/Dockerfile @@ -28,6 +28,15 @@ RUN apt-get update -q && apt-get install -y google-chrome-stable && apt-get clea RUN wget -q https://chromedriver.storage.googleapis.com/$(wget -q -O - https://chromedriver.storage.googleapis.com/LATEST_RELEASE)/chromedriver_linux64.zip RUN unzip chromedriver_linux64.zip -d /usr/local/bin +## +# Install gcloud and kubectl CLI used in Auto DevOps test to create K8s +# clusters +# +RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \ + echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \ + curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \ + apt-get update -y && apt-get install google-cloud-sdk kubectl -y + WORKDIR /home/qa COPY ./Gemfile* ./ RUN bundle install @@ -41,6 +41,7 @@ module QA autoload :SecretVariable, 'qa/factory/resource/secret_variable' autoload :Runner, 'qa/factory/resource/runner' autoload :PersonalAccessToken, 'qa/factory/resource/personal_access_token' + autoload :KubernetesCluster, 'qa/factory/resource/kubernetes_cluster' end module Repository @@ -72,6 +73,7 @@ module QA module Integration autoload :LDAP, 'qa/scenario/test/integration/ldap' + autoload :Kubernetes, 'qa/scenario/test/integration/kubernetes' autoload :Mattermost, 'qa/scenario/test/integration/mattermost' end @@ -150,6 +152,15 @@ module QA autoload :Show, 'qa/page/project/issue/show' autoload :Index, 'qa/page/project/issue/index' end + + module Operations + module Kubernetes + autoload :Index, 'qa/page/project/operations/kubernetes/index' + autoload :Add, 'qa/page/project/operations/kubernetes/add' + autoload :AddExisting, 'qa/page/project/operations/kubernetes/add_existing' + autoload :Show, 'qa/page/project/operations/kubernetes/show' + end + end end module Profile @@ -195,6 +206,7 @@ module QA # module Service autoload :Shellout, 'qa/service/shellout' + autoload :KubernetesCluster, 'qa/service/kubernetes_cluster' autoload :Omnibus, 'qa/service/omnibus' autoload :Runner, 'qa/service/runner' end diff --git a/qa/qa/factory/repository/push.rb b/qa/qa/factory/repository/push.rb index 795f1f9cb1a..28711c12701 100644 --- a/qa/qa/factory/repository/push.rb +++ b/qa/qa/factory/repository/push.rb @@ -15,7 +15,7 @@ module QA def initialize @file_name = 'file.txt' @file_content = '# This is test project' - @commit_message = "Add #{@file_name}" + @commit_message = "This is a test commit" @branch_name = 'master' @new_branch = true end @@ -24,6 +24,12 @@ module QA @remote_branch ||= branch_name end + def directory=(dir) + raise "Must set directory as a Pathname" unless dir.is_a?(Pathname) + + @directory = dir + end + def fabricate! project.visit! @@ -43,7 +49,14 @@ module QA repository.checkout(branch_name) end - repository.add_file(file_name, file_content) + if @directory + @directory.each_child do |f| + repository.add_file(f.basename, f.read) if f.file? + end + else + repository.add_file(file_name, file_content) + end + repository.commit(commit_message) repository.push_changes("#{branch_name}:#{remote_branch}") end diff --git a/qa/qa/factory/resource/kubernetes_cluster.rb b/qa/qa/factory/resource/kubernetes_cluster.rb new file mode 100644 index 00000000000..f32cf985e9d --- /dev/null +++ b/qa/qa/factory/resource/kubernetes_cluster.rb @@ -0,0 +1,55 @@ +require 'securerandom' + +module QA + module Factory + module Resource + class KubernetesCluster < Factory::Base + attr_writer :project, :cluster, + :install_helm_tiller, :install_ingress, :install_prometheus, :install_runner + + product :ingress_ip do + Page::Project::Operations::Kubernetes::Show.perform do |page| + page.ingress_ip + end + end + + def fabricate! + @project.visit! + + Page::Menu::Side.act { click_operations_kubernetes } + + Page::Project::Operations::Kubernetes::Index.perform do |page| + page.add_kubernetes_cluster + end + + Page::Project::Operations::Kubernetes::Add.perform do |page| + page.add_existing_cluster + end + + Page::Project::Operations::Kubernetes::AddExisting.perform do |page| + page.set_cluster_name(@cluster.cluster_name) + page.set_api_url(@cluster.api_url) + page.set_ca_certificate(@cluster.ca_certificate) + page.set_token(@cluster.token) + page.add_cluster! + end + + if @install_helm_tiller + Page::Project::Operations::Kubernetes::Show.perform do |page| + # Helm must be installed before everything else + page.install!(:helm) + page.await_installed(:helm) + + page.install!(:ingress) if @install_ingress + page.await_installed(:ingress) if @install_ingress + page.install!(:prometheus) if @install_prometheus + page.await_installed(:prometheus) if @install_prometheus + page.install!(:runner) if @install_runner + page.await_installed(:runner) if @install_runner + end + end + end + end + end + end +end diff --git a/qa/qa/fixtures/auto_devops_rack/Gemfile b/qa/qa/fixtures/auto_devops_rack/Gemfile new file mode 100644 index 00000000000..fc7514242d0 --- /dev/null +++ b/qa/qa/fixtures/auto_devops_rack/Gemfile @@ -0,0 +1,3 @@ +source 'https://rubygems.org' +gem 'rack' +gem 'rake' diff --git a/qa/qa/fixtures/auto_devops_rack/Gemfile.lock b/qa/qa/fixtures/auto_devops_rack/Gemfile.lock new file mode 100644 index 00000000000..09cf72c48ac --- /dev/null +++ b/qa/qa/fixtures/auto_devops_rack/Gemfile.lock @@ -0,0 +1,15 @@ +GEM + remote: https://rubygems.org/ + specs: + rack (2.0.4) + rake (12.3.0) + +PLATFORMS + ruby + +DEPENDENCIES + rack + rake + +BUNDLED WITH + 1.16.1 diff --git a/qa/qa/fixtures/auto_devops_rack/Rakefile b/qa/qa/fixtures/auto_devops_rack/Rakefile new file mode 100644 index 00000000000..c865c9aaac1 --- /dev/null +++ b/qa/qa/fixtures/auto_devops_rack/Rakefile @@ -0,0 +1,7 @@ +require 'rake/testtask' + +task default: %w[test] + +task :test do + puts "ok" +end diff --git a/qa/qa/fixtures/auto_devops_rack/config.ru b/qa/qa/fixtures/auto_devops_rack/config.ru new file mode 100644 index 00000000000..bde8e15488a --- /dev/null +++ b/qa/qa/fixtures/auto_devops_rack/config.ru @@ -0,0 +1 @@ +run lambda { |env| [200, { 'Content-Type' => 'text/plain' }, StringIO.new("Hello World!\n")] } diff --git a/qa/qa/page/menu/side.rb b/qa/qa/page/menu/side.rb index 7e028add2ef..3630b7e8568 100644 --- a/qa/qa/page/menu/side.rb +++ b/qa/qa/page/menu/side.rb @@ -7,9 +7,11 @@ module QA element :settings_link, 'link_to edit_project_path' element :repository_link, "title: 'Repository'" element :pipelines_settings_link, "title: 'CI / CD'" + element :operations_kubernetes_link, "title: _('Kubernetes')" element :issues_link, /link_to.*shortcuts-issues/ element :issues_link_text, "Issues" element :top_level_items, '.sidebar-top-level-items' + element :operations_section, "class: 'shortcuts-operations'" element :activity_link, "title: 'Activity'" end @@ -33,6 +35,14 @@ module QA end end + def click_operations_kubernetes + hover_operations do + within_submenu do + click_link('Kubernetes') + end + end + end + def click_ci_cd_pipelines within_sidebar do click_link('CI / CD') @@ -61,6 +71,14 @@ module QA end end + def hover_operations + within_sidebar do + find('.shortcuts-operations').hover + + yield + end + end + def within_sidebar page.within('.sidebar-top-level-items') do yield diff --git a/qa/qa/page/project/job/show.rb b/qa/qa/page/project/job/show.rb index 83bb224b5c3..f1a859fd8ee 100644 --- a/qa/qa/page/project/job/show.rb +++ b/qa/qa/page/project/job/show.rb @@ -4,7 +4,7 @@ module QA::Page COMPLETED_STATUSES = %w[passed failed canceled blocked skipped manual].freeze # excludes created, pending, running PASSED_STATUS = 'passed'.freeze - view 'app/views/projects/jobs/show.html.haml' do + view 'app/views/shared/builds/_build_output.html.haml' do element :build_output, '.js-build-output' end diff --git a/qa/qa/page/project/operations/kubernetes/add.rb b/qa/qa/page/project/operations/kubernetes/add.rb new file mode 100644 index 00000000000..9b3c482fa6c --- /dev/null +++ b/qa/qa/page/project/operations/kubernetes/add.rb @@ -0,0 +1,19 @@ +module QA + module Page + module Project + module Operations + module Kubernetes + class Add < Page::Base + view 'app/views/projects/clusters/new.html.haml' do + element :add_kubernetes_cluster_button, "link_to s_('ClusterIntegration|Add an existing Kubernetes cluster')" + end + + def add_existing_cluster + click_on 'Add an existing Kubernetes cluster' + end + end + end + end + end + end +end diff --git a/qa/qa/page/project/operations/kubernetes/add_existing.rb b/qa/qa/page/project/operations/kubernetes/add_existing.rb new file mode 100644 index 00000000000..eef82b5f329 --- /dev/null +++ b/qa/qa/page/project/operations/kubernetes/add_existing.rb @@ -0,0 +1,39 @@ +module QA + module Page + module Project + module Operations + module Kubernetes + class AddExisting < Page::Base + view 'app/views/projects/clusters/user/_form.html.haml' do + element :cluster_name, 'text_field :name' + element :api_url, 'text_field :api_url' + element :ca_certificate, 'text_area :ca_cert' + element :token, 'text_field :token' + element :add_cluster_button, "submit s_('ClusterIntegration|Add Kubernetes cluster')" + end + + def set_cluster_name(name) + fill_in 'cluster_name', with: name + end + + def set_api_url(api_url) + fill_in 'cluster_platform_kubernetes_attributes_api_url', with: api_url + end + + def set_ca_certificate(ca_certificate) + fill_in 'cluster_platform_kubernetes_attributes_ca_cert', with: ca_certificate + end + + def set_token(token) + fill_in 'cluster_platform_kubernetes_attributes_token', with: token + end + + def add_cluster! + click_on 'Add Kubernetes cluster' + end + end + end + end + end + end +end diff --git a/qa/qa/page/project/operations/kubernetes/index.rb b/qa/qa/page/project/operations/kubernetes/index.rb new file mode 100644 index 00000000000..7261b5645da --- /dev/null +++ b/qa/qa/page/project/operations/kubernetes/index.rb @@ -0,0 +1,19 @@ +module QA + module Page + module Project + module Operations + module Kubernetes + class Index < Page::Base + view 'app/views/projects/clusters/_empty_state.html.haml' do + element :add_kubernetes_cluster_button, "link_to s_('ClusterIntegration|Add Kubernetes cluster')" + end + + def add_kubernetes_cluster + click_on 'Add Kubernetes cluster' + end + end + end + end + end + end +end diff --git a/qa/qa/page/project/operations/kubernetes/show.rb b/qa/qa/page/project/operations/kubernetes/show.rb new file mode 100644 index 00000000000..4923304133e --- /dev/null +++ b/qa/qa/page/project/operations/kubernetes/show.rb @@ -0,0 +1,39 @@ +module QA + module Page + module Project + module Operations + module Kubernetes + class Show < Page::Base + view 'app/assets/javascripts/clusters/components/application_row.vue' do + element :application_row, 'js-cluster-application-row-${this.id}' + element :install_button, "s__('ClusterIntegration|Install')" + element :installed_button, "s__('ClusterIntegration|Installed')" + end + + view 'app/assets/javascripts/clusters/components/applications.vue' do + element :ingress_ip_address, 'id="ingress-ip-address"' + end + + def install!(application_name) + within(".js-cluster-application-row-#{application_name}") do + click_on 'Install' + end + end + + def await_installed(application_name) + within(".js-cluster-application-row-#{application_name}") do + page.has_text?('Installed', wait: 300) + end + end + + def ingress_ip + # We need to wait longer since it can take some time before the + # ip address is assigned for the ingress controller + page.find('#ingress-ip-address', wait: 500).value + end + end + end + end + end + end +end diff --git a/qa/qa/page/project/pipeline/show.rb b/qa/qa/page/project/pipeline/show.rb index ec61c47b3bb..de849b3eee8 100644 --- a/qa/qa/page/project/pipeline/show.rb +++ b/qa/qa/page/project/pipeline/show.rb @@ -24,10 +24,10 @@ module QA::Page end end - def has_build?(name, status: :success) + def has_build?(name, status: :success, wait:) within('.pipeline-graph') do within('.ci-job-component', text: name) do - has_selector?(".ci-status-icon-#{status}") + has_selector?(".ci-status-icon-#{status}", wait: wait) end end end diff --git a/qa/qa/page/project/settings/ci_cd.rb b/qa/qa/page/project/settings/ci_cd.rb index 145c3d3ddfa..dfb71e0a9f0 100644 --- a/qa/qa/page/project/settings/ci_cd.rb +++ b/qa/qa/page/project/settings/ci_cd.rb @@ -8,6 +8,13 @@ module QA # rubocop:disable Naming/FileName view 'app/views/projects/settings/ci_cd/show.html.haml' do element :runners_settings, 'Runners settings' element :secret_variables, 'Variables' + element :auto_devops_section, 'Auto DevOps' + end + + view 'app/views/projects/settings/ci_cd/_autodevops_form.html.haml' do + element :enable_auto_devops_button, 'Enable Auto DevOps' + element :domain_input, 'Domain' + element :save_changes_button, "submit 'Save changes'" end def expand_runners_settings(&block) @@ -21,6 +28,14 @@ module QA # rubocop:disable Naming/FileName Settings::SecretVariables.perform(&block) end end + + def enable_auto_devops_with_domain(domain) + expand_section('Auto DevOps') do + choose 'Enable Auto DevOps' + fill_in 'Domain', with: domain + click_on 'Save changes' + end + end end end end diff --git a/qa/qa/page/project/settings/protected_branches.rb b/qa/qa/page/project/settings/protected_branches.rb index 63bc3aaa2bc..a0903c3c4dc 100644 --- a/qa/qa/page/project/settings/protected_branches.rb +++ b/qa/qa/page/project/settings/protected_branches.rb @@ -41,7 +41,7 @@ module QA end def allow_devs_and_masters_to_push - click_allow(:push, 'Developers + Masters') + click_allow(:push, 'Developers + Maintainers') end def allow_no_one_to_merge @@ -49,7 +49,7 @@ module QA end def allow_devs_and_masters_to_merge - click_allow(:merge, 'Developers + Masters') + click_allow(:merge, 'Developers + Maintainers') end def protect_branch diff --git a/qa/qa/scenario/test/integration/kubernetes.rb b/qa/qa/scenario/test/integration/kubernetes.rb new file mode 100644 index 00000000000..7479073e979 --- /dev/null +++ b/qa/qa/scenario/test/integration/kubernetes.rb @@ -0,0 +1,11 @@ +module QA + module Scenario + module Test + module Integration + class Kubernetes < Test::Instance + tags :kubernetes + end + end + end + end +end diff --git a/qa/qa/service/kubernetes_cluster.rb b/qa/qa/service/kubernetes_cluster.rb new file mode 100644 index 00000000000..604bc522983 --- /dev/null +++ b/qa/qa/service/kubernetes_cluster.rb @@ -0,0 +1,66 @@ +require 'securerandom' +require 'mkmf' + +module QA + module Service + class KubernetesCluster + include Service::Shellout + + attr_reader :api_url, :ca_certificate, :token + + def cluster_name + @cluster_name ||= "qa-cluster-#{SecureRandom.hex(4)}-#{Time.now.utc.strftime("%Y%m%d%H%M%S")}" + end + + def create! + validate_dependencies + login_if_not_already_logged_in + + shell <<~CMD.tr("\n", ' ') + gcloud container clusters + create #{cluster_name} + --enable-legacy-authorization + --zone us-central1-a + && gcloud container clusters + get-credentials #{cluster_name} + CMD + + @api_url = `kubectl config view --minify -o jsonpath='{.clusters[].cluster.server}'` + @ca_certificate = Base64.decode64(`kubectl get secrets -o jsonpath="{.items[0].data['ca\\.crt']}"`) + @token = Base64.decode64(`kubectl get secrets -o jsonpath='{.items[0].data.token}'`) + self + end + + def remove! + shell("gcloud container clusters delete #{cluster_name} --quiet --async") + end + + private + + def validate_dependencies + find_executable('gcloud') || raise("You must first install `gcloud` executable to run these tests.") + find_executable('kubectl') || raise("You must first install `kubectl` executable to run these tests.") + end + + def login_if_not_already_logged_in + account = `gcloud auth list --filter=status:ACTIVE --format="value(account)"` + if account.empty? + attempt_login_with_env_vars + else + puts "gcloud account found. Using: #{account} for creating K8s cluster." + end + end + + def attempt_login_with_env_vars + puts "No gcloud account. Attempting to login from env vars GCLOUD_ACCOUNT_EMAIL and GCLOUD_ACCOUNT_KEY." + gcloud_account_key = Tempfile.new('gcloud-account-key') + gcloud_account_key.write(ENV.fetch("GCLOUD_ACCOUNT_KEY")) + gcloud_account_key.close + gcloud_account_email = ENV.fetch("GCLOUD_ACCOUNT_EMAIL") + shell("gcloud auth activate-service-account #{gcloud_account_email} --key-file #{gcloud_account_key.path}") + ensure + gcloud_account_key && gcloud_account_key.unlink + end + end + end +end diff --git a/qa/qa/service/runner.rb b/qa/qa/service/runner.rb index c0352e0467a..9417c707105 100644 --- a/qa/qa/service/runner.rb +++ b/qa/qa/service/runner.rb @@ -3,7 +3,6 @@ require 'securerandom' module QA module Service class Runner - include Scenario::Actable include Service::Shellout attr_accessor :token, :address, :tags, :image diff --git a/qa/qa/specs/features/project/auto_devops_spec.rb b/qa/qa/specs/features/project/auto_devops_spec.rb new file mode 100644 index 00000000000..f3f59d33457 --- /dev/null +++ b/qa/qa/specs/features/project/auto_devops_spec.rb @@ -0,0 +1,55 @@ +module QA + feature 'Auto Devops', :kubernetes do + after do + @cluster&.remove! + end + + scenario 'user creates a new project and runs auto devops' do + Runtime::Browser.visit(:gitlab, Page::Main::Login) + Page::Main::Login.act { sign_in_using_credentials } + + project = Factory::Resource::Project.fabricate! do |p| + p.name = 'project-with-autodevops' + p.description = 'Project with Auto Devops' + end + + # Create Auto Devops compatible repo + Factory::Repository::Push.fabricate! do |push| + push.project = project + push.directory = Pathname + .new(__dir__) + .join('../../../fixtures/auto_devops_rack') + push.commit_message = 'Create Auto DevOps compatible rack application' + end + + Page::Project::Show.act { wait_for_push } + + # Create and connect K8s cluster + @cluster = Service::KubernetesCluster.new.create! + kubernetes_cluster = Factory::Resource::KubernetesCluster.fabricate! do |cluster| + cluster.project = project + cluster.cluster = @cluster + cluster.install_helm_tiller = true + cluster.install_ingress = true + cluster.install_prometheus = true + cluster.install_runner = true + end + + project.visit! + Page::Menu::Side.act { click_ci_cd_settings } + Page::Project::Settings::CICD.perform do |p| + p.enable_auto_devops_with_domain("#{kubernetes_cluster.ingress_ip}.nip.io") + end + + project.visit! + Page::Menu::Side.act { click_ci_cd_pipelines } + Page::Project::Pipeline::Index.act { go_to_latest_pipeline } + + Page::Project::Pipeline::Show.perform do |pipeline| + expect(pipeline).to have_build('build', status: :success, wait: 600) + expect(pipeline).to have_build('test', status: :success, wait: 600) + expect(pipeline).to have_build('production', status: :success, wait: 600) + end + end + end +end diff --git a/qa/qa/specs/features/repository/protected_branches_spec.rb b/qa/qa/specs/features/repository/protected_branches_spec.rb index 406b2772b64..9e438aa3c30 100644 --- a/qa/qa/specs/features/repository/protected_branches_spec.rb +++ b/qa/qa/specs/features/repository/protected_branches_spec.rb @@ -35,7 +35,7 @@ module QA end expect(protected_branch.name).to have_content(branch_name) - expect(protected_branch.push_allowance).to have_content('Developers + Masters') + expect(protected_branch.push_allowance).to have_content('Developers + Maintainers') end scenario 'users without authorization cannot push to protected branch' do diff --git a/spec/controllers/application_controller_spec.rb b/spec/controllers/application_controller_spec.rb index b048da1991c..773bf25ed44 100644 --- a/spec/controllers/application_controller_spec.rb +++ b/spec/controllers/application_controller_spec.rb @@ -1,3 +1,4 @@ +# coding: utf-8 require 'spec_helper' describe ApplicationController do @@ -477,4 +478,85 @@ describe ApplicationController do end end end + + describe '#append_info_to_payload' do + controller(described_class) do + attr_reader :last_payload + + def index + render text: 'authenticated' + end + + def append_info_to_payload(payload) + super + + @last_payload = payload + end + end + + it 'does not log errors with a 200 response' do + get :index + + expect(controller.last_payload.has_key?(:response)).to be_falsey + end + + context '422 errors' do + it 'logs a response with a string' do + response = spy(ActionDispatch::Response, status: 422, body: 'Hello world', content_type: 'application/json') + allow(controller).to receive(:response).and_return(response) + get :index + + expect(controller.last_payload[:response]).to eq('Hello world') + end + + it 'logs a response with an array' do + body = ['I want', 'my hat back'] + response = spy(ActionDispatch::Response, status: 422, body: body, content_type: 'application/json') + allow(controller).to receive(:response).and_return(response) + get :index + + expect(controller.last_payload[:response]).to eq(body) + end + + it 'does not log a string with an empty body' do + response = spy(ActionDispatch::Response, status: 422, body: nil, content_type: 'application/json') + allow(controller).to receive(:response).and_return(response) + get :index + + expect(controller.last_payload.has_key?(:response)).to be_falsey + end + + it 'does not log an HTML body' do + response = spy(ActionDispatch::Response, status: 422, body: 'This is a test', content_type: 'application/html') + allow(controller).to receive(:response).and_return(response) + get :index + + expect(controller.last_payload.has_key?(:response)).to be_falsey + end + end + end + + describe '#access_denied' do + controller(described_class) do + def index + access_denied!(params[:message]) + end + end + + before do + sign_in user + end + + it 'renders a 404 without a message' do + get :index + + expect(response).to have_gitlab_http_status(404) + end + + it 'renders a 403 when a message is passed to access denied' do + get :index, message: 'None shall pass' + + expect(response).to have_gitlab_http_status(403) + end + end end diff --git a/spec/controllers/concerns/controller_with_cross_project_access_check_spec.rb b/spec/controllers/concerns/controller_with_cross_project_access_check_spec.rb index 27f558e1b5d..d20471ef603 100644 --- a/spec/controllers/concerns/controller_with_cross_project_access_check_spec.rb +++ b/spec/controllers/concerns/controller_with_cross_project_access_check_spec.rb @@ -43,13 +43,13 @@ describe ControllerWithCrossProjectAccessCheck do end end - it 'renders a 404 with trying to access a cross project page' do + it 'renders a 403 with trying to access a cross project page' do message = "This page is unavailable because you are not allowed to read "\ "information across multiple projects." get :index - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) expect(response.body).to match(/#{message}/) end @@ -119,7 +119,7 @@ describe ControllerWithCrossProjectAccessCheck do get :index - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) end it 'is executed when the `unless` condition returns true' do @@ -127,19 +127,19 @@ describe ControllerWithCrossProjectAccessCheck do get :index - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) end it 'does not skip the check on an action that is not skipped' do get :show, id: 'hello' - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) end it 'does not skip the check on an action that was not defined to skip' do get :edit, id: 'hello' - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) end end end diff --git a/spec/controllers/graphql_controller_spec.rb b/spec/controllers/graphql_controller_spec.rb new file mode 100644 index 00000000000..1449036e148 --- /dev/null +++ b/spec/controllers/graphql_controller_spec.rb @@ -0,0 +1,69 @@ +require 'spec_helper' + +describe GraphqlController do + describe 'execute' do + let(:user) { nil } + + before do + sign_in(user) if user + + run_test_query! + end + + subject { query_response } + + context 'graphql is disabled by feature flag' do + let(:user) { nil } + + before do + stub_feature_flags(graphql: false) + end + + it 'returns 404' do + run_test_query! + + expect(response).to have_gitlab_http_status(404) + end + end + + context 'signed out' do + let(:user) { nil } + + it 'runs the query with current_user: nil' do + is_expected.to eq('echo' => 'nil says: test success') + end + end + + context 'signed in' do + let(:user) { create(:user, username: 'Simon') } + + it 'runs the query with current_user set' do + is_expected.to eq('echo' => '"Simon" says: test success') + end + end + + context 'invalid variables' do + it 'returns an error' do + run_test_query!(variables: "This is not JSON") + + expect(response).to have_gitlab_http_status(422) + expect(json_response['errors'].first['message']).not_to be_nil + end + end + end + + # Chosen to exercise all the moving parts in GraphqlController#execute + def run_test_query!(variables: { 'text' => 'test success' }) + query = <<~QUERY + query Echo($text: String) { + echo(text: $text) + } + QUERY + + post :execute, query: query, operationName: 'Echo', variables: variables + end + + def query_response + json_response['data'] + end +end diff --git a/spec/controllers/groups/shared_projects_controller_spec.rb b/spec/controllers/groups/shared_projects_controller_spec.rb index d8fa41abb18..003c8c262e7 100644 --- a/spec/controllers/groups/shared_projects_controller_spec.rb +++ b/spec/controllers/groups/shared_projects_controller_spec.rb @@ -38,7 +38,7 @@ describe Groups::SharedProjectsController do end it 'allows filtering shared projects' do - project = create(:project, :archived, namespace: user.namespace, name: "Searching for") + project = create(:project, namespace: user.namespace, name: "Searching for") share_project(project) get_shared_projects(filter: 'search') @@ -55,5 +55,14 @@ describe Groups::SharedProjectsController do expect(json_project_ids).to eq([second_project.id, shared_project.id]) end + + it 'does not include archived projects' do + archived_project = create(:project, :archived, namespace: user.namespace) + share_project(archived_project) + + get_shared_projects + + expect(json_project_ids).to contain_exactly(shared_project.id) + end end end diff --git a/spec/controllers/projects/merge_requests_controller_spec.rb b/spec/controllers/projects/merge_requests_controller_spec.rb index 6e8de6db9c3..6e710c9b20b 100644 --- a/spec/controllers/projects/merge_requests_controller_spec.rb +++ b/spec/controllers/projects/merge_requests_controller_spec.rb @@ -80,6 +80,16 @@ describe Projects::MergeRequestsController do )) end end + + context "that is invalid" do + let(:merge_request) { create(:invalid_merge_request, target_project: project, source_project: project) } + + it "renders merge request page" do + go(format: :html) + + expect(response).to be_success + end + end end describe 'as json' do @@ -106,6 +116,16 @@ describe Projects::MergeRequestsController do expect(response).to match_response_schema('entities/merge_request_widget') end end + + context "that is invalid" do + let(:merge_request) { create(:invalid_merge_request, target_project: project, source_project: project) } + + it "renders merge request page" do + go(format: :json) + + expect(response).to be_success + end + end end describe "as diff" do diff --git a/spec/controllers/projects/milestones_controller_spec.rb b/spec/controllers/projects/milestones_controller_spec.rb index 548c5ef36e7..02b30f9bc6d 100644 --- a/spec/controllers/projects/milestones_controller_spec.rb +++ b/spec/controllers/projects/milestones_controller_spec.rb @@ -57,19 +57,36 @@ describe Projects::MilestonesController do context "as json" do let!(:group) { create(:group, :public) } let!(:group_milestone) { create(:milestone, group: group) } - let!(:group_member) { create(:group_member, group: group, user: user) } - before do - project.update(namespace: group) - get :index, namespace_id: project.namespace.id, project_id: project.id, format: :json + context 'with a single group ancestor' do + before do + project.update(namespace: group) + get :index, namespace_id: project.namespace.id, project_id: project.id, format: :json + end + + it "queries projects milestones and groups milestones" do + milestones = assigns(:milestones) + + expect(milestones.count).to eq(2) + expect(milestones).to match_array([milestone, group_milestone]) + end end - it "queries projects milestones and groups milestones" do - milestones = assigns(:milestones) + context 'with nested groups', :nested_groups do + let!(:subgroup) { create(:group, :public, parent: group) } + let!(:subgroup_milestone) { create(:milestone, group: subgroup) } + + before do + project.update(namespace: subgroup) + get :index, namespace_id: project.namespace.id, project_id: project.id, format: :json + end + + it "queries projects milestones and all ancestors milestones" do + milestones = assigns(:milestones) - expect(milestones.count).to eq(2) - expect(milestones.where(project_id: nil).first).to eq(group_milestone) - expect(milestones.where(group_id: nil).first).to eq(milestone) + expect(milestones.count).to eq(3) + expect(milestones).to match_array([milestone, group_milestone, subgroup_milestone]) + end end end end diff --git a/spec/controllers/projects/pipelines_controller_spec.rb b/spec/controllers/projects/pipelines_controller_spec.rb index 9e7bc20a6d1..92886e93077 100644 --- a/spec/controllers/projects/pipelines_controller_spec.rb +++ b/spec/controllers/projects/pipelines_controller_spec.rb @@ -17,44 +17,103 @@ describe Projects::PipelinesController do describe 'GET index.json' do before do - %w(pending running created success).each_with_index do |status, index| - sha = project.commit("HEAD~#{index}") - create(:ci_empty_pipeline, status: status, project: project, sha: sha) + %w(pending running success failed canceled).each_with_index do |status, index| + create_pipeline(status, project.commit("HEAD~#{index}")) end end - subject do - get :index, namespace_id: project.namespace, project_id: project, format: :json + context 'when using persisted stages', :request_store do + before do + stub_feature_flags(ci_pipeline_persisted_stages: true) + end + + it 'returns serialized pipelines', :request_store do + queries = ActiveRecord::QueryRecorder.new do + get_pipelines_index_json + end + + expect(response).to have_gitlab_http_status(:ok) + expect(response).to match_response_schema('pipeline') + + expect(json_response).to include('pipelines') + expect(json_response['pipelines'].count).to eq 5 + expect(json_response['count']['all']).to eq '5' + expect(json_response['count']['running']).to eq '1' + expect(json_response['count']['pending']).to eq '1' + expect(json_response['count']['finished']).to eq '3' + + json_response.dig('pipelines', 0, 'details', 'stages').tap do |stages| + expect(stages.count).to eq 3 + end + + expect(queries.count).to be + end end - it 'returns JSON with serialized pipelines' do - subject + context 'when using legacy stages', :request_store do + before do + stub_feature_flags(ci_pipeline_persisted_stages: false) + end - expect(response).to have_gitlab_http_status(:ok) - expect(response).to match_response_schema('pipeline') + it 'returns JSON with serialized pipelines', :request_store do + queries = ActiveRecord::QueryRecorder.new do + get_pipelines_index_json + end + + expect(response).to have_gitlab_http_status(:ok) + expect(response).to match_response_schema('pipeline') - expect(json_response).to include('pipelines') - expect(json_response['pipelines'].count).to eq 4 - expect(json_response['count']['all']).to eq '4' - expect(json_response['count']['running']).to eq '1' - expect(json_response['count']['pending']).to eq '1' - expect(json_response['count']['finished']).to eq '1' + expect(json_response).to include('pipelines') + expect(json_response['pipelines'].count).to eq 5 + expect(json_response['count']['all']).to eq '5' + expect(json_response['count']['running']).to eq '1' + expect(json_response['count']['pending']).to eq '1' + expect(json_response['count']['finished']).to eq '3' + + json_response.dig('pipelines', 0, 'details', 'stages').tap do |stages| + expect(stages.count).to eq 3 + end + + expect(queries.count).to be_within(3).of(30) + end end it 'does not include coverage data for the pipelines' do - subject + get_pipelines_index_json expect(json_response['pipelines'][0]).not_to include('coverage') end context 'when performing gitaly calls', :request_store do it 'limits the Gitaly requests' do - expect { subject }.to change { Gitlab::GitalyClient.get_request_count }.by(3) + expect { get_pipelines_index_json } + .to change { Gitlab::GitalyClient.get_request_count }.by(2) end end + + def get_pipelines_index_json + get :index, namespace_id: project.namespace, + project_id: project, + format: :json + end + + def create_pipeline(status, sha) + pipeline = create(:ci_empty_pipeline, status: status, + project: project, + sha: sha) + + create_build(pipeline, 'build', 1, 'build') + create_build(pipeline, 'test', 2, 'test') + create_build(pipeline, 'deploy', 3, 'deploy') + end + + def create_build(pipeline, stage, stage_idx, name) + status = %w[created running pending success failed canceled].sample + create(:ci_build, pipeline: pipeline, stage: stage, stage_idx: stage_idx, name: name, status: status) + end end - describe 'GET show JSON' do + describe 'GET show.json' do let(:pipeline) { create(:ci_pipeline_with_one_job, project: project) } it 'returns the pipeline' do @@ -67,6 +126,14 @@ describe Projects::PipelinesController do end context 'when the pipeline has multiple stages and groups', :request_store do + let(:project) { create(:project, :repository) } + + let(:pipeline) do + create(:ci_empty_pipeline, project: project, + user: user, + sha: project.commit.id) + end + before do create_build('build', 0, 'build') create_build('test', 1, 'rspec 0') @@ -74,11 +141,6 @@ describe Projects::PipelinesController do create_build('post deploy', 3, 'pages 0') end - let(:project) { create(:project, :repository) } - let(:pipeline) do - create(:ci_empty_pipeline, project: project, user: user, sha: project.commit.id) - end - it 'does not perform N + 1 queries' do control_count = ActiveRecord::QueryRecorder.new { get_pipeline_json }.count @@ -90,6 +152,7 @@ describe Projects::PipelinesController do create_build('post deploy', 3, 'pages 2') new_count = ActiveRecord::QueryRecorder.new { get_pipeline_json }.count + expect(new_count).to be_within(12).of(control_count) end end diff --git a/spec/controllers/search_controller_spec.rb b/spec/controllers/search_controller_spec.rb index 30c06ddf744..416a09e1684 100644 --- a/spec/controllers/search_controller_spec.rb +++ b/spec/controllers/search_controller_spec.rb @@ -32,7 +32,7 @@ describe SearchController do it 'still blocks searches without a project_id' do get :show, search: 'hello' - expect(response).to have_gitlab_http_status(404) + expect(response).to have_gitlab_http_status(403) end it 'allows searches with a project_id' do diff --git a/spec/controllers/users/terms_controller_spec.rb b/spec/controllers/users/terms_controller_spec.rb index a744463413c..0d77e91a67d 100644 --- a/spec/controllers/users/terms_controller_spec.rb +++ b/spec/controllers/users/terms_controller_spec.rb @@ -1,6 +1,7 @@ require 'spec_helper' describe Users::TermsController do + include TermsHelper let(:user) { create(:user) } let(:term) { create(:term) } @@ -15,10 +16,25 @@ describe Users::TermsController do expect(response).to have_gitlab_http_status(:redirect) end - it 'shows terms when they exist' do - term + context 'when terms exist' do + before do + stub_env('IN_MEMORY_APPLICATION_SETTINGS', 'false') + term + end + + it 'shows terms when they exist' do + get :index + + expect(response).to have_gitlab_http_status(:success) + end - expect(response).to have_gitlab_http_status(:success) + it 'shows a message when the user already accepted the terms' do + accept_terms(user) + + get :index + + expect(controller).to set_flash.now[:notice].to(/already accepted/) + end end end diff --git a/spec/factories/merge_requests.rb b/spec/factories/merge_requests.rb index fab0ec22450..3441ce1b8cb 100644 --- a/spec/factories/merge_requests.rb +++ b/spec/factories/merge_requests.rb @@ -54,6 +54,11 @@ FactoryBot.define do state :opened end + trait :invalid do + source_branch "feature_one" + target_branch "feature_two" + end + trait :locked do state :locked end @@ -98,6 +103,7 @@ FactoryBot.define do factory :merged_merge_request, traits: [:merged] factory :closed_merge_request, traits: [:closed] factory :reopened_merge_request, traits: [:opened] + factory :invalid_merge_request, traits: [:invalid] factory :merge_request_with_diffs, traits: [:with_diffs] factory :merge_request_with_diff_notes do after(:create) do |mr| diff --git a/spec/factories/term_agreements.rb b/spec/factories/term_agreements.rb index 557599e663d..3c4eebd0196 100644 --- a/spec/factories/term_agreements.rb +++ b/spec/factories/term_agreements.rb @@ -3,4 +3,12 @@ FactoryBot.define do term user end + + trait :declined do + accepted false + end + + trait :accepted do + accepted true + end end diff --git a/spec/features/markdown/markdown_spec.rb b/spec/features/markdown/markdown_spec.rb index f13d78d24e3..c86ba8c50a5 100644 --- a/spec/features/markdown/markdown_spec.rb +++ b/spec/features/markdown/markdown_spec.rb @@ -24,7 +24,7 @@ require 'erb' # # See the MarkdownFeature class for setup details. -describe 'GitLab Markdown' do +describe 'GitLab Markdown', :aggregate_failures do include Capybara::Node::Matchers include MarkupHelper include MarkdownMatchers @@ -44,112 +44,102 @@ describe 'GitLab Markdown' do # Shared behavior that all pipelines should exhibit shared_examples 'all pipelines' do - describe 'Redcarpet extensions' do - it 'does not parse emphasis inside of words' do + it 'includes Redcarpet extensions' do + aggregate_failures 'does not parse emphasis inside of words' do expect(doc.to_html).not_to match('foo<em>bar</em>baz') end - it 'parses table Markdown' do - aggregate_failures do - expect(doc).to have_selector('th:contains("Header")') - expect(doc).to have_selector('th:contains("Row")') - expect(doc).to have_selector('th:contains("Example")') - end + aggregate_failures 'parses table Markdown' do + expect(doc).to have_selector('th:contains("Header")') + expect(doc).to have_selector('th:contains("Row")') + expect(doc).to have_selector('th:contains("Example")') end - it 'allows Markdown in tables' do + aggregate_failures 'allows Markdown in tables' do expect(doc.at_css('td:contains("Baz")').children.to_html) .to eq '<strong>Baz</strong>' end - it 'parses fenced code blocks' do - aggregate_failures do - expect(doc).to have_selector('pre.code.highlight.js-syntax-highlight.c') - expect(doc).to have_selector('pre.code.highlight.js-syntax-highlight.python') - end + aggregate_failures 'parses fenced code blocks' do + expect(doc).to have_selector('pre.code.highlight.js-syntax-highlight.c') + expect(doc).to have_selector('pre.code.highlight.js-syntax-highlight.python') end - it 'parses mermaid code block' do - aggregate_failures do - expect(doc).to have_selector('pre[lang=mermaid] > code.js-render-mermaid') - end + aggregate_failures 'parses mermaid code block' do + expect(doc).to have_selector('pre[lang=mermaid] > code.js-render-mermaid') end - it 'parses strikethroughs' do + aggregate_failures 'parses strikethroughs' do expect(doc).to have_selector(%{del:contains("and this text doesn't")}) end - it 'parses superscript' do + aggregate_failures 'parses superscript' do expect(doc).to have_selector('sup', count: 2) end end - describe 'SanitizationFilter' do - it 'permits b elements' do + it 'includes SanitizationFilter' do + aggregate_failures 'permits b elements' do expect(doc).to have_selector('b:contains("b tag")') end - it 'permits em elements' do + aggregate_failures 'permits em elements' do expect(doc).to have_selector('em:contains("em tag")') end - it 'permits code elements' do + aggregate_failures 'permits code elements' do expect(doc).to have_selector('code:contains("code tag")') end - it 'permits kbd elements' do + aggregate_failures 'permits kbd elements' do expect(doc).to have_selector('kbd:contains("s")') end - it 'permits strike elements' do + aggregate_failures 'permits strike elements' do expect(doc).to have_selector('strike:contains(Emoji)') end - it 'permits img elements' do + aggregate_failures 'permits img elements' do expect(doc).to have_selector('img[data-src*="smile.png"]') end - it 'permits br elements' do + aggregate_failures 'permits br elements' do expect(doc).to have_selector('br') end - it 'permits hr elements' do + aggregate_failures 'permits hr elements' do expect(doc).to have_selector('hr') end - it 'permits span elements' do + aggregate_failures 'permits span elements' do expect(doc).to have_selector('span:contains("span tag")') end - it 'permits details elements' do + aggregate_failures 'permits details elements' do expect(doc).to have_selector('details:contains("Hiding the details")') end - it 'permits summary elements' do + aggregate_failures 'permits summary elements' do expect(doc).to have_selector('details summary:contains("collapsible")') end - it 'permits style attribute in th elements' do - aggregate_failures do - expect(doc.at_css('th:contains("Header")')['style']).to eq 'text-align: center' - expect(doc.at_css('th:contains("Row")')['style']).to eq 'text-align: right' - expect(doc.at_css('th:contains("Example")')['style']).to eq 'text-align: left' - end + aggregate_failures 'permits style attribute in th elements' do + expect(doc.at_css('th:contains("Header")')['style']).to eq 'text-align: center' + expect(doc.at_css('th:contains("Row")')['style']).to eq 'text-align: right' + expect(doc.at_css('th:contains("Example")')['style']).to eq 'text-align: left' end - it 'permits style attribute in td elements' do - aggregate_failures do - expect(doc.at_css('td:contains("Foo")')['style']).to eq 'text-align: center' - expect(doc.at_css('td:contains("Bar")')['style']).to eq 'text-align: right' - expect(doc.at_css('td:contains("Baz")')['style']).to eq 'text-align: left' - end + aggregate_failures 'permits style attribute in td elements' do + expect(doc.at_css('td:contains("Foo")')['style']).to eq 'text-align: center' + expect(doc.at_css('td:contains("Bar")')['style']).to eq 'text-align: right' + expect(doc.at_css('td:contains("Baz")')['style']).to eq 'text-align: left' end - it 'removes `rel` attribute from links' do + aggregate_failures 'removes `rel` attribute from links' do expect(doc).not_to have_selector('a[rel="bookmark"]') end - it "removes `href` from `a` elements if it's fishy" do + aggregate_failures "removes `href` from `a` elements if it's fishy" do expect(doc).not_to have_selector('a[href*="javascript"]') end end @@ -176,26 +166,26 @@ describe 'GitLab Markdown' do end end - describe 'ExternalLinkFilter' do - it 'adds nofollow to external link' do + it 'includes ExternalLinkFilter' do + aggregate_failures 'adds nofollow to external link' do link = doc.at_css('a:contains("Google")') expect(link.attr('rel')).to include('nofollow') end - it 'adds noreferrer to external link' do + aggregate_failures 'adds noreferrer to external link' do link = doc.at_css('a:contains("Google")') expect(link.attr('rel')).to include('noreferrer') end - it 'adds _blank to target attribute for external links' do + aggregate_failures 'adds _blank to target attribute for external links' do link = doc.at_css('a:contains("Google")') expect(link.attr('target')).to match('_blank') end - it 'ignores internal link' do + aggregate_failures 'ignores internal link' do link = doc.at_css('a:contains("GitLab Root")') expect(link.attr('rel')).not_to match 'nofollow' @@ -219,24 +209,24 @@ describe 'GitLab Markdown' do it_behaves_like 'all pipelines' - it 'includes RelativeLinkFilter' do - expect(doc).to parse_relative_links - end + it 'includes custom filters' do + aggregate_failures 'RelativeLinkFilter' do + expect(doc).to parse_relative_links + end - it 'includes EmojiFilter' do - expect(doc).to parse_emoji - end + aggregate_failures 'EmojiFilter' do + expect(doc).to parse_emoji + end - it 'includes TableOfContentsFilter' do - expect(doc).to create_header_links - end + aggregate_failures 'TableOfContentsFilter' do + expect(doc).to create_header_links + end - it 'includes AutolinkFilter' do - expect(doc).to create_autolinks - end + aggregate_failures 'AutolinkFilter' do + expect(doc).to create_autolinks + end - it 'includes all reference filters' do - aggregate_failures do + aggregate_failures 'all reference filters' do expect(doc).to reference_users expect(doc).to reference_issues expect(doc).to reference_merge_requests @@ -246,22 +236,22 @@ describe 'GitLab Markdown' do expect(doc).to reference_labels expect(doc).to reference_milestones end - end - it 'includes TaskListFilter' do - expect(doc).to parse_task_lists - end + aggregate_failures 'TaskListFilter' do + expect(doc).to parse_task_lists + end - it 'includes InlineDiffFilter' do - expect(doc).to parse_inline_diffs - end + aggregate_failures 'InlineDiffFilter' do + expect(doc).to parse_inline_diffs + end - it 'includes VideoLinkFilter' do - expect(doc).to parse_video_links - end + aggregate_failures 'VideoLinkFilter' do + expect(doc).to parse_video_links + end - it 'includes ColorFilter' do - expect(doc).to parse_colors + aggregate_failures 'ColorFilter' do + expect(doc).to parse_colors + end end end @@ -280,24 +270,24 @@ describe 'GitLab Markdown' do it_behaves_like 'all pipelines' - it 'includes RelativeLinkFilter' do - expect(doc).not_to parse_relative_links - end + it 'includes custom filters' do + aggregate_failures 'RelativeLinkFilter' do + expect(doc).not_to parse_relative_links + end - it 'includes EmojiFilter' do - expect(doc).to parse_emoji - end + aggregate_failures 'EmojiFilter' do + expect(doc).to parse_emoji + end - it 'includes TableOfContentsFilter' do - expect(doc).to create_header_links - end + aggregate_failures 'TableOfContentsFilter' do + expect(doc).to create_header_links + end - it 'includes AutolinkFilter' do - expect(doc).to create_autolinks - end + aggregate_failures 'AutolinkFilter' do + expect(doc).to create_autolinks + end - it 'includes all reference filters' do - aggregate_failures do + aggregate_failures 'all reference filters' do expect(doc).to reference_users expect(doc).to reference_issues expect(doc).to reference_merge_requests @@ -307,26 +297,26 @@ describe 'GitLab Markdown' do expect(doc).to reference_labels expect(doc).to reference_milestones end - end - it 'includes TaskListFilter' do - expect(doc).to parse_task_lists - end + aggregate_failures 'TaskListFilter' do + expect(doc).to parse_task_lists + end - it 'includes GollumTagsFilter' do - expect(doc).to parse_gollum_tags - end + aggregate_failures 'GollumTagsFilter' do + expect(doc).to parse_gollum_tags + end - it 'includes InlineDiffFilter' do - expect(doc).to parse_inline_diffs - end + aggregate_failures 'InlineDiffFilter' do + expect(doc).to parse_inline_diffs + end - it 'includes VideoLinkFilter' do - expect(doc).to parse_video_links - end + aggregate_failures 'VideoLinkFilter' do + expect(doc).to parse_video_links + end - it 'includes ColorFilter' do - expect(doc).to parse_colors + aggregate_failures 'ColorFilter' do + expect(doc).to parse_colors + end end end diff --git a/spec/features/merge_request/maintainer_edits_fork_spec.rb b/spec/features/merge_request/maintainer_edits_fork_spec.rb index a3323da1b1f..1808d0c0a0c 100644 --- a/spec/features/merge_request/maintainer_edits_fork_spec.rb +++ b/spec/features/merge_request/maintainer_edits_fork_spec.rb @@ -14,7 +14,7 @@ describe 'a maintainer edits files on a source-branch of an MR from a fork', :js source_branch: 'fix', target_branch: 'master', author: author, - allow_maintainer_to_push: true) + allow_collaboration: true) end before do diff --git a/spec/features/merge_request/user_allows_a_maintainer_to_push_spec.rb b/spec/features/merge_request/user_allows_commits_from_memebers_who_can_merge_spec.rb index eb41d7de8ed..0af37d76539 100644 --- a/spec/features/merge_request/user_allows_a_maintainer_to_push_spec.rb +++ b/spec/features/merge_request/user_allows_commits_from_memebers_who_can_merge_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper' -describe 'create a merge request that allows maintainers to push', :js do +describe 'create a merge request, allowing commits from members who can merge to the target branch', :js do include ProjectForksHelper let(:user) { create(:user) } let(:target_project) { create(:project, :public, :repository) } @@ -21,16 +21,16 @@ describe 'create a merge request that allows maintainers to push', :js do sign_in(user) end - it 'allows setting maintainer push possible' do + it 'allows setting possible' do visit_new_merge_request - check 'Allow edits from maintainers' + check 'Allow commits from members who can merge to the target branch' click_button 'Submit merge request' wait_for_requests - expect(page).to have_content('Allows edits from maintainers') + expect(page).to have_content('Allows commits from members who can merge to the target branch') end it 'shows a message when one of the projects is private' do @@ -57,12 +57,12 @@ describe 'create a merge request that allows maintainers to push', :js do visit_new_merge_request - expect(page).not_to have_content('Allows edits from maintainers') + expect(page).not_to have_content('Allows commits from members who can merge to the target branch') end end - context 'when a maintainer tries to edit the option' do - let(:maintainer) { create(:user) } + context 'when a member who can merge tries to edit the option' do + let(:member) { create(:user) } let(:merge_request) do create(:merge_request, source_project: source_project, @@ -71,15 +71,15 @@ describe 'create a merge request that allows maintainers to push', :js do end before do - target_project.add_master(maintainer) + target_project.add_master(member) - sign_in(maintainer) + sign_in(member) end - it 'it hides the option from maintainers' do + it 'it hides the option from members' do visit edit_project_merge_request_path(target_project, merge_request) - expect(page).not_to have_content('Allows edits from maintainers') + expect(page).not_to have_content('Allows commits from members who can merge to the target branch') end end end diff --git a/spec/features/projects/import_export/import_file_spec.rb b/spec/features/projects/import_export/import_file_spec.rb index 60fe30bd898..d0912e645bc 100644 --- a/spec/features/projects/import_export/import_file_spec.rb +++ b/spec/features/projects/import_export/import_file_spec.rb @@ -87,11 +87,13 @@ feature 'Import/Export - project import integration test', :js do def wiki_exists?(project) wiki = ProjectWiki.new(project) - File.exist?(wiki.repository.path_to_repo) && !wiki.repository.empty? + wiki.repository.exists? && !wiki.repository.empty? end def project_hook_exists?(project) - Gitlab::Git::Hook.new('post-receive', project.repository.raw_repository).exists? + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + Gitlab::Git::Hook.new('post-receive', project.repository.raw_repository).exists? + end end def click_import_project_tab diff --git a/spec/features/projects/jobs/user_browses_job_spec.rb b/spec/features/projects/jobs/user_browses_job_spec.rb index bff5bbe99af..ce0b38b7239 100644 --- a/spec/features/projects/jobs/user_browses_job_spec.rb +++ b/spec/features/projects/jobs/user_browses_job_spec.rb @@ -32,8 +32,6 @@ describe 'User browses a job', :js do page.within('.erased') do expect(page).to have_content('Job has been erased') end - - expect(build.project.running_or_pending_build_count).to eq(build.project.builds.running_or_pending.count(:all)) end context 'with a failed job' do diff --git a/spec/features/projects/settings/user_manages_group_links_spec.rb b/spec/features/projects/settings/user_manages_group_links_spec.rb index fdf42797091..92ce2ca83c7 100644 --- a/spec/features/projects/settings/user_manages_group_links_spec.rb +++ b/spec/features/projects/settings/user_manages_group_links_spec.rb @@ -30,7 +30,7 @@ describe 'Projects > Settings > User manages group links' do click_link('Share with group') select2(group_market.id, from: '#link_group_id') - select('Master', from: 'link_group_access') + select('Maintainer', from: 'link_group_access') click_button('Share') diff --git a/spec/features/projects/settings/user_manages_project_members_spec.rb b/spec/features/projects/settings/user_manages_project_members_spec.rb index 8af95522165..d3003753ae6 100644 --- a/spec/features/projects/settings/user_manages_project_members_spec.rb +++ b/spec/features/projects/settings/user_manages_project_members_spec.rb @@ -62,7 +62,7 @@ describe 'Projects > Settings > User manages project members' do page.within('.project-members-groups') do expect(page).to have_content('OpenSource') - expect(first('.group_member')).to have_content('Master') + expect(first('.group_member')).to have_content('Maintainer') end end end diff --git a/spec/features/protected_branches_spec.rb b/spec/features/protected_branches_spec.rb index 0c28a853b54..4c0f9971425 100644 --- a/spec/features/protected_branches_spec.rb +++ b/spec/features/protected_branches_spec.rb @@ -72,7 +72,7 @@ feature 'Protected Branches', :js do click_link 'No one' find(".js-allowed-to-push").click wait_for_requests - click_link 'Developers + Masters' + click_link 'Developers + Maintainers' end visit project_protected_branches_path(project) @@ -82,7 +82,7 @@ feature 'Protected Branches', :js do expect(page.find(".dropdown-toggle-text")).to have_content("No one") end page.within(".js-allowed-to-push") do - expect(page.find(".dropdown-toggle-text")).to have_content("Developers + Masters") + expect(page.find(".dropdown-toggle-text")).to have_content("Developers + Maintainers") end end end diff --git a/spec/features/runners_spec.rb b/spec/features/runners_spec.rb index 9ce7d538004..fe0b03a7e00 100644 --- a/spec/features/runners_spec.rb +++ b/spec/features/runners_spec.rb @@ -184,7 +184,7 @@ feature 'Runners' do given(:group) { create :group } - context 'as project and group master' do + context 'as project and group maintainer' do background do group.add_master(user) end @@ -197,13 +197,13 @@ feature 'Runners' do expect(page).to have_content 'This group does not provide any group Runners yet' - expect(page).to have_content 'Group masters can register group runners in the Group CI/CD settings' - expect(page).not_to have_content 'Ask your group master to setup a group Runner' + expect(page).to have_content 'Group maintainers can register group runners in the Group CI/CD settings' + expect(page).not_to have_content 'Ask your group maintainer to setup a group Runner' end end end - context 'as project master' do + context 'as project maintainer' do context 'project without a group' do given(:project) { create :project } @@ -223,8 +223,8 @@ feature 'Runners' do expect(page).to have_content 'This group does not provide any group Runners yet.' - expect(page).not_to have_content 'Group masters can register group runners in the Group CI/CD settings' - expect(page).to have_content 'Ask your group master to setup a group Runner.' + expect(page).not_to have_content 'Group maintainers can register group runners in the Group CI/CD settings' + expect(page).to have_content 'Ask your group maintainer to setup a group Runner.' end end diff --git a/spec/features/users/terms_spec.rb b/spec/features/users/terms_spec.rb index 1efa5cd5490..af407c52917 100644 --- a/spec/features/users/terms_spec.rb +++ b/spec/features/users/terms_spec.rb @@ -39,6 +39,22 @@ describe 'Users > Terms' do end end + context 'when the user has already accepted the terms' do + before do + accept_terms(user) + end + + it 'allows the user to continue to the app' do + visit terms_path + + expect(page).to have_content "You have already accepted the Terms of Service as #{user.to_reference}" + + click_link 'Continue' + + expect(current_path).to eq(root_path) + end + end + context 'terms were enforced while session is active', :js do let(:project) { create(:project) } diff --git a/spec/fixtures/api/schemas/entities/merge_request_basic.json b/spec/fixtures/api/schemas/entities/merge_request_basic.json index 46031961cca..f7bc137c90c 100644 --- a/spec/fixtures/api/schemas/entities/merge_request_basic.json +++ b/spec/fixtures/api/schemas/entities/merge_request_basic.json @@ -13,6 +13,7 @@ "assignee_id": { "type": ["integer", "null"] }, "subscribed": { "type": ["boolean", "null"] }, "participants": { "type": "array" }, + "allow_collaboration": { "type": "boolean"}, "allow_maintainer_to_push": { "type": "boolean"} }, "additionalProperties": false diff --git a/spec/fixtures/api/schemas/entities/merge_request_widget.json b/spec/fixtures/api/schemas/entities/merge_request_widget.json index 7be8c9e3e67..ee5588fa6c6 100644 --- a/spec/fixtures/api/schemas/entities/merge_request_widget.json +++ b/spec/fixtures/api/schemas/entities/merge_request_widget.json @@ -31,7 +31,7 @@ "source_project_id": { "type": "integer" }, "target_branch": { "type": "string" }, "target_project_id": { "type": "integer" }, - "allow_maintainer_to_push": { "type": "boolean"}, + "allow_collaboration": { "type": "boolean"}, "metrics": { "oneOf": [ { "type": "null" }, diff --git a/spec/fixtures/api/schemas/public_api/v4/merge_requests.json b/spec/fixtures/api/schemas/public_api/v4/merge_requests.json index f97461ce9cc..f7adc4e0b91 100644 --- a/spec/fixtures/api/schemas/public_api/v4/merge_requests.json +++ b/spec/fixtures/api/schemas/public_api/v4/merge_requests.json @@ -82,6 +82,7 @@ "human_time_estimate": { "type": ["string", "null"] }, "human_total_time_spent": { "type": ["string", "null"] } }, + "allow_collaboration": { "type": ["boolean", "null"] }, "allow_maintainer_to_push": { "type": ["boolean", "null"] } }, "required": [ diff --git a/spec/graphql/gitlab_schema_spec.rb b/spec/graphql/gitlab_schema_spec.rb new file mode 100644 index 00000000000..b892f6b44ed --- /dev/null +++ b/spec/graphql/gitlab_schema_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe GitlabSchema do + it 'uses batch loading' do + expect(field_instrumenters).to include(BatchLoader::GraphQL) + end + + it 'enables the preload instrumenter' do + expect(field_instrumenters).to include(BatchLoader::GraphQL) + end + + it 'enables the authorization instrumenter' do + expect(field_instrumenters).to include(instance_of(::Gitlab::Graphql::Authorize::Instrumentation)) + end + + it 'enables using presenters' do + expect(field_instrumenters).to include(instance_of(::Gitlab::Graphql::Present::Instrumentation)) + end + + it 'has the base mutation' do + pending('Adding an empty mutation breaks the documentation explorer') + + expect(described_class.mutation).to eq(::Types::MutationType.to_graphql) + end + + it 'has the base query' do + expect(described_class.query).to eq(::Types::QueryType.to_graphql) + end + + def field_instrumenters + described_class.instrumenters[:field] + end +end diff --git a/spec/graphql/resolvers/merge_request_resolver_spec.rb b/spec/graphql/resolvers/merge_request_resolver_spec.rb new file mode 100644 index 00000000000..af015533209 --- /dev/null +++ b/spec/graphql/resolvers/merge_request_resolver_spec.rb @@ -0,0 +1,58 @@ +require 'spec_helper' + +describe Resolvers::MergeRequestResolver do + include GraphqlHelpers + + set(:project) { create(:project, :repository) } + set(:merge_request_1) { create(:merge_request, :simple, source_project: project, target_project: project) } + set(:merge_request_2) { create(:merge_request, :rebased, source_project: project, target_project: project) } + + set(:other_project) { create(:project, :repository) } + set(:other_merge_request) { create(:merge_request, source_project: other_project, target_project: other_project) } + + let(:full_path) { project.full_path } + let(:iid_1) { merge_request_1.iid } + let(:iid_2) { merge_request_2.iid } + + let(:other_full_path) { other_project.full_path } + let(:other_iid) { other_merge_request.iid } + + describe '#resolve' do + it 'batch-resolves merge requests by target project full path and IID' do + path = full_path # avoid database query + + result = batch(max_queries: 2) do + [resolve_mr(path, iid_1), resolve_mr(path, iid_2)] + end + + expect(result).to contain_exactly(merge_request_1, merge_request_2) + end + + it 'can batch-resolve merge requests from different projects' do + path = project.full_path # avoid database queries + other_path = other_full_path + + result = batch(max_queries: 3) do + [resolve_mr(path, iid_1), resolve_mr(path, iid_2), resolve_mr(other_path, other_iid)] + end + + expect(result).to contain_exactly(merge_request_1, merge_request_2, other_merge_request) + end + + it 'resolves an unknown iid to nil' do + result = batch { resolve_mr(full_path, -1) } + + expect(result).to be_nil + end + + it 'resolves a known iid for an unknown full_path to nil' do + result = batch { resolve_mr('unknown/project', iid_1) } + + expect(result).to be_nil + end + end + + def resolve_mr(full_path, iid) + resolve(described_class, args: { full_path: full_path, iid: iid }) + end +end diff --git a/spec/graphql/resolvers/project_resolver_spec.rb b/spec/graphql/resolvers/project_resolver_spec.rb new file mode 100644 index 00000000000..d4990c6492c --- /dev/null +++ b/spec/graphql/resolvers/project_resolver_spec.rb @@ -0,0 +1,32 @@ +require 'spec_helper' + +describe Resolvers::ProjectResolver do + include GraphqlHelpers + + set(:project1) { create(:project) } + set(:project2) { create(:project) } + + set(:other_project) { create(:project) } + + describe '#resolve' do + it 'batch-resolves projects by full path' do + paths = [project1.full_path, project2.full_path] + + result = batch(max_queries: 1) do + paths.map { |path| resolve_project(path) } + end + + expect(result).to contain_exactly(project1, project2) + end + + it 'resolves an unknown full_path to nil' do + result = batch { resolve_project('unknown/project') } + + expect(result).to be_nil + end + end + + def resolve_project(full_path) + resolve(described_class, args: { full_path: full_path }) + end +end diff --git a/spec/graphql/types/project_type_spec.rb b/spec/graphql/types/project_type_spec.rb new file mode 100644 index 00000000000..e0f89105b86 --- /dev/null +++ b/spec/graphql/types/project_type_spec.rb @@ -0,0 +1,5 @@ +require 'spec_helper' + +describe GitlabSchema.types['Project'] do + it { expect(described_class.graphql_name).to eq('Project') } +end diff --git a/spec/graphql/types/query_type_spec.rb b/spec/graphql/types/query_type_spec.rb new file mode 100644 index 00000000000..8488252fd59 --- /dev/null +++ b/spec/graphql/types/query_type_spec.rb @@ -0,0 +1,37 @@ +require 'spec_helper' + +describe GitlabSchema.types['Query'] do + it 'is called Query' do + expect(described_class.graphql_name).to eq('Query') + end + + it { is_expected.to have_graphql_fields(:project, :merge_request, :echo) } + + describe 'project field' do + subject { described_class.fields['project'] } + + it 'finds projects by full path' do + is_expected.to have_graphql_arguments(:full_path) + is_expected.to have_graphql_type(Types::ProjectType) + is_expected.to have_graphql_resolver(Resolvers::ProjectResolver) + end + + it 'authorizes with read_project' do + is_expected.to require_graphql_authorizations(:read_project) + end + end + + describe 'merge_request field' do + subject { described_class.fields['mergeRequest'] } + + it 'finds MRs by project and IID' do + is_expected.to have_graphql_arguments(:full_path, :iid) + is_expected.to have_graphql_type(Types::MergeRequestType) + is_expected.to have_graphql_resolver(Resolvers::MergeRequestResolver) + end + + it 'authorizes with read_merge_request' do + is_expected.to require_graphql_authorizations(:read_merge_request) + end + end +end diff --git a/spec/graphql/types/time_type_spec.rb b/spec/graphql/types/time_type_spec.rb new file mode 100644 index 00000000000..4196d9d27d4 --- /dev/null +++ b/spec/graphql/types/time_type_spec.rb @@ -0,0 +1,16 @@ +require 'spec_helper' + +describe GitlabSchema.types['Time'] do + let(:iso) { "2018-06-04T15:23:50+02:00" } + let(:time) { Time.parse(iso) } + + it { expect(described_class.graphql_name).to eq('Time') } + + it 'coerces Time object into ISO 8601' do + expect(described_class.coerce_isolated_result(time)).to eq(iso) + end + + it 'coerces an ISO-time into Time object' do + expect(described_class.coerce_isolated_input(iso)).to eq(time) + end +end diff --git a/spec/helpers/preferences_helper_spec.rb b/spec/helpers/preferences_helper_spec.rb index c9d2ec8a4ae..9940656fb68 100644 --- a/spec/helpers/preferences_helper_spec.rb +++ b/spec/helpers/preferences_helper_spec.rb @@ -31,9 +31,9 @@ describe PreferencesHelper do describe '#user_application_theme' do context 'with a user' do it "returns user's theme's css_class" do - stub_user(theme_id: 3) + stub_user(theme_id: 10) - expect(helper.user_application_theme).to eq 'ui_light' + expect(helper.user_application_theme).to eq 'ui-light' end it 'returns the default when id is invalid' do @@ -41,7 +41,7 @@ describe PreferencesHelper do allow(Gitlab.config.gitlab).to receive(:default_theme).and_return(1) - expect(helper.user_application_theme).to eq 'ui_indigo' + expect(helper.user_application_theme).to eq 'ui-indigo' end end diff --git a/spec/initializers/artifacts_direct_upload_support_spec.rb b/spec/initializers/artifacts_direct_upload_support_spec.rb deleted file mode 100644 index bfb71da3388..00000000000 --- a/spec/initializers/artifacts_direct_upload_support_spec.rb +++ /dev/null @@ -1,71 +0,0 @@ -require 'spec_helper' - -describe 'Artifacts direct upload support' do - subject do - load Rails.root.join('config/initializers/artifacts_direct_upload_support.rb') - end - - let(:connection) do - { provider: provider } - end - - before do - stub_artifacts_setting( - object_store: { - enabled: enabled, - direct_upload: direct_upload, - connection: connection - }) - end - - context 'when object storage is enabled' do - let(:enabled) { true } - - context 'when direct upload is enabled' do - let(:direct_upload) { true } - - context 'when provider is Google' do - let(:provider) { 'Google' } - - it 'succeeds' do - expect { subject }.not_to raise_error - end - end - - context 'when connection is empty' do - let(:connection) { nil } - - it 'raises an error' do - expect { subject }.to raise_error /object storage provider when 'direct_upload' of artifacts is used/ - end - end - - context 'when other provider is used' do - let(:provider) { 'AWS' } - - it 'raises an error' do - expect { subject }.to raise_error /object storage provider when 'direct_upload' of artifacts is used/ - end - end - end - - context 'when direct upload is disabled' do - let(:direct_upload) { false } - let(:provider) { 'AWS' } - - it 'succeeds' do - expect { subject }.not_to raise_error - end - end - end - - context 'when object storage is disabled' do - let(:enabled) { false } - let(:direct_upload) { false } - let(:provider) { 'AWS' } - - it 'succeeds' do - expect { subject }.not_to raise_error - end - end -end diff --git a/spec/initializers/direct_upload_support_spec.rb b/spec/initializers/direct_upload_support_spec.rb new file mode 100644 index 00000000000..e51d404e030 --- /dev/null +++ b/spec/initializers/direct_upload_support_spec.rb @@ -0,0 +1,90 @@ +require 'spec_helper' + +describe 'Direct upload support' do + subject do + load Rails.root.join('config/initializers/direct_upload_support.rb') + end + + where(:config_name) do + %w(lfs artifacts uploads) + end + + with_them do + let(:connection) do + { provider: provider } + end + + let(:object_store) do + { + enabled: enabled, + direct_upload: direct_upload, + connection: connection + } + end + + before do + allow(Gitlab.config).to receive_messages(to_settings(config_name => { + object_store: object_store + })) + end + + context 'when object storage is enabled' do + let(:enabled) { true } + + context 'when direct upload is enabled' do + let(:direct_upload) { true } + + context 'when provider is AWS' do + let(:provider) { 'AWS' } + + it 'succeeds' do + expect { subject }.not_to raise_error + end + end + + context 'when provider is Google' do + let(:provider) { 'Google' } + + it 'succeeds' do + expect { subject }.not_to raise_error + end + end + + context 'when connection is empty' do + let(:connection) { nil } + + it 'raises an error' do + expect { subject }.to raise_error /are supported as a object storage provider when 'direct_upload' is used/ + end + end + + context 'when other provider is used' do + let(:provider) { 'Rackspace' } + + it 'raises an error' do + expect { subject }.to raise_error /are supported as a object storage provider when 'direct_upload' is used/ + end + end + end + + context 'when direct upload is disabled' do + let(:direct_upload) { false } + let(:provider) { 'AWS' } + + it 'succeeds' do + expect { subject }.not_to raise_error + end + end + end + + context 'when object storage is disabled' do + let(:enabled) { false } + let(:direct_upload) { false } + let(:provider) { 'Rackspace' } + + it 'succeeds' do + expect { subject }.not_to raise_error + end + end + end +end diff --git a/spec/initializers/fog_google_https_private_urls_spec.rb b/spec/initializers/fog_google_https_private_urls_spec.rb index de3c157ab7b..08346b71fee 100644 --- a/spec/initializers/fog_google_https_private_urls_spec.rb +++ b/spec/initializers/fog_google_https_private_urls_spec.rb @@ -1,13 +1,13 @@ require 'spec_helper' -describe 'Fog::Storage::GoogleXML::File' do +describe 'Fog::Storage::GoogleXML::File', :fog_requests do let(:storage) do Fog.mock! - Fog::Storage.new({ - google_storage_access_key_id: "asdf", - google_storage_secret_access_key: "asdf", - provider: "Google" - }) + Fog::Storage.new( + google_storage_access_key_id: "asdf", + google_storage_secret_access_key: "asdf", + provider: "Google" + ) end let(:file) do diff --git a/spec/javascripts/ide/components/jobs/detail/description_spec.js b/spec/javascripts/ide/components/jobs/detail/description_spec.js new file mode 100644 index 00000000000..9b715a41499 --- /dev/null +++ b/spec/javascripts/ide/components/jobs/detail/description_spec.js @@ -0,0 +1,28 @@ +import Vue from 'vue'; +import Description from '~/ide/components/jobs/detail/description.vue'; +import mountComponent from '../../../../helpers/vue_mount_component_helper'; +import { jobs } from '../../../mock_data'; + +describe('IDE job description', () => { + const Component = Vue.extend(Description); + let vm; + + beforeEach(() => { + vm = mountComponent(Component, { + job: jobs[0], + }); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('renders job details', () => { + expect(vm.$el.textContent).toContain('#1'); + expect(vm.$el.textContent).toContain('test'); + }); + + it('renders CI icon', () => { + expect(vm.$el.querySelector('.ci-status-icon .ic-status_passed_borderless')).not.toBe(null); + }); +}); diff --git a/spec/javascripts/ide/components/jobs/detail/scroll_button_spec.js b/spec/javascripts/ide/components/jobs/detail/scroll_button_spec.js new file mode 100644 index 00000000000..fff382a107f --- /dev/null +++ b/spec/javascripts/ide/components/jobs/detail/scroll_button_spec.js @@ -0,0 +1,59 @@ +import Vue from 'vue'; +import ScrollButton from '~/ide/components/jobs/detail/scroll_button.vue'; +import mountComponent from '../../../../helpers/vue_mount_component_helper'; + +describe('IDE job log scroll button', () => { + const Component = Vue.extend(ScrollButton); + let vm; + + beforeEach(() => { + vm = mountComponent(Component, { + direction: 'up', + disabled: false, + }); + }); + + afterEach(() => { + vm.$destroy(); + }); + + describe('iconName', () => { + ['up', 'down'].forEach(direction => { + it(`returns icon name for ${direction}`, () => { + vm.direction = direction; + + expect(vm.iconName).toBe(`scroll_${direction}`); + }); + }); + }); + + describe('tooltipTitle', () => { + it('returns title for up', () => { + expect(vm.tooltipTitle).toBe('Scroll to top'); + }); + + it('returns title for down', () => { + vm.direction = 'down'; + + expect(vm.tooltipTitle).toBe('Scroll to bottom'); + }); + }); + + it('emits click event on click', () => { + spyOn(vm, '$emit'); + + vm.$el.querySelector('.btn-scroll').click(); + + expect(vm.$emit).toHaveBeenCalledWith('click'); + }); + + it('disables button when disabled is true', done => { + vm.disabled = true; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.btn-scroll').hasAttribute('disabled')).toBe(true); + + done(); + }); + }); +}); diff --git a/spec/javascripts/ide/components/jobs/detail_spec.js b/spec/javascripts/ide/components/jobs/detail_spec.js new file mode 100644 index 00000000000..641ba06f653 --- /dev/null +++ b/spec/javascripts/ide/components/jobs/detail_spec.js @@ -0,0 +1,180 @@ +import Vue from 'vue'; +import JobDetail from '~/ide/components/jobs/detail.vue'; +import { createStore } from '~/ide/stores'; +import { createComponentWithStore } from '../../../helpers/vue_mount_component_helper'; +import { jobs } from '../../mock_data'; + +describe('IDE jobs detail view', () => { + const Component = Vue.extend(JobDetail); + let vm; + + beforeEach(() => { + const store = createStore(); + + store.state.pipelines.detailJob = { + ...jobs[0], + isLoading: true, + output: 'testing', + rawPath: `${gl.TEST_HOST}/raw`, + }; + + vm = createComponentWithStore(Component, store); + + spyOn(vm, 'fetchJobTrace').and.returnValue(Promise.resolve()); + + vm = vm.$mount(); + + spyOn(vm.$refs.buildTrace, 'scrollTo'); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('calls fetchJobTrace on mount', () => { + expect(vm.fetchJobTrace).toHaveBeenCalled(); + }); + + it('scrolls to bottom on mount', done => { + setTimeout(() => { + expect(vm.$refs.buildTrace.scrollTo).toHaveBeenCalled(); + + done(); + }); + }); + + it('renders job output', () => { + expect(vm.$el.querySelector('.bash').textContent).toContain('testing'); + }); + + it('renders empty message output', done => { + vm.$store.state.pipelines.detailJob.output = ''; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.bash').textContent).toContain('No messages were logged'); + + done(); + }); + }); + + it('renders loading icon', () => { + expect(vm.$el.querySelector('.build-loader-animation')).not.toBe(null); + expect(vm.$el.querySelector('.build-loader-animation').style.display).toBe(''); + }); + + it('hide loading icon when isLoading is false', done => { + vm.$store.state.pipelines.detailJob.isLoading = false; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.build-loader-animation').style.display).toBe('none'); + + done(); + }); + }); + + it('resets detailJob when clicking header button', () => { + spyOn(vm, 'setDetailJob'); + + vm.$el.querySelector('.btn').click(); + + expect(vm.setDetailJob).toHaveBeenCalledWith(null); + }); + + it('renders raw path link', () => { + expect(vm.$el.querySelector('.controllers-buttons').getAttribute('href')).toBe( + `${gl.TEST_HOST}/raw`, + ); + }); + + describe('scroll buttons', () => { + it('triggers scrollDown when clicking down button', done => { + spyOn(vm, 'scrollDown'); + + vm.$el.querySelectorAll('.btn-scroll')[1].click(); + + vm.$nextTick(() => { + expect(vm.scrollDown).toHaveBeenCalled(); + + done(); + }); + }); + + it('triggers scrollUp when clicking up button', done => { + spyOn(vm, 'scrollUp'); + + vm.scrollPos = 1; + + vm + .$nextTick() + .then(() => vm.$el.querySelector('.btn-scroll').click()) + .then(() => vm.$nextTick()) + .then(() => { + expect(vm.scrollUp).toHaveBeenCalled(); + }) + .then(done) + .catch(done.fail); + }); + }); + + describe('scrollDown', () => { + it('scrolls build trace to bottom', () => { + spyOnProperty(vm.$refs.buildTrace, 'scrollHeight').and.returnValue(1000); + + vm.scrollDown(); + + expect(vm.$refs.buildTrace.scrollTo).toHaveBeenCalledWith(0, 1000); + }); + }); + + describe('scrollUp', () => { + it('scrolls build trace to top', () => { + vm.scrollUp(); + + expect(vm.$refs.buildTrace.scrollTo).toHaveBeenCalledWith(0, 0); + }); + }); + + describe('scrollBuildLog', () => { + beforeEach(() => { + spyOnProperty(vm.$refs.buildTrace, 'offsetHeight').and.returnValue(100); + spyOnProperty(vm.$refs.buildTrace, 'scrollHeight').and.returnValue(200); + }); + + it('sets scrollPos to bottom when at the bottom', done => { + spyOnProperty(vm.$refs.buildTrace, 'scrollTop').and.returnValue(100); + + vm.scrollBuildLog(); + + setTimeout(() => { + expect(vm.scrollPos).toBe(1); + + done(); + }); + }); + + it('sets scrollPos to top when at the top', done => { + spyOnProperty(vm.$refs.buildTrace, 'scrollTop').and.returnValue(0); + vm.scrollPos = 1; + + vm.scrollBuildLog(); + + setTimeout(() => { + expect(vm.scrollPos).toBe(0); + + done(); + }); + }); + + it('resets scrollPos when not at top or bottom', done => { + spyOnProperty(vm.$refs.buildTrace, 'scrollTop').and.returnValue(10); + + vm.scrollBuildLog(); + + setTimeout(() => { + expect(vm.scrollPos).toBe(''); + + done(); + }); + }); + }); +}); diff --git a/spec/javascripts/ide/components/jobs/item_spec.js b/spec/javascripts/ide/components/jobs/item_spec.js index 7c1dd4e475c..79e07f00e7b 100644 --- a/spec/javascripts/ide/components/jobs/item_spec.js +++ b/spec/javascripts/ide/components/jobs/item_spec.js @@ -26,4 +26,14 @@ describe('IDE jobs item', () => { it('renders CI icon', () => { expect(vm.$el.querySelector('.ic-status_passed_borderless')).not.toBe(null); }); + + it('does not render view logs button if not started', done => { + vm.job.started = false; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.btn')).toBe(null); + + done(); + }); + }); }); diff --git a/spec/javascripts/ide/components/merge_requests/dropdown_spec.js b/spec/javascripts/ide/components/merge_requests/dropdown_spec.js new file mode 100644 index 00000000000..74884c9a362 --- /dev/null +++ b/spec/javascripts/ide/components/merge_requests/dropdown_spec.js @@ -0,0 +1,47 @@ +import Vue from 'vue'; +import { createStore } from '~/ide/stores'; +import Dropdown from '~/ide/components/merge_requests/dropdown.vue'; +import { createComponentWithStore } from '../../../helpers/vue_mount_component_helper'; +import { mergeRequests } from '../../mock_data'; + +describe('IDE merge requests dropdown', () => { + const Component = Vue.extend(Dropdown); + let vm; + + beforeEach(() => { + const store = createStore(); + + vm = createComponentWithStore(Component, store, { show: false }).$mount(); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('does not render tabs when show is false', () => { + expect(vm.$el.querySelector('.nav-links')).toBe(null); + }); + + describe('when show is true', () => { + beforeEach(done => { + vm.show = true; + vm.$store.state.mergeRequests.assigned.mergeRequests.push(mergeRequests[0]); + + vm.$nextTick(done); + }); + + it('renders tabs', () => { + expect(vm.$el.querySelector('.nav-links')).not.toBe(null); + }); + + it('renders count for assigned & created data', () => { + expect(vm.$el.querySelector('.nav-links a').textContent).toContain('Created by me'); + expect(vm.$el.querySelector('.nav-links a .badge').textContent).toContain('0'); + + expect(vm.$el.querySelectorAll('.nav-links a')[1].textContent).toContain('Assigned to me'); + expect( + vm.$el.querySelectorAll('.nav-links a')[1].querySelector('.badge').textContent, + ).toContain('1'); + }); + }); +}); diff --git a/spec/javascripts/ide/components/merge_requests/item_spec.js b/spec/javascripts/ide/components/merge_requests/item_spec.js new file mode 100644 index 00000000000..51c4cddef2f --- /dev/null +++ b/spec/javascripts/ide/components/merge_requests/item_spec.js @@ -0,0 +1,61 @@ +import Vue from 'vue'; +import Item from '~/ide/components/merge_requests/item.vue'; +import mountCompontent from '../../../helpers/vue_mount_component_helper'; + +describe('IDE merge request item', () => { + const Component = Vue.extend(Item); + let vm; + + beforeEach(() => { + vm = mountCompontent(Component, { + item: { + iid: 1, + projectPathWithNamespace: 'gitlab-org/gitlab-ce', + title: 'Merge request title', + }, + currentId: '1', + currentProjectId: 'gitlab-org/gitlab-ce', + }); + }); + + afterEach(() => { + vm.$destroy(); + }); + + it('renders merge requests data', () => { + expect(vm.$el.textContent).toContain('Merge request title'); + expect(vm.$el.textContent).toContain('gitlab-org/gitlab-ce!1'); + }); + + it('renders icon if ID matches currentId', () => { + expect(vm.$el.querySelector('.ic-mobile-issue-close')).not.toBe(null); + }); + + it('does not render icon if ID does not match currentId', done => { + vm.currentId = '2'; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.ic-mobile-issue-close')).toBe(null); + + done(); + }); + }); + + it('does not render icon if project ID does not match', done => { + vm.currentProjectId = 'test/test'; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.ic-mobile-issue-close')).toBe(null); + + done(); + }); + }); + + it('emits click event on click', () => { + spyOn(vm, '$emit'); + + vm.$el.click(); + + expect(vm.$emit).toHaveBeenCalledWith('click', vm.item); + }); +}); diff --git a/spec/javascripts/ide/components/merge_requests/list_spec.js b/spec/javascripts/ide/components/merge_requests/list_spec.js new file mode 100644 index 00000000000..f4b393778dc --- /dev/null +++ b/spec/javascripts/ide/components/merge_requests/list_spec.js @@ -0,0 +1,126 @@ +import Vue from 'vue'; +import store from '~/ide/stores'; +import List from '~/ide/components/merge_requests/list.vue'; +import { createComponentWithStore } from '../../../helpers/vue_mount_component_helper'; +import { mergeRequests } from '../../mock_data'; +import { resetStore } from '../../helpers'; + +describe('IDE merge requests list', () => { + const Component = Vue.extend(List); + let vm; + + beforeEach(() => { + vm = createComponentWithStore(Component, store, { + type: 'created', + emptyText: 'empty text', + }); + + spyOn(vm, 'fetchMergeRequests'); + + vm.$mount(); + }); + + afterEach(() => { + vm.$destroy(); + + resetStore(vm.$store); + }); + + it('calls fetch on mounted', () => { + expect(vm.fetchMergeRequests).toHaveBeenCalledWith({ + type: 'created', + search: '', + }); + }); + + it('renders loading icon', done => { + vm.$store.state.mergeRequests.created.isLoading = true; + + vm.$nextTick(() => { + expect(vm.$el.querySelector('.loading-container')).not.toBe(null); + + done(); + }); + }); + + it('renders empty text when no merge requests exist', () => { + expect(vm.$el.textContent).toContain('empty text'); + }); + + it('renders no search results text when search is not empty', done => { + vm.search = 'testing'; + + vm.$nextTick(() => { + expect(vm.$el.textContent).toContain('No merge requests found'); + + done(); + }); + }); + + describe('with merge requests', () => { + beforeEach(done => { + vm.$store.state.mergeRequests.created.mergeRequests.push({ + ...mergeRequests[0], + projectPathWithNamespace: 'gitlab-org/gitlab-ce', + }); + + vm.$nextTick(done); + }); + + it('renders list', () => { + expect(vm.$el.querySelectorAll('li').length).toBe(1); + expect(vm.$el.querySelector('li').textContent).toContain(mergeRequests[0].title); + }); + + it('calls openMergeRequest when clicking merge request', done => { + spyOn(vm, 'openMergeRequest'); + vm.$el.querySelector('li button').click(); + + vm.$nextTick(() => { + expect(vm.openMergeRequest).toHaveBeenCalledWith({ + projectPath: 'gitlab-org/gitlab-ce', + id: 1, + }); + + done(); + }); + }); + }); + + describe('focusSearch', () => { + it('focuses search input when loading is false', done => { + spyOn(vm.$refs.searchInput, 'focus'); + + vm.$store.state.mergeRequests.created.isLoading = false; + vm.focusSearch(); + + vm.$nextTick(() => { + expect(vm.$refs.searchInput.focus).toHaveBeenCalled(); + + done(); + }); + }); + }); + + describe('searchMergeRequests', () => { + beforeEach(() => { + spyOn(vm, 'loadMergeRequests'); + + jasmine.clock().install(); + }); + + afterEach(() => { + jasmine.clock().uninstall(); + }); + + it('calls loadMergeRequests on input in search field', () => { + const event = new Event('input'); + + vm.$el.querySelector('input').dispatchEvent(event); + + jasmine.clock().tick(300); + + expect(vm.loadMergeRequests).toHaveBeenCalled(); + }); + }); +}); diff --git a/spec/javascripts/ide/mock_data.js b/spec/javascripts/ide/mock_data.js index dcf857f7e04..dd87a43f370 100644 --- a/spec/javascripts/ide/mock_data.js +++ b/spec/javascripts/ide/mock_data.js @@ -75,6 +75,7 @@ export const jobs = [ }, stage: 'test', duration: 1, + started: new Date(), }, { id: 2, @@ -86,6 +87,7 @@ export const jobs = [ }, stage: 'test', duration: 1, + started: new Date(), }, { id: 3, @@ -97,6 +99,7 @@ export const jobs = [ }, stage: 'test', duration: 1, + started: new Date(), }, { id: 4, @@ -108,6 +111,7 @@ export const jobs = [ }, stage: 'build', duration: 1, + started: new Date(), }, ]; diff --git a/spec/javascripts/ide/stores/modules/merge_requests/actions_spec.js b/spec/javascripts/ide/stores/modules/merge_requests/actions_spec.js index b571cfb963a..d178a44b76a 100644 --- a/spec/javascripts/ide/stores/modules/merge_requests/actions_spec.js +++ b/spec/javascripts/ide/stores/modules/merge_requests/actions_spec.js @@ -8,7 +8,9 @@ import actions, { receiveMergeRequestsSuccess, fetchMergeRequests, resetMergeRequests, + openMergeRequest, } from '~/ide/stores/modules/merge_requests/actions'; +import router from '~/ide/ide_router'; import { mergeRequests } from '../../../mock_data'; import testAction from '../../../../helpers/vuex_action_helper'; @@ -29,9 +31,9 @@ describe('IDE merge requests actions', () => { it('should should commit request', done => { testAction( requestMergeRequests, - null, + 'created', mockedState, - [{ type: types.REQUEST_MERGE_REQUESTS }], + [{ type: types.REQUEST_MERGE_REQUESTS, payload: 'created' }], [], done, ); @@ -48,16 +50,16 @@ describe('IDE merge requests actions', () => { it('should should commit error', done => { testAction( receiveMergeRequestsError, - null, + 'created', mockedState, - [{ type: types.RECEIVE_MERGE_REQUESTS_ERROR }], + [{ type: types.RECEIVE_MERGE_REQUESTS_ERROR, payload: 'created' }], [], done, ); }); it('creates flash message', () => { - receiveMergeRequestsError({ commit() {} }); + receiveMergeRequestsError({ commit() {} }, 'created'); expect(flashSpy).toHaveBeenCalled(); }); @@ -67,9 +69,14 @@ describe('IDE merge requests actions', () => { it('should commit received data', done => { testAction( receiveMergeRequestsSuccess, - 'data', + { type: 'created', data: 'data' }, mockedState, - [{ type: types.RECEIVE_MERGE_REQUESTS_SUCCESS, payload: 'data' }], + [ + { + type: types.RECEIVE_MERGE_REQUESTS_SUCCESS, + payload: { type: 'created', data: 'data' }, + }, + ], [], done, ); @@ -86,14 +93,14 @@ describe('IDE merge requests actions', () => { mock.onGet(/\/api\/v4\/merge_requests(.*)$/).replyOnce(200, mergeRequests); }); - it('calls API with params from state', () => { + it('calls API with params', () => { const apiSpy = spyOn(axios, 'get').and.callThrough(); - fetchMergeRequests({ dispatch() {}, state: mockedState }); + fetchMergeRequests({ dispatch() {}, state: mockedState }, { type: 'created' }); expect(apiSpy).toHaveBeenCalledWith(jasmine.anything(), { params: { - scope: 'assigned-to-me', + scope: 'created-by-me', state: 'opened', search: '', }, @@ -103,11 +110,14 @@ describe('IDE merge requests actions', () => { it('calls API with search', () => { const apiSpy = spyOn(axios, 'get').and.callThrough(); - fetchMergeRequests({ dispatch() {}, state: mockedState }, 'testing search'); + fetchMergeRequests( + { dispatch() {}, state: mockedState }, + { type: 'created', search: 'testing search' }, + ); expect(apiSpy).toHaveBeenCalledWith(jasmine.anything(), { params: { - scope: 'assigned-to-me', + scope: 'created-by-me', state: 'opened', search: 'testing search', }, @@ -117,7 +127,7 @@ describe('IDE merge requests actions', () => { it('dispatches request', done => { testAction( fetchMergeRequests, - null, + { type: 'created' }, mockedState, [], [ @@ -132,13 +142,16 @@ describe('IDE merge requests actions', () => { it('dispatches success with received data', done => { testAction( fetchMergeRequests, - null, + { type: 'created' }, mockedState, [], [ { type: 'requestMergeRequests' }, { type: 'resetMergeRequests' }, - { type: 'receiveMergeRequestsSuccess', payload: mergeRequests }, + { + type: 'receiveMergeRequestsSuccess', + payload: { type: 'created', data: mergeRequests }, + }, ], done, ); @@ -153,7 +166,7 @@ describe('IDE merge requests actions', () => { it('dispatches error', done => { testAction( fetchMergeRequests, - null, + { type: 'created' }, mockedState, [], [ @@ -171,12 +184,47 @@ describe('IDE merge requests actions', () => { it('commits reset', done => { testAction( resetMergeRequests, - null, + 'created', mockedState, - [{ type: types.RESET_MERGE_REQUESTS }], + [{ type: types.RESET_MERGE_REQUESTS, payload: 'created' }], [], done, ); }); }); + + describe('openMergeRequest', () => { + beforeEach(() => { + spyOn(router, 'push'); + }); + + it('commits reset mutations and actions', done => { + testAction( + openMergeRequest, + { projectPath: 'gitlab-org/gitlab-ce', id: '1' }, + mockedState, + [ + { type: 'CLEAR_PROJECTS' }, + { type: 'SET_CURRENT_MERGE_REQUEST', payload: '1' }, + { type: 'RESET_OPEN_FILES' }, + ], + [ + { type: 'pipelines/stopPipelinePolling' }, + { type: 'pipelines/clearEtagPoll' }, + { type: 'pipelines/resetLatestPipeline' }, + { type: 'setCurrentBranchId', payload: '' }, + ], + done, + ); + }); + + it('pushes new route', () => { + openMergeRequest( + { commit() {}, dispatch() {} }, + { projectPath: 'gitlab-org/gitlab-ce', id: '1' }, + ); + + expect(router.push).toHaveBeenCalledWith('/project/gitlab-org/gitlab-ce/merge_requests/1'); + }); + }); }); diff --git a/spec/javascripts/ide/stores/modules/merge_requests/mutations_spec.js b/spec/javascripts/ide/stores/modules/merge_requests/mutations_spec.js index 664d3914564..ea03131d90d 100644 --- a/spec/javascripts/ide/stores/modules/merge_requests/mutations_spec.js +++ b/spec/javascripts/ide/stores/modules/merge_requests/mutations_spec.js @@ -12,26 +12,29 @@ describe('IDE merge requests mutations', () => { describe(types.REQUEST_MERGE_REQUESTS, () => { it('sets loading to true', () => { - mutations[types.REQUEST_MERGE_REQUESTS](mockedState); + mutations[types.REQUEST_MERGE_REQUESTS](mockedState, 'created'); - expect(mockedState.isLoading).toBe(true); + expect(mockedState.created.isLoading).toBe(true); }); }); describe(types.RECEIVE_MERGE_REQUESTS_ERROR, () => { it('sets loading to false', () => { - mutations[types.RECEIVE_MERGE_REQUESTS_ERROR](mockedState); + mutations[types.RECEIVE_MERGE_REQUESTS_ERROR](mockedState, 'created'); - expect(mockedState.isLoading).toBe(false); + expect(mockedState.created.isLoading).toBe(false); }); }); describe(types.RECEIVE_MERGE_REQUESTS_SUCCESS, () => { it('sets merge requests', () => { gon.gitlab_url = gl.TEST_HOST; - mutations[types.RECEIVE_MERGE_REQUESTS_SUCCESS](mockedState, mergeRequests); + mutations[types.RECEIVE_MERGE_REQUESTS_SUCCESS](mockedState, { + type: 'created', + data: mergeRequests, + }); - expect(mockedState.mergeRequests).toEqual([ + expect(mockedState.created.mergeRequests).toEqual([ { id: 1, iid: 1, @@ -47,9 +50,9 @@ describe('IDE merge requests mutations', () => { it('clears merge request array', () => { mockedState.mergeRequests = ['test']; - mutations[types.RESET_MERGE_REQUESTS](mockedState); + mutations[types.RESET_MERGE_REQUESTS](mockedState, 'created'); - expect(mockedState.mergeRequests).toEqual([]); + expect(mockedState.created.mergeRequests).toEqual([]); }); }); }); diff --git a/spec/javascripts/ide/stores/modules/pipelines/actions_spec.js b/spec/javascripts/ide/stores/modules/pipelines/actions_spec.js index f26eaf9c81f..f2f8e780cd1 100644 --- a/spec/javascripts/ide/stores/modules/pipelines/actions_spec.js +++ b/spec/javascripts/ide/stores/modules/pipelines/actions_spec.js @@ -13,9 +13,15 @@ import actions, { receiveJobsSuccess, fetchJobs, toggleStageCollapsed, + setDetailJob, + requestJobTrace, + receiveJobTraceError, + receiveJobTraceSuccess, + fetchJobTrace, } from '~/ide/stores/modules/pipelines/actions'; import state from '~/ide/stores/modules/pipelines/state'; import * as types from '~/ide/stores/modules/pipelines/mutation_types'; +import { rightSidebarViews } from '~/ide/constants'; import testAction from '../../../../helpers/vuex_action_helper'; import { pipelines, jobs } from '../../../mock_data'; @@ -281,4 +287,133 @@ describe('IDE pipelines actions', () => { ); }); }); + + describe('setDetailJob', () => { + it('commits job', done => { + testAction( + setDetailJob, + 'job', + mockedState, + [{ type: types.SET_DETAIL_JOB, payload: 'job' }], + [{ type: 'setRightPane' }], + done, + ); + }); + + it('dispatches setRightPane as pipeline when job is null', done => { + testAction( + setDetailJob, + null, + mockedState, + [{ type: types.SET_DETAIL_JOB }], + [{ type: 'setRightPane', payload: rightSidebarViews.pipelines }], + done, + ); + }); + + it('dispatches setRightPane as job', done => { + testAction( + setDetailJob, + 'job', + mockedState, + [{ type: types.SET_DETAIL_JOB }], + [{ type: 'setRightPane', payload: rightSidebarViews.jobsDetail }], + done, + ); + }); + }); + + describe('requestJobTrace', () => { + it('commits request', done => { + testAction(requestJobTrace, null, mockedState, [{ type: types.REQUEST_JOB_TRACE }], [], done); + }); + }); + + describe('receiveJobTraceError', () => { + it('commits error', done => { + testAction( + receiveJobTraceError, + null, + mockedState, + [{ type: types.RECEIVE_JOB_TRACE_ERROR }], + [], + done, + ); + }); + + it('creates flash message', () => { + const flashSpy = spyOnDependency(actions, 'flash'); + + receiveJobTraceError({ commit() {} }); + + expect(flashSpy).toHaveBeenCalled(); + }); + }); + + describe('receiveJobTraceSuccess', () => { + it('commits data', done => { + testAction( + receiveJobTraceSuccess, + 'data', + mockedState, + [{ type: types.RECEIVE_JOB_TRACE_SUCCESS, payload: 'data' }], + [], + done, + ); + }); + }); + + describe('fetchJobTrace', () => { + beforeEach(() => { + mockedState.detailJob = { + path: `${gl.TEST_HOST}/project/builds`, + }; + }); + + describe('success', () => { + beforeEach(() => { + spyOn(axios, 'get').and.callThrough(); + mock.onGet(`${gl.TEST_HOST}/project/builds/trace`).replyOnce(200, { html: 'html' }); + }); + + it('dispatches request', done => { + testAction( + fetchJobTrace, + null, + mockedState, + [], + [ + { type: 'requestJobTrace' }, + { type: 'receiveJobTraceSuccess', payload: { html: 'html' } }, + ], + done, + ); + }); + + it('sends get request to correct URL', () => { + fetchJobTrace({ state: mockedState, dispatch() {} }); + + expect(axios.get).toHaveBeenCalledWith(`${gl.TEST_HOST}/project/builds/trace`, { + params: { format: 'json' }, + }); + }); + }); + + describe('error', () => { + beforeEach(() => { + mock.onGet(`${gl.TEST_HOST}/project/builds/trace`).replyOnce(500); + }); + + it('dispatches error', done => { + testAction( + fetchJobTrace, + null, + mockedState, + [], + [{ type: 'requestJobTrace' }, { type: 'receiveJobTraceError' }], + done, + ); + }); + }); + }); }); diff --git a/spec/javascripts/ide/stores/modules/pipelines/mutations_spec.js b/spec/javascripts/ide/stores/modules/pipelines/mutations_spec.js index 6285c01d483..eb7346bd5fc 100644 --- a/spec/javascripts/ide/stores/modules/pipelines/mutations_spec.js +++ b/spec/javascripts/ide/stores/modules/pipelines/mutations_spec.js @@ -147,6 +147,10 @@ describe('IDE pipelines mutations', () => { name: job.name, status: job.status, path: job.build_path, + rawPath: `${job.build_path}/raw`, + started: job.started, + isLoading: false, + output: '', })), ); }); @@ -171,4 +175,49 @@ describe('IDE pipelines mutations', () => { expect(mockedState.stages[0].isCollapsed).toBe(false); }); }); + + describe(types.SET_DETAIL_JOB, () => { + it('sets detail job', () => { + mutations[types.SET_DETAIL_JOB](mockedState, jobs[0]); + + expect(mockedState.detailJob).toEqual(jobs[0]); + }); + }); + + describe(types.REQUEST_JOB_TRACE, () => { + beforeEach(() => { + mockedState.detailJob = { ...jobs[0] }; + }); + + it('sets loading on detail job', () => { + mutations[types.REQUEST_JOB_TRACE](mockedState); + + expect(mockedState.detailJob.isLoading).toBe(true); + }); + }); + + describe(types.RECEIVE_JOB_TRACE_ERROR, () => { + beforeEach(() => { + mockedState.detailJob = { ...jobs[0], isLoading: true }; + }); + + it('sets loading to false on detail job', () => { + mutations[types.RECEIVE_JOB_TRACE_ERROR](mockedState); + + expect(mockedState.detailJob.isLoading).toBe(false); + }); + }); + + describe(types.RECEIVE_JOB_TRACE_SUCCESS, () => { + beforeEach(() => { + mockedState.detailJob = { ...jobs[0], isLoading: true }; + }); + + it('sets output on detail job', () => { + mutations[types.RECEIVE_JOB_TRACE_SUCCESS](mockedState, { html: 'html' }); + + expect(mockedState.detailJob.output).toBe('html'); + expect(mockedState.detailJob.isLoading).toBe(false); + }); + }); }); diff --git a/spec/javascripts/importer_status_spec.js b/spec/javascripts/importer_status_spec.js index 87b46ccf7c3..63cdb3d5114 100644 --- a/spec/javascripts/importer_status_spec.js +++ b/spec/javascripts/importer_status_spec.js @@ -50,6 +50,24 @@ describe('Importer Status', () => { }) .catch(done.fail); }); + + it('shows error message after failed POST request', (done) => { + appendSetFixtures('<div class="flash-container"></div>'); + + mock.onPost(importUrl).reply(422, { + errors: 'You forgot your lunch', + }); + + instance.addToImport({ + currentTarget: document.querySelector('.js-add-to-import'), + }) + .then(() => { + const flashMessage = document.querySelector('.flash-text'); + expect(flashMessage.textContent.trim()).toEqual('An error occurred while importing project: You forgot your lunch'); + done(); + }) + .catch(done.fail); + }); }); describe('autoUpdate', () => { diff --git a/spec/javascripts/jobs/header_spec.js b/spec/javascripts/jobs/header_spec.js index 4f861c39d3f..cef30a007db 100644 --- a/spec/javascripts/jobs/header_spec.js +++ b/spec/javascripts/jobs/header_spec.js @@ -13,6 +13,9 @@ describe('Job details header', () => { const threeWeeksAgo = new Date(); threeWeeksAgo.setDate(threeWeeksAgo.getDate() - 21); + const twoDaysAgo = new Date(); + twoDaysAgo.setDate(twoDaysAgo.getDate() - 2); + props = { job: { status: { @@ -31,7 +34,7 @@ describe('Job details header', () => { email: 'foo@bar.com', avatar_url: 'link', }, - started: '2018-01-08T09:48:27.319Z', + started: twoDaysAgo.toISOString(), new_issue_path: 'path', }, isLoading: false, @@ -69,7 +72,7 @@ describe('Job details header', () => { .querySelector('.header-main-content') .textContent.replace(/\s+/g, ' ') .trim(), - ).toEqual('failed Job #123 triggered 3 weeks ago by Foo'); + ).toEqual('failed Job #123 triggered 2 days ago by Foo'); }); it('should render new issue link', () => { diff --git a/spec/javascripts/monitoring/graph_spec.js b/spec/javascripts/monitoring/graph_spec.js index 220228e5c08..a46a387a534 100644 --- a/spec/javascripts/monitoring/graph_spec.js +++ b/spec/javascripts/monitoring/graph_spec.js @@ -18,9 +18,7 @@ const createComponent = propsData => { }).$mount(); }; -const convertedMetrics = convertDatesMultipleSeries( - singleRowMetricsMultipleSeries, -); +const convertedMetrics = convertDatesMultipleSeries(singleRowMetricsMultipleSeries); describe('Graph', () => { beforeEach(() => { @@ -36,7 +34,7 @@ describe('Graph', () => { projectPath, }); - expect(component.$el.querySelector('.text-center').innerText.trim()).toBe( + expect(component.$el.querySelector('.prometheus-graph-title').innerText.trim()).toBe( component.graphData.title, ); }); @@ -52,9 +50,7 @@ describe('Graph', () => { }); const transformedHeight = `${component.graphHeight - 100}`; - expect(component.axisTransform.indexOf(transformedHeight)).not.toEqual( - -1, - ); + expect(component.axisTransform.indexOf(transformedHeight)).not.toEqual(-1); }); it('outerViewBox gets a width and height property based on the DOM size of the element', () => { diff --git a/spec/javascripts/notes/components/note_actions_spec.js b/spec/javascripts/notes/components/note_actions_spec.js index 1dfe890e05e..c9e549d2096 100644 --- a/spec/javascripts/notes/components/note_actions_spec.js +++ b/spec/javascripts/notes/components/note_actions_spec.js @@ -20,7 +20,7 @@ describe('issue_note_actions component', () => { beforeEach(() => { props = { - accessLevel: 'Master', + accessLevel: 'Maintainer', authorId: 26, canDelete: true, canEdit: true, @@ -67,7 +67,7 @@ describe('issue_note_actions component', () => { beforeEach(() => { store.dispatch('setUserData', {}); props = { - accessLevel: 'Master', + accessLevel: 'Maintainer', authorId: 26, canDelete: false, canEdit: false, diff --git a/spec/javascripts/notes/mock_data.js b/spec/javascripts/notes/mock_data.js index bfe3a65feee..fa7adc32193 100644 --- a/spec/javascripts/notes/mock_data.js +++ b/spec/javascripts/notes/mock_data.js @@ -340,6 +340,79 @@ export const loggedOutnoteableData = { '/gitlab-org/gitlab-ce/preview_markdown?quick_actions_target_id=98&quick_actions_target_type=Issue', }; +export const collapseNotesMock = [ + { + expanded: true, + id: '0fb4e0e3f9276e55ff32eb4195add694aece4edd', + individual_note: true, + notes: [ + { + id: 1390, + attachment: null, + author: { + id: 1, + name: 'Root', + username: 'root', + state: 'active', + avatar_url: 'test', + path: '/root', + }, + created_at: '2018-02-26T18:07:41.071Z', + updated_at: '2018-02-26T18:07:41.071Z', + system: true, + system_note_icon_name: 'pencil', + noteable_id: 98, + noteable_type: 'Issue', + type: null, + human_access: 'Owner', + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false }, + discussion_id: 'b97fb7bda470a65b3e009377a9032edec0a4dd05', + emoji_awardable: false, + path: '/h5bp/html5-boilerplate/notes/1057', + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fh5bp%2Fhtml5-boilerplate%2Fissues%2F10%23note_1057&user_id=1', + }, + ], + }, + { + expanded: true, + id: 'ffde43f25984ad7f2b4275135e0e2846875336c0', + individual_note: true, + notes: [ + { + id: 1391, + attachment: null, + author: { + id: 1, + name: 'Root', + username: 'root', + state: 'active', + avatar_url: 'test', + path: '/root', + }, + created_at: '2018-02-26T18:13:24.071Z', + updated_at: '2018-02-26T18:13:24.071Z', + system: true, + system_note_icon_name: 'pencil', + noteable_id: 99, + noteable_type: 'Issue', + type: null, + human_access: 'Owner', + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false }, + discussion_id: '3eb958b4d81dec207ec3537a2f3bd8b9f271bb34', + emoji_awardable: false, + path: '/h5bp/html5-boilerplate/notes/1057', + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fh5bp%2Fhtml5-boilerplate%2Fissues%2F10%23note_1057&user_id=1', + }, + ], + }, +]; + export const INDIVIDUAL_NOTE_RESPONSE_MAP = { GET: { '/gitlab-org/gitlab-ce/issues/26/discussions.json': [ @@ -575,3 +648,508 @@ export function discussionNoteInterceptor(request, next) { }), ); } + +export const notesWithDescriptionChanges = [ + { + id: '39b271c2033e9ed43d8edb393702f65f7a830459', + reply_id: '39b271c2033e9ed43d8edb393702f65f7a830459', + expanded: true, + notes: [ + { + id: 901, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:05:36.117Z', + updated_at: '2018-05-29T12:05:36.117Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: + 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.', + note_html: + '<p dir="auto">Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '39b271c2033e9ed43d8edb393702f65f7a830459', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_901&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/901/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/901', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + reply_id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + expanded: true, + notes: [ + { + id: 902, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:05:58.694Z', + updated_at: '2018-05-29T12:05:58.694Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: + 'Varius vel pharetra vel turpis nunc eget lorem. Ipsum dolor sit amet consectetur adipiscing.', + note_html: + '<p dir="auto">Varius vel pharetra vel turpis nunc eget lorem. Ipsum dolor sit amet consectetur adipiscing.</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_902&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/902/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/902', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '7f1feda384083eb31763366e6392399fde6f3f31', + reply_id: '7f1feda384083eb31763366e6392399fde6f3f31', + expanded: true, + notes: [ + { + id: 903, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:06:05.772Z', + updated_at: '2018-05-29T12:06:05.772Z', + system: true, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false, can_award_emoji: true }, + resolved: false, + resolved_by: null, + system_note_icon_name: 'pencil-square', + discussion_id: '7f1feda384083eb31763366e6392399fde6f3f31', + emoji_awardable: false, + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_903&user_id=1', + human_access: 'Owner', + path: '/gitlab-org/gitlab-shell/notes/903', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + reply_id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + expanded: true, + notes: [ + { + id: 904, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:06:16.112Z', + updated_at: '2018-05-29T12:06:16.112Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'Ullamcorper eget nulla facilisi etiam', + note_html: '<p dir="auto">Ullamcorper eget nulla facilisi etiam</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_904&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/904/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/904', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + reply_id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + expanded: true, + notes: [ + { + id: 905, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:06:28.851Z', + updated_at: '2018-05-29T12:06:28.851Z', + system: true, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false, can_award_emoji: true }, + resolved: false, + resolved_by: null, + system_note_icon_name: 'pencil-square', + discussion_id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + emoji_awardable: false, + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_905&user_id=1', + human_access: 'Owner', + path: '/gitlab-org/gitlab-shell/notes/905', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '70411b08cdfc01f24187a06d77daa33464cb2620', + reply_id: '70411b08cdfc01f24187a06d77daa33464cb2620', + expanded: true, + notes: [ + { + id: 906, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:20:02.925Z', + updated_at: '2018-05-29T12:20:02.925Z', + system: true, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false, can_award_emoji: true }, + resolved: false, + resolved_by: null, + system_note_icon_name: 'pencil-square', + discussion_id: '70411b08cdfc01f24187a06d77daa33464cb2620', + emoji_awardable: false, + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_906&user_id=1', + human_access: 'Owner', + path: '/gitlab-org/gitlab-shell/notes/906', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, +]; + +export const collapsedSystemNotes = [ + { + id: '39b271c2033e9ed43d8edb393702f65f7a830459', + reply_id: '39b271c2033e9ed43d8edb393702f65f7a830459', + expanded: true, + notes: [ + { + id: 901, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:05:36.117Z', + updated_at: '2018-05-29T12:05:36.117Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: + 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.', + note_html: + '<p dir="auto">Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '39b271c2033e9ed43d8edb393702f65f7a830459', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_901&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/901/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/901', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + reply_id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + expanded: true, + notes: [ + { + id: 902, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:05:58.694Z', + updated_at: '2018-05-29T12:05:58.694Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: + 'Varius vel pharetra vel turpis nunc eget lorem. Ipsum dolor sit amet consectetur adipiscing.', + note_html: + '<p dir="auto">Varius vel pharetra vel turpis nunc eget lorem. Ipsum dolor sit amet consectetur adipiscing.</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '4852335d7dc40b9ceb8fde1a2bb9c1b67e4c7795', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_902&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/902/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/902', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + reply_id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + expanded: true, + notes: [ + { + id: 904, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:06:16.112Z', + updated_at: '2018-05-29T12:06:16.112Z', + system: false, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'Ullamcorper eget nulla facilisi etiam', + note_html: '<p dir="auto">Ullamcorper eget nulla facilisi etiam</p>', + current_user: { can_edit: true, can_award_emoji: true }, + resolved: false, + resolved_by: null, + discussion_id: '091865fe3ae20f0045234a3d103e3b15e73405b5', + emoji_awardable: true, + award_emoji: [], + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_904&user_id=1', + human_access: 'Owner', + toggle_award_path: '/gitlab-org/gitlab-shell/notes/904/toggle_award_emoji', + path: '/gitlab-org/gitlab-shell/notes/904', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + reply_id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + expanded: true, + notes: [ + { + id: 905, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:06:28.851Z', + updated_at: '2018-05-29T12:06:28.851Z', + system: true, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'changed the description', + note_html: '\n <p dir="auto">changed the description 2 times within 1 minute </p>', + current_user: { can_edit: false, can_award_emoji: true }, + resolved: false, + resolved_by: null, + system_note_icon_name: 'pencil-square', + discussion_id: 'a21cf2e804acc3c60d07e37d75e395f5a9a4d044', + emoji_awardable: false, + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_905&user_id=1', + human_access: 'Owner', + path: '/gitlab-org/gitlab-shell/notes/905', + times_updated: 2, + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, + { + id: '70411b08cdfc01f24187a06d77daa33464cb2620', + reply_id: '70411b08cdfc01f24187a06d77daa33464cb2620', + expanded: true, + notes: [ + { + id: 906, + type: null, + attachment: null, + author: { + id: 1, + name: 'Administrator', + username: 'root', + state: 'active', + avatar_url: + 'https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon', + path: '/root', + }, + created_at: '2018-05-29T12:20:02.925Z', + updated_at: '2018-05-29T12:20:02.925Z', + system: true, + noteable_id: 182, + noteable_type: 'Issue', + resolvable: false, + noteable_iid: 12, + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + current_user: { can_edit: false, can_award_emoji: true }, + resolved: false, + resolved_by: null, + system_note_icon_name: 'pencil-square', + discussion_id: '70411b08cdfc01f24187a06d77daa33464cb2620', + emoji_awardable: false, + report_abuse_path: + '/abuse_reports/new?ref_url=http%3A%2F%2Flocalhost%3A3000%2Fgitlab-org%2Fgitlab-shell%2Fissues%2F12%23note_906&user_id=1', + human_access: 'Owner', + path: '/gitlab-org/gitlab-shell/notes/906', + }, + ], + individual_note: true, + resolvable: false, + resolved: false, + diff_discussion: false, + }, +]; diff --git a/spec/javascripts/notes/stores/collapse_utils_spec.js b/spec/javascripts/notes/stores/collapse_utils_spec.js new file mode 100644 index 00000000000..06a6aab932a --- /dev/null +++ b/spec/javascripts/notes/stores/collapse_utils_spec.js @@ -0,0 +1,46 @@ +import { + isDescriptionSystemNote, + changeDescriptionNote, + getTimeDifferenceMinutes, + collapseSystemNotes, +} from '~/notes/stores/collapse_utils'; +import { + notesWithDescriptionChanges, + collapsedSystemNotes, +} from '../mock_data'; + +describe('Collapse utils', () => { + const mockSystemNote = { + note: 'changed the description', + note_html: '<p dir="auto">changed the description</p>', + system: true, + created_at: '2018-05-14T21:28:00.000Z', + }; + + it('checks if a system note is of a description type', () => { + expect(isDescriptionSystemNote(mockSystemNote)).toEqual(true); + }); + + it('returns false when a system note is not a description type', () => { + expect(isDescriptionSystemNote(Object.assign({}, mockSystemNote, { note: 'foo' }))).toEqual(false); + }); + + it('changes the description to contain the number of changed times', () => { + const changedNote = changeDescriptionNote(mockSystemNote, 3, 5); + + expect(changedNote.times_updated).toEqual(3); + expect(changedNote.note_html.trim()).toContain('<p dir="auto">changed the description 3 times within 5 minutes </p>'); + }); + + it('gets the time difference between two notes', () => { + const anotherSystemNote = { + created_at: '2018-05-14T21:33:00.000Z', + }; + + expect(getTimeDifferenceMinutes(mockSystemNote, anotherSystemNote)).toEqual(5); + }); + + it('collapses all description system notes made within 10 minutes or less from each other', () => { + expect(collapseSystemNotes(notesWithDescriptionChanges)).toEqual(collapsedSystemNotes); + }); +}); diff --git a/spec/javascripts/notes/stores/getters_spec.js b/spec/javascripts/notes/stores/getters_spec.js index 8b2a8d2cd7a..e5550580bf8 100644 --- a/spec/javascripts/notes/stores/getters_spec.js +++ b/spec/javascripts/notes/stores/getters_spec.js @@ -1,8 +1,9 @@ import * as getters from '~/notes/stores/getters'; -import { notesDataMock, userDataMock, noteableDataMock, individualNote } from '../mock_data'; +import { notesDataMock, userDataMock, noteableDataMock, individualNote, collapseNotesMock } from '../mock_data'; describe('Getters Notes Store', () => { let state; + beforeEach(() => { state = { notes: [individualNote], @@ -20,6 +21,22 @@ describe('Getters Notes Store', () => { }); }); + describe('Collapsed notes', () => { + const stateCollapsedNotes = { + notes: collapseNotesMock, + targetNoteHash: 'hash', + lastFetchedAt: 'timestamp', + + notesData: notesDataMock, + userData: userDataMock, + noteableData: noteableDataMock, + }; + + it('should return a single system note when a description was updated multiple times', () => { + expect(getters.notes(stateCollapsedNotes).length).toEqual(1); + }); + }); + describe('targetNoteHash', () => { it('should return `targetNoteHash`', () => { expect(getters.targetNoteHash(state)).toEqual('hash'); diff --git a/spec/javascripts/vue_shared/components/gl_modal_spec.js b/spec/javascripts/vue_shared/components/gl_modal_spec.js index 85cb1b90fc6..23be8d93b81 100644 --- a/spec/javascripts/vue_shared/components/gl_modal_spec.js +++ b/spec/javascripts/vue_shared/components/gl_modal_spec.js @@ -190,4 +190,37 @@ describe('GlModal', () => { }); }); }); + + describe('handling sizes', () => { + it('should render modal-sm', () => { + vm = mountComponent(modalComponent, { + modalSize: 'sm', + }); + + expect(vm.$el.querySelector('.modal-dialog').classList.contains('modal-sm')).toEqual(true); + }); + + it('should render modal-lg', () => { + vm = mountComponent(modalComponent, { + modalSize: 'lg', + }); + + expect(vm.$el.querySelector('.modal-dialog').classList.contains('modal-lg')).toEqual(true); + }); + + it('should not add modal size classes when md size is passed', () => { + vm = mountComponent(modalComponent, { + modalSize: 'md', + }); + + expect(vm.$el.querySelector('.modal-dialog').classList.contains('modal-md')).toEqual(false); + }); + + it('should not add modal size classes by default', () => { + vm = mountComponent(modalComponent, {}); + + expect(vm.$el.querySelector('.modal-dialog').classList.contains('modal-sm')).toEqual(false); + expect(vm.$el.querySelector('.modal-dialog').classList.contains('modal-lg')).toEqual(false); + }); + }); }); diff --git a/spec/lib/backup/files_spec.rb b/spec/lib/backup/files_spec.rb index 99872211a4e..63f2298357f 100644 --- a/spec/lib/backup/files_spec.rb +++ b/spec/lib/backup/files_spec.rb @@ -46,7 +46,9 @@ describe Backup::Files do end it 'calls tar command with unlink' do - expect(subject).to receive(:run_pipeline!).with([%w(gzip -cd), %w(tar --unlink-first --recursive-unlink -C /var/gitlab-registry -xf -)], any_args) + expect(subject).to receive(:tar).and_return('blabla-tar') + + expect(subject).to receive(:run_pipeline!).with([%w(gzip -cd), %w(blabla-tar --unlink-first --recursive-unlink -C /var/gitlab-registry -xf -)], any_args) subject.restore end end diff --git a/spec/lib/backup/manager_spec.rb b/spec/lib/backup/manager_spec.rb index 23c04a1a101..ca319679e80 100644 --- a/spec/lib/backup/manager_spec.rb +++ b/spec/lib/backup/manager_spec.rb @@ -274,16 +274,13 @@ describe Backup::Manager do } ) - # the Fog mock only knows about directories we create explicitly Fog.mock! + + # the Fog mock only knows about directories we create explicitly connection = ::Fog::Storage.new(Gitlab.config.backup.upload.connection.symbolize_keys) connection.directories.create(key: Gitlab.config.backup.upload.remote_directory) end - after do - Fog.unmock! - end - context 'target path' do it 'uses the tar filename by default' do expect_any_instance_of(Fog::Collection).to receive(:create) diff --git a/spec/lib/backup/repository_spec.rb b/spec/lib/backup/repository_spec.rb index f583b2021a2..92a27e308d2 100644 --- a/spec/lib/backup/repository_spec.rb +++ b/spec/lib/backup/repository_spec.rb @@ -34,7 +34,9 @@ describe Backup::Repository do let(:timestamp) { Time.utc(2017, 3, 22) } let(:temp_dirs) do Gitlab.config.repositories.storages.map do |name, storage| - File.join(storage.legacy_disk_path, '..', 'repositories.old.' + timestamp.to_i.to_s) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + File.join(storage.legacy_disk_path, '..', 'repositories.old.' + timestamp.to_i.to_s) + end end end diff --git a/spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb b/spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb new file mode 100644 index 00000000000..877c061d11b --- /dev/null +++ b/spec/lib/gitlab/background_migration/archive_legacy_traces_spec.rb @@ -0,0 +1,59 @@ +require 'spec_helper' + +describe Gitlab::BackgroundMigration::ArchiveLegacyTraces, :migration, schema: 20180529152628 do + include TraceHelpers + + let(:namespaces) { table(:namespaces) } + let(:projects) { table(:projects) } + let(:builds) { table(:ci_builds) } + let(:job_artifacts) { table(:ci_job_artifacts) } + + before do + namespaces.create!(id: 123, name: 'gitlab1', path: 'gitlab1') + projects.create!(id: 123, name: 'gitlab1', path: 'gitlab1', namespace_id: 123) + @build = builds.create!(id: 1, project_id: 123, status: 'success', type: 'Ci::Build') + end + + context 'when trace file exsits at the right place' do + before do + create_legacy_trace(@build, 'trace in file') + end + + it 'correctly archive legacy traces' do + expect(job_artifacts.count).to eq(0) + expect(File.exist?(legacy_trace_path(@build))).to be_truthy + + described_class.new.perform(1, 1) + + expect(job_artifacts.count).to eq(1) + expect(File.exist?(legacy_trace_path(@build))).to be_falsy + expect(File.read(archived_trace_path(job_artifacts.first))).to eq('trace in file') + end + end + + context 'when trace file does not exsits at the right place' do + it 'does not raise errors nor create job artifact' do + expect { described_class.new.perform(1, 1) }.not_to raise_error + + expect(job_artifacts.count).to eq(0) + end + end + + context 'when trace data exsits in database' do + before do + create_legacy_trace_in_db(@build, 'trace in db') + end + + it 'correctly archive legacy traces' do + expect(job_artifacts.count).to eq(0) + expect(@build.read_attribute(:trace)).not_to be_empty + + described_class.new.perform(1, 1) + + @build.reload + expect(job_artifacts.count).to eq(1) + expect(@build.read_attribute(:trace)).to be_nil + expect(File.read(archived_trace_path(job_artifacts.first))).to eq('trace in db') + end + end +end diff --git a/spec/lib/gitlab/bare_repository_import/importer_spec.rb b/spec/lib/gitlab/bare_repository_import/importer_spec.rb index 5c8a19a53bc..468f6ff6d24 100644 --- a/spec/lib/gitlab/bare_repository_import/importer_spec.rb +++ b/spec/lib/gitlab/bare_repository_import/importer_spec.rb @@ -20,6 +20,13 @@ describe Gitlab::BareRepositoryImport::Importer, repository: true do Rainbow.enabled = @rainbow end + around do |example| + # TODO migrate BareRepositoryImport https://gitlab.com/gitlab-org/gitaly/issues/953 + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + example.run + end + end + shared_examples 'importing a repository' do describe '.execute' do it 'creates a project for a repository in storage' do diff --git a/spec/lib/gitlab/bare_repository_import/repository_spec.rb b/spec/lib/gitlab/bare_repository_import/repository_spec.rb index 1504826c7a5..afd8f5da39f 100644 --- a/spec/lib/gitlab/bare_repository_import/repository_spec.rb +++ b/spec/lib/gitlab/bare_repository_import/repository_spec.rb @@ -62,8 +62,10 @@ describe ::Gitlab::BareRepositoryImport::Repository do before do gitlab_shell.create_repository(repository_storage, hashed_path) - repository = Rugged::Repository.new(repo_path) - repository.config['gitlab.fullpath'] = 'to/repo' + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + repository = Rugged::Repository.new(repo_path) + repository.config['gitlab.fullpath'] = 'to/repo' + end end after do diff --git a/spec/lib/gitlab/checks/change_access_spec.rb b/spec/lib/gitlab/checks/change_access_spec.rb index 48e9902027c..1cb8143a9e9 100644 --- a/spec/lib/gitlab/checks/change_access_spec.rb +++ b/spec/lib/gitlab/checks/change_access_spec.rb @@ -52,7 +52,7 @@ describe Gitlab::Checks::ChangeAccess do context 'with protected tag' do let!(:protected_tag) { create(:protected_tag, project: project, name: 'v*') } - context 'as master' do + context 'as maintainer' do before do project.add_master(user) end @@ -138,7 +138,7 @@ describe Gitlab::Checks::ChangeAccess do context 'if the user is not allowed to delete protected branches' do it 'raises an error' do - expect { subject.exec }.to raise_error(Gitlab::GitAccess::UnauthorizedError, 'You are not allowed to delete protected branches from this project. Only a project master or owner can delete a protected branch.') + expect { subject.exec }.to raise_error(Gitlab::GitAccess::UnauthorizedError, 'You are not allowed to delete protected branches from this project. Only a project maintainer or owner can delete a protected branch.') end end diff --git a/spec/lib/gitlab/ci/pipeline/chain/populate_spec.rb b/spec/lib/gitlab/ci/pipeline/chain/populate_spec.rb index 4d7d6951a51..c5a4d9b4778 100644 --- a/spec/lib/gitlab/ci/pipeline/chain/populate_spec.rb +++ b/spec/lib/gitlab/ci/pipeline/chain/populate_spec.rb @@ -42,6 +42,10 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do it 'correctly assigns user' do expect(pipeline.builds).to all(have_attributes(user: user)) end + + it 'has pipeline iid' do + expect(pipeline.iid).to be > 0 + end end context 'when pipeline is empty' do @@ -68,6 +72,10 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do expect(pipeline.errors.to_a) .to include 'No stages / jobs for this pipeline.' end + + it 'wastes pipeline iid' do + expect(InternalId.ci_pipelines.where(project_id: project.id).last.last_value).to be > 0 + end end context 'when pipeline has validation errors' do @@ -87,6 +95,10 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do expect(pipeline.errors.to_a) .to include 'Failed to build the pipeline!' end + + it 'wastes pipeline iid' do + expect(InternalId.ci_pipelines.where(project_id: project.id).last.last_value).to be > 0 + end end context 'when there is a seed blocks present' do @@ -111,6 +123,12 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do expect(pipeline.variables.first.key).to eq 'VAR' expect(pipeline.variables.first.value).to eq '123' end + + it 'has pipeline iid' do + step.perform! + + expect(pipeline.iid).to be > 0 + end end context 'when seeds block tries to persist some resources' do @@ -121,6 +139,12 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do it 'raises exception' do expect { step.perform! }.to raise_error(ActiveRecord::RecordNotSaved) end + + it 'wastes pipeline iid' do + expect { step.perform! }.to raise_error + + expect(InternalId.ci_pipelines.where(project_id: project.id).last.last_value).to be > 0 + end end end @@ -132,22 +156,39 @@ describe Gitlab::Ci::Pipeline::Chain::Populate do end end - context 'when using only/except build policies' do - let(:config) do - { rspec: { script: 'rspec', stage: 'test', only: ['master'] }, - prod: { script: 'cap prod', stage: 'deploy', only: ['tags'] } } - end + context 'when variables policy is specified' do + shared_examples_for 'a correct pipeline' do + it 'populates pipeline according to used policies' do + step.perform! - let(:pipeline) do - build(:ci_pipeline, ref: 'master', config: config) + expect(pipeline.stages.size).to eq 1 + expect(pipeline.stages.first.builds.size).to eq 1 + expect(pipeline.stages.first.builds.first.name).to eq 'rspec' + end end - it 'populates pipeline according to used policies' do - step.perform! + context 'when using only/except build policies' do + let(:config) do + { rspec: { script: 'rspec', stage: 'test', only: ['master'] }, + prod: { script: 'cap prod', stage: 'deploy', only: ['tags'] } } + end + + let(:pipeline) do + build(:ci_pipeline, ref: 'master', config: config) + end - expect(pipeline.stages.size).to eq 1 - expect(pipeline.stages.first.builds.size).to eq 1 - expect(pipeline.stages.first.builds.first.name).to eq 'rspec' + it_behaves_like 'a correct pipeline' + + context 'when variables expression is specified' do + context 'when pipeline iid is the subject' do + let(:config) do + { rspec: { script: 'rspec', only: { variables: ["$CI_PIPELINE_IID == '1'"] } }, + prod: { script: 'cap prod', only: { variables: ["$CI_PIPELINE_IID == '1000'"] } } } + end + + it_behaves_like 'a correct pipeline' + end + end end end end diff --git a/spec/lib/gitlab/ci/pipeline/preloader_spec.rb b/spec/lib/gitlab/ci/pipeline/preloader_spec.rb index 477c7477df0..40dfd893465 100644 --- a/spec/lib/gitlab/ci/pipeline/preloader_spec.rb +++ b/spec/lib/gitlab/ci/pipeline/preloader_spec.rb @@ -3,18 +3,47 @@ require 'spec_helper' describe Gitlab::Ci::Pipeline::Preloader do - describe '.preload' do - it 'preloads the author of every pipeline commit' do - commit = double(:commit) - pipeline = double(:pipeline, commit: commit) + let(:stage) { double(:stage) } + let(:commit) { double(:commit) } - expect(commit) - .to receive(:lazy_author) + let(:pipeline) do + double(:pipeline, commit: commit, stages: [stage]) + end + + describe '.preload!' do + context 'when preloading multiple commits' do + let(:project) { create(:project, :repository) } + + it 'preloads all commits once' do + expect(Commit).to receive(:decorate).once.and_call_original + + pipelines = [build_pipeline(ref: 'HEAD'), + build_pipeline(ref: 'HEAD~1')] + + described_class.preload!(pipelines) + end + + def build_pipeline(ref:) + build_stubbed(:ci_pipeline, project: project, sha: project.commit(ref).id) + end + end + + it 'preloads commit authors and number of warnings' do + expect(commit).to receive(:lazy_author) + expect(pipeline).to receive(:number_of_warnings) + expect(stage).to receive(:number_of_warnings) + + described_class.preload!([pipeline]) + end + + it 'returns original collection' do + allow(commit).to receive(:lazy_author) + allow(pipeline).to receive(:number_of_warnings) + allow(stage).to receive(:number_of_warnings) - expect(pipeline) - .to receive(:number_of_warnings) + pipelines = [pipeline, pipeline] - described_class.preload([pipeline]) + expect(described_class.preload!(pipelines)).to eq pipelines end end end diff --git a/spec/lib/gitlab/ci/trace_spec.rb b/spec/lib/gitlab/ci/trace_spec.rb index e9d755c2021..d6510649dba 100644 --- a/spec/lib/gitlab/ci/trace_spec.rb +++ b/spec/lib/gitlab/ci/trace_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper' -describe Gitlab::Ci::Trace, :clean_gitlab_redis_cache do +describe Gitlab::Ci::Trace, :clean_gitlab_redis_shared_state do let(:build) { create(:ci_build) } let(:trace) { described_class.new(build) } diff --git a/spec/lib/gitlab/current_settings_spec.rb b/spec/lib/gitlab/current_settings_spec.rb index 19028495f52..55490f37ac7 100644 --- a/spec/lib/gitlab/current_settings_spec.rb +++ b/spec/lib/gitlab/current_settings_spec.rb @@ -5,6 +5,13 @@ describe Gitlab::CurrentSettings do stub_env('IN_MEMORY_APPLICATION_SETTINGS', 'false') end + shared_context 'with settings in cache' do + before do + create(:application_setting) + described_class.current_application_settings # warm the cache + end + end + describe '#current_application_settings', :use_clean_rails_memory_store_caching do it 'allows keys to be called directly' do db_settings = create(:application_setting, @@ -31,16 +38,29 @@ describe Gitlab::CurrentSettings do end context 'with DB unavailable' do - before do - # For some reason, `allow(described_class).to receive(:connect_to_db?).and_return(false)` causes issues - # during the initialization phase of the test suite, so instead let's mock the internals of it - allow(ActiveRecord::Base.connection).to receive(:active?).and_return(false) + context 'and settings in cache' do + include_context 'with settings in cache' + + it 'fetches the settings from cache without issuing any query' do + expect(ActiveRecord::QueryRecorder.new { described_class.current_application_settings }.count).to eq(0) + end end - it 'returns an in-memory ApplicationSetting object' do - expect(ApplicationSetting).not_to receive(:current) + context 'and no settings in cache' do + before do + # For some reason, `allow(described_class).to receive(:connect_to_db?).and_return(false)` causes issues + # during the initialization phase of the test suite, so instead let's mock the internals of it + allow(ActiveRecord::Base.connection).to receive(:active?).and_return(false) + expect(ApplicationSetting).not_to receive(:current) + end - expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + it 'returns an in-memory ApplicationSetting object' do + expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + end + + it 'does not issue any query' do + expect(ActiveRecord::QueryRecorder.new { described_class.current_application_settings }.count).to eq(0) + end end end @@ -52,73 +72,86 @@ describe Gitlab::CurrentSettings do ar_wrapped_defaults.slice(*::ApplicationSetting.defaults.keys) end - before do - # For some reason, `allow(described_class).to receive(:connect_to_db?).and_return(true)` causes issues - # during the initialization phase of the test suite, so instead let's mock the internals of it - allow(ActiveRecord::Base.connection).to receive(:active?).and_return(true) - allow(ActiveRecord::Base.connection).to receive(:cached_table_exists?).with('application_settings').and_return(true) - end + context 'and settings in cache' do + include_context 'with settings in cache' - it 'creates default ApplicationSettings if none are present' do - settings = described_class.current_application_settings - - expect(settings).to be_a(ApplicationSetting) - expect(settings).to be_persisted - expect(settings).to have_attributes(settings_from_defaults) + it 'fetches the settings from cache' do + # For some reason, `allow(described_class).to receive(:connect_to_db?).and_return(true)` causes issues + # during the initialization phase of the test suite, so instead let's mock the internals of it + expect(ActiveRecord::Base.connection).not_to receive(:active?) + expect(ActiveRecord::Base.connection).not_to receive(:cached_table_exists?) + expect(ActiveRecord::Migrator).not_to receive(:needs_migration?) + expect(ActiveRecord::QueryRecorder.new { described_class.current_application_settings }.count).to eq(0) + end end - context 'with migrations pending' do + context 'and no settings in cache' do before do - expect(ActiveRecord::Migrator).to receive(:needs_migration?).and_return(true) + allow(ActiveRecord::Base.connection).to receive(:active?).and_return(true) + allow(ActiveRecord::Base.connection).to receive(:cached_table_exists?).with('application_settings').and_return(true) end - it 'returns an in-memory ApplicationSetting object' do + it 'creates default ApplicationSettings if none are present' do settings = described_class.current_application_settings - expect(settings).to be_a(Gitlab::FakeApplicationSettings) - expect(settings.sign_in_enabled?).to eq(settings.sign_in_enabled) - expect(settings.sign_up_enabled?).to eq(settings.sign_up_enabled) + expect(settings).to be_a(ApplicationSetting) + expect(settings).to be_persisted + expect(settings).to have_attributes(settings_from_defaults) end - it 'uses the existing database settings and falls back to defaults' do - db_settings = create(:application_setting, - home_page_url: 'http://mydomain.com', - signup_enabled: false) - settings = described_class.current_application_settings - app_defaults = ApplicationSetting.last - - expect(settings).to be_a(Gitlab::FakeApplicationSettings) - expect(settings.home_page_url).to eq(db_settings.home_page_url) - expect(settings.signup_enabled?).to be_falsey - expect(settings.signup_enabled).to be_falsey - - # Check that unspecified values use the defaults - settings.reject! { |key, _| [:home_page_url, :signup_enabled].include? key } - settings.each { |key, _| expect(settings[key]).to eq(app_defaults[key]) } + context 'with migrations pending' do + before do + expect(ActiveRecord::Migrator).to receive(:needs_migration?).and_return(true) + end + + it 'returns an in-memory ApplicationSetting object' do + settings = described_class.current_application_settings + + expect(settings).to be_a(Gitlab::FakeApplicationSettings) + expect(settings.sign_in_enabled?).to eq(settings.sign_in_enabled) + expect(settings.sign_up_enabled?).to eq(settings.sign_up_enabled) + end + + it 'uses the existing database settings and falls back to defaults' do + db_settings = create(:application_setting, + home_page_url: 'http://mydomain.com', + signup_enabled: false) + settings = described_class.current_application_settings + app_defaults = ApplicationSetting.last + + expect(settings).to be_a(Gitlab::FakeApplicationSettings) + expect(settings.home_page_url).to eq(db_settings.home_page_url) + expect(settings.signup_enabled?).to be_falsey + expect(settings.signup_enabled).to be_falsey + + # Check that unspecified values use the defaults + settings.reject! { |key, _| [:home_page_url, :signup_enabled].include? key } + settings.each { |key, _| expect(settings[key]).to eq(app_defaults[key]) } + end end - end - context 'when ApplicationSettings.current is present' do - it 'returns the existing application settings' do - expect(ApplicationSetting).to receive(:current).and_return(:current_settings) + context 'when ApplicationSettings.current is present' do + it 'returns the existing application settings' do + expect(ApplicationSetting).to receive(:current).and_return(:current_settings) - expect(described_class.current_application_settings).to eq(:current_settings) + expect(described_class.current_application_settings).to eq(:current_settings) + end end - end - context 'when the application_settings table does not exists' do - it 'returns an in-memory ApplicationSetting object' do - expect(ApplicationSetting).to receive(:create_from_defaults).and_raise(ActiveRecord::StatementInvalid) + context 'when the application_settings table does not exists' do + it 'returns an in-memory ApplicationSetting object' do + expect(ApplicationSetting).to receive(:create_from_defaults).and_raise(ActiveRecord::StatementInvalid) - expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + end end - end - context 'when the application_settings table is not fully migrated' do - it 'returns an in-memory ApplicationSetting object' do - expect(ApplicationSetting).to receive(:create_from_defaults).and_raise(ActiveRecord::UnknownAttributeError) + context 'when the application_settings table is not fully migrated' do + it 'returns an in-memory ApplicationSetting object' do + expect(ApplicationSetting).to receive(:create_from_defaults).and_raise(ActiveRecord::UnknownAttributeError) - expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + expect(described_class.current_application_settings).to be_a(Gitlab::FakeApplicationSettings) + end end end end diff --git a/spec/lib/gitlab/git/repository_spec.rb b/spec/lib/gitlab/git/repository_spec.rb index 7a9621d9c78..1744db1b17e 100644 --- a/spec/lib/gitlab/git/repository_spec.rb +++ b/spec/lib/gitlab/git/repository_spec.rb @@ -159,6 +159,7 @@ describe Gitlab::Git::Repository, seed_helper: true do let(:feature2) { 'feature2' } around do |example| + # discover_default_branch will be moved to gitaly-ruby Gitlab::GitalyClient::StorageSettings.allow_disk_access do example.run end @@ -255,7 +256,7 @@ describe Gitlab::Git::Repository, seed_helper: true do let(:expected_path) { File.join(storage_path, cache_key, expected_filename) } let(:expected_prefix) { "gitlab-git-test-#{ref}-#{SeedRepo::LastCommit::ID}" } - subject(:metadata) { repository.archive_metadata(ref, storage_path, format, append_sha: append_sha) } + subject(:metadata) { repository.archive_metadata(ref, storage_path, 'gitlab-git-test', format, append_sha: append_sha) } it 'sets CommitId to the commit SHA' do expect(metadata['CommitId']).to eq(SeedRepo::LastCommit::ID) @@ -373,6 +374,7 @@ describe Gitlab::Git::Repository, seed_helper: true do context '#submodules' do around do |example| + # TODO #submodules will be removed, has been migrated to gitaly Gitlab::GitalyClient::StorageSettings.allow_disk_access do example.run end @@ -1055,6 +1057,7 @@ describe Gitlab::Git::Repository, seed_helper: true do describe "#rugged_commits_between" do around do |example| + # TODO #rugged_commits_between will be removed, has been migrated to gitaly Gitlab::GitalyClient::StorageSettings.allow_disk_access do example.run end @@ -1703,6 +1706,7 @@ describe Gitlab::Git::Repository, seed_helper: true do let(:refs) { ['deadbeef', SeedRepo::RubyBlob::ID, '909e6157199'] } around do |example| + # TODO #batch_existence isn't used anywhere, can we remove it? Gitlab::GitalyClient::StorageSettings.allow_disk_access do example.run end @@ -2002,6 +2006,18 @@ describe Gitlab::Git::Repository, seed_helper: true do expect(config).to include("fullpath = #{repository_path}") end end + + context 'repository does not exist' do + it 'raises NoRepository and does not call Gitaly WriteConfig' do + repository = Gitlab::Git::Repository.new('default', 'does/not/exist.git', '') + + expect(repository.gitaly_repository_client).not_to receive(:write_config) + + expect do + repository.write_config(full_path: 'foo/bar.git') + end.to raise_error(Gitlab::Git::Repository::NoRepository) + end + end end context "when gitaly_write_config is enabled" do diff --git a/spec/lib/gitlab/git/rev_list_spec.rb b/spec/lib/gitlab/git/rev_list_spec.rb index 32ec1e029c8..70e90659b0f 100644 --- a/spec/lib/gitlab/git/rev_list_spec.rb +++ b/spec/lib/gitlab/git/rev_list_spec.rb @@ -9,9 +9,11 @@ describe Gitlab::Git::RevList do end def stub_popen_rev_list(*additional_args, with_lazy_block: true, output:) + repo_path = Gitlab::GitalyClient::StorageSettings.allow_disk_access { repository.path } + params = [ args_for_popen(additional_args), - repository.path, + repo_path, {}, hash_including(lazy_block: with_lazy_block ? anything : nil) ] @@ -86,7 +88,7 @@ describe Gitlab::Git::RevList do context '#all_objects' do it 'fetches list of all pushed objects using rev-list' do - stub_popen_rev_list('--all', '--objects', output: "sha1\nsha2") + stub_popen_rev_list('--all', '--objects', '--filter=blob:limit=200', output: "sha1\nsha2") expect { |b| rev_list.all_objects(&b) }.to yield_with_args(%w[sha1 sha2]) end diff --git a/spec/lib/gitlab/git_access_spec.rb b/spec/lib/gitlab/git_access_spec.rb index dfffea7797f..0d5f6a0b576 100644 --- a/spec/lib/gitlab/git_access_spec.rb +++ b/spec/lib/gitlab/git_access_spec.rb @@ -552,7 +552,7 @@ describe Gitlab::GitAccess do it 'returns not found' do project.add_guest(user) repo = project.repository - FileUtils.rm_rf(repo.path) + Gitlab::GitalyClient::StorageSettings.allow_disk_access { FileUtils.rm_rf(repo.path) } # Sanity check for rm_rf expect(repo.exists?).to eq(false) @@ -750,20 +750,22 @@ describe Gitlab::GitAccess do def merge_into_protected_branch @protected_branch_merge_commit ||= begin - stub_git_hooks - project.repository.add_branch(user, unprotected_branch, 'feature') - target_branch = project.repository.lookup('feature') - source_branch = project.repository.create_file( - user, - 'filename', - 'This is the file content', - message: 'This is a good commit message', - branch_name: unprotected_branch) - rugged = project.repository.rugged - author = { email: "email@example.com", time: Time.now, name: "Example Git User" } - - merge_index = rugged.merge_commits(target_branch, source_branch) - Rugged::Commit.create(rugged, author: author, committer: author, message: "commit message", parents: [target_branch, source_branch], tree: merge_index.write_tree(rugged)) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + stub_git_hooks + project.repository.add_branch(user, unprotected_branch, 'feature') + target_branch = project.repository.lookup('feature') + source_branch = project.repository.create_file( + user, + 'filename', + 'This is the file content', + message: 'This is a good commit message', + branch_name: unprotected_branch) + rugged = project.repository.rugged + author = { email: "email@example.com", time: Time.now, name: "Example Git User" } + + merge_index = rugged.merge_commits(target_branch, source_branch) + Rugged::Commit.create(rugged, author: author, committer: author, message: "commit message", parents: [target_branch, source_branch], tree: merge_index.write_tree(rugged)) + end end end diff --git a/spec/lib/gitlab/github_import/importer/lfs_object_importer_spec.rb b/spec/lib/gitlab/github_import/importer/lfs_object_importer_spec.rb new file mode 100644 index 00000000000..4857f2afbe2 --- /dev/null +++ b/spec/lib/gitlab/github_import/importer/lfs_object_importer_spec.rb @@ -0,0 +1,23 @@ +require 'spec_helper' + +describe Gitlab::GithubImport::Importer::LfsObjectImporter do + let(:project) { create(:project) } + let(:download_link) { "http://www.gitlab.com/lfs_objects/oid" } + + let(:github_lfs_object) do + Gitlab::GithubImport::Representation::LfsObject.new( + oid: 'oid', download_link: download_link + ) + end + + let(:importer) { described_class.new(github_lfs_object, project, nil) } + + describe '#execute' do + it 'calls the LfsDownloadService with the lfs object attributes' do + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService) + .to receive(:execute).with('oid', download_link) + + importer.execute + end + end +end diff --git a/spec/lib/gitlab/github_import/importer/lfs_objects_importer_spec.rb b/spec/lib/gitlab/github_import/importer/lfs_objects_importer_spec.rb new file mode 100644 index 00000000000..5f5c6b803c0 --- /dev/null +++ b/spec/lib/gitlab/github_import/importer/lfs_objects_importer_spec.rb @@ -0,0 +1,94 @@ +require 'spec_helper' + +describe Gitlab::GithubImport::Importer::LfsObjectsImporter do + let(:project) { double(:project, id: 4, import_source: 'foo/bar') } + let(:client) { double(:client) } + let(:download_link) { "http://www.gitlab.com/lfs_objects/oid" } + + let(:github_lfs_object) { ['oid', download_link] } + + describe '#parallel?' do + it 'returns true when running in parallel mode' do + importer = described_class.new(project, client) + expect(importer).to be_parallel + end + + it 'returns false when running in sequential mode' do + importer = described_class.new(project, client, parallel: false) + expect(importer).not_to be_parallel + end + end + + describe '#execute' do + context 'when running in parallel mode' do + it 'imports lfs objects in parallel' do + importer = described_class.new(project, client) + + expect(importer).to receive(:parallel_import) + + importer.execute + end + end + + context 'when running in sequential mode' do + it 'imports lfs objects in sequence' do + importer = described_class.new(project, client, parallel: false) + + expect(importer).to receive(:sequential_import) + + importer.execute + end + end + end + + describe '#sequential_import' do + it 'imports each lfs object in sequence' do + importer = described_class.new(project, client, parallel: false) + lfs_object_importer = double(:lfs_object_importer) + + allow(importer) + .to receive(:each_object_to_import) + .and_yield(['oid', download_link]) + + expect(Gitlab::GithubImport::Importer::LfsObjectImporter) + .to receive(:new) + .with( + an_instance_of(Gitlab::GithubImport::Representation::LfsObject), + project, + client + ) + .and_return(lfs_object_importer) + + expect(lfs_object_importer).to receive(:execute) + + importer.sequential_import + end + end + + describe '#parallel_import' do + it 'imports each lfs object in parallel' do + importer = described_class.new(project, client) + + allow(importer) + .to receive(:each_object_to_import) + .and_yield(github_lfs_object) + + expect(Gitlab::GithubImport::ImportLfsObjectWorker) + .to receive(:perform_async) + .with(project.id, an_instance_of(Hash), an_instance_of(String)) + + waiter = importer.parallel_import + + expect(waiter).to be_an_instance_of(Gitlab::JobWaiter) + expect(waiter.jobs_remaining).to eq(1) + end + end + + describe '#collection_options' do + it 'returns an empty Hash' do + importer = described_class.new(project, client) + + expect(importer.collection_options).to eq({}) + end + end +end diff --git a/spec/lib/gitlab/github_import/importer/repository_importer_spec.rb b/spec/lib/gitlab/github_import/importer/repository_importer_spec.rb index cc9e4b67e72..d8f01dcb76b 100644 --- a/spec/lib/gitlab/github_import/importer/repository_importer_spec.rb +++ b/spec/lib/gitlab/github_import/importer/repository_importer_spec.rb @@ -14,7 +14,8 @@ describe Gitlab::GithubImport::Importer::RepositoryImporter do disk_path: 'foo', repository: repository, create_wiki: true, - import_state: import_state + import_state: import_state, + lfs_enabled?: true ) end diff --git a/spec/lib/gitlab/gpg_spec.rb b/spec/lib/gitlab/gpg_spec.rb index ab9a166db00..47f37cae98f 100644 --- a/spec/lib/gitlab/gpg_spec.rb +++ b/spec/lib/gitlab/gpg_spec.rb @@ -74,6 +74,19 @@ describe Gitlab::Gpg do email: 'nannie.bernhard@example.com' }]) end + + it 'rejects non UTF-8 names and addresses' do + public_key = double(:key) + fingerprints = double(:fingerprints) + email = "\xEEch@test.com".force_encoding('ASCII-8BIT') + uid = double(:uid, name: 'Test User', email: email) + raw_key = double(:raw_key, uids: [uid]) + allow(Gitlab::Gpg::CurrentKeyChain).to receive(:fingerprints_from_key).with(public_key).and_return(fingerprints) + allow(GPGME::Key).to receive(:find).with(:public, anything).and_return([raw_key]) + + user_infos = described_class.user_infos_from_key(public_key) + expect(user_infos).to eq([]) + end end describe '.current_home_dir' do diff --git a/spec/lib/gitlab/import_export/all_models.yml b/spec/lib/gitlab/import_export/all_models.yml index a129855dbd8..2ea66479c1b 100644 --- a/spec/lib/gitlab/import_export/all_models.yml +++ b/spec/lib/gitlab/import_export/all_models.yml @@ -257,6 +257,7 @@ project: - import_data - commit_statuses - pipelines +- stages - builds - runner_projects - runners diff --git a/spec/lib/gitlab/import_export/fork_spec.rb b/spec/lib/gitlab/import_export/fork_spec.rb index 17e06a6a83f..71fd5a51c3b 100644 --- a/spec/lib/gitlab/import_export/fork_spec.rb +++ b/spec/lib/gitlab/import_export/fork_spec.rb @@ -41,8 +41,10 @@ describe 'forked project import' do after do FileUtils.rm_rf(export_path) - FileUtils.rm_rf(project_with_repo.repository.path_to_repo) - FileUtils.rm_rf(project.repository.path_to_repo) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + FileUtils.rm_rf(project_with_repo.repository.path_to_repo) + FileUtils.rm_rf(project.repository.path_to_repo) + end end it 'can access the MR' do diff --git a/spec/lib/gitlab/import_export/repo_restorer_spec.rb b/spec/lib/gitlab/import_export/repo_restorer_spec.rb index dc806d036ff..013b8895f67 100644 --- a/spec/lib/gitlab/import_export/repo_restorer_spec.rb +++ b/spec/lib/gitlab/import_export/repo_restorer_spec.rb @@ -23,8 +23,10 @@ describe Gitlab::ImportExport::RepoRestorer do after do FileUtils.rm_rf(export_path) - FileUtils.rm_rf(project_with_repo.repository.path_to_repo) - FileUtils.rm_rf(project.repository.path_to_repo) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + FileUtils.rm_rf(project_with_repo.repository.path_to_repo) + FileUtils.rm_rf(project.repository.path_to_repo) + end end it 'restores the repo successfully' do @@ -34,7 +36,9 @@ describe Gitlab::ImportExport::RepoRestorer do it 'has the webhooks' do restorer.restore - expect(Gitlab::Git::Hook.new('post-receive', project.repository.raw_repository)).to exist + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + expect(Gitlab::Git::Hook.new('post-receive', project.repository.raw_repository)).to exist + end end end end diff --git a/spec/lib/gitlab/import_export/safe_model_attributes.yml b/spec/lib/gitlab/import_export/safe_model_attributes.yml index 74e7a45fd6c..5b289ceb3b2 100644 --- a/spec/lib/gitlab/import_export/safe_model_attributes.yml +++ b/spec/lib/gitlab/import_export/safe_model_attributes.yml @@ -170,7 +170,7 @@ MergeRequest: - last_edited_by_id - head_pipeline_id - discussion_locked -- allow_maintainer_to_push +- allow_collaboration MergeRequestDiff: - id - state @@ -242,6 +242,7 @@ Ci::Pipeline: - config_source - failure_reason - protected +- iid Ci::Stage: - id - name diff --git a/spec/lib/gitlab/import_sources_spec.rb b/spec/lib/gitlab/import_sources_spec.rb index f2fa315e3ec..10341486512 100644 --- a/spec/lib/gitlab/import_sources_spec.rb +++ b/spec/lib/gitlab/import_sources_spec.rb @@ -91,4 +91,23 @@ describe Gitlab::ImportSources do end end end + + describe 'imports_repository? checker' do + let(:allowed_importers) { %w[github gitlab_project] } + + it 'fails if any importer other than the allowed ones implements this method' do + current_importers = described_class.values.select { |kind| described_class.importer(kind).try(:imports_repository?) } + not_allowed_importers = current_importers - allowed_importers + + expect(not_allowed_importers).to be_empty, failure_message(not_allowed_importers) + end + + def failure_message(importers_class_names) + <<-MSG + It looks like the #{importers_class_names.join(', ')} importers implements its own way to import the repository. + That means that the lfs object download must be handled for each of them. You can use 'LfsImportService' and + 'LfsDownloadService' to implement it. After that, add the importer name to the list of allowed importers in this spec. + MSG + end + end end diff --git a/spec/lib/gitlab/legacy_github_import/project_creator_spec.rb b/spec/lib/gitlab/legacy_github_import/project_creator_spec.rb index eb1b13704ea..972b17d5b12 100644 --- a/spec/lib/gitlab/legacy_github_import/project_creator_spec.rb +++ b/spec/lib/gitlab/legacy_github_import/project_creator_spec.rb @@ -44,16 +44,44 @@ describe Gitlab::LegacyGithubImport::ProjectCreator do end context 'when GitHub project is public' do - before do - allow_any_instance_of(ApplicationSetting).to receive(:default_project_visibility).and_return(Gitlab::VisibilityLevel::INTERNAL) - end - - it 'sets project visibility to the default project visibility' do + it 'sets project visibility to public' do repo.private = false project = service.execute - expect(project.visibility_level).to eq(Gitlab::VisibilityLevel::INTERNAL) + expect(project.visibility_level).to eq(Gitlab::VisibilityLevel::PUBLIC) + end + end + + context 'when visibility level is restricted' do + context 'when GitHub project is private' do + before do + stub_application_setting(restricted_visibility_levels: [Gitlab::VisibilityLevel::PRIVATE]) + allow_any_instance_of(ApplicationSetting).to receive(:default_project_visibility).and_return(Gitlab::VisibilityLevel::INTERNAL) + end + + it 'sets project visibility to the default project visibility' do + repo.private = true + + project = service.execute + + expect(project.visibility_level).to eq(Gitlab::VisibilityLevel::INTERNAL) + end + end + + context 'when GitHub project is public' do + before do + stub_application_setting(restricted_visibility_levels: [Gitlab::VisibilityLevel::PUBLIC]) + allow_any_instance_of(ApplicationSetting).to receive(:default_project_visibility).and_return(Gitlab::VisibilityLevel::INTERNAL) + end + + it 'sets project visibility to the default project visibility' do + repo.private = false + + project = service.execute + + expect(project.visibility_level).to eq(Gitlab::VisibilityLevel::INTERNAL) + end end end diff --git a/spec/lib/gitlab/path_regex_spec.rb b/spec/lib/gitlab/path_regex_spec.rb index a40330d853f..e90e0aba0a4 100644 --- a/spec/lib/gitlab/path_regex_spec.rb +++ b/spec/lib/gitlab/path_regex_spec.rb @@ -90,11 +90,13 @@ describe Gitlab::PathRegex do let(:routes_not_starting_in_wildcard) { routes_without_format.select { |p| p !~ %r{^/[:*]} } } let(:top_level_words) do - words = routes_not_starting_in_wildcard.map do |route| - route.split('/')[1] - end.compact - - (words + ee_top_level_words + files_in_public + Array(API::API.prefix.to_s)).uniq + routes_not_starting_in_wildcard + .map { |route| route.split('/')[1] } + .concat(ee_top_level_words) + .concat(files_in_public) + .concat(Array(API::API.prefix.to_s)) + .compact + .uniq end let(:ee_top_level_words) do diff --git a/spec/lib/gitlab/shell_spec.rb b/spec/lib/gitlab/shell_spec.rb index bf6ee4b0b59..14eae22a2ec 100644 --- a/spec/lib/gitlab/shell_spec.rb +++ b/spec/lib/gitlab/shell_spec.rb @@ -405,7 +405,11 @@ describe Gitlab::Shell do describe '#create_repository' do shared_examples '#create_repository' do let(:repository_storage) { 'default' } - let(:repository_storage_path) { Gitlab.config.repositories.storages[repository_storage].legacy_disk_path } + let(:repository_storage_path) do + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + Gitlab.config.repositories.storages[repository_storage].legacy_disk_path + end + end let(:repo_name) { 'project/path' } let(:created_path) { File.join(repository_storage_path, repo_name + '.git') } diff --git a/spec/lib/gitlab/sql/glob_spec.rb b/spec/lib/gitlab/sql/glob_spec.rb index f0bb4294d62..1cf8935bfe3 100644 --- a/spec/lib/gitlab/sql/glob_spec.rb +++ b/spec/lib/gitlab/sql/glob_spec.rb @@ -35,8 +35,9 @@ describe Gitlab::SQL::Glob do value = query("SELECT #{quote(string)} LIKE #{pattern}") .rows.flatten.first + check = Gitlab.rails5? ? true : 't' case value - when 't', 1 + when check, 1 true else false diff --git a/spec/lib/gitlab/themes_spec.rb b/spec/lib/gitlab/themes_spec.rb index ecacea6bb35..af2f4568017 100644 --- a/spec/lib/gitlab/themes_spec.rb +++ b/spec/lib/gitlab/themes_spec.rb @@ -5,16 +5,16 @@ describe Gitlab::Themes, lib: true do it 'returns a space-separated list of class names' do css = described_class.body_classes - expect(css).to include('ui_indigo') - expect(css).to include(' ui_dark ') - expect(css).to include(' ui_blue') + expect(css).to include('ui-indigo') + expect(css).to include('ui-dark ') + expect(css).to include('ui-blue') end end describe '.by_id' do it 'returns a Theme by its ID' do expect(described_class.by_id(1).name).to eq 'Indigo' - expect(described_class.by_id(3).name).to eq 'Light' + expect(described_class.by_id(10).name).to eq 'Light' end end diff --git a/spec/lib/gitlab/user_access_spec.rb b/spec/lib/gitlab/user_access_spec.rb index 97b6069f64d..0469d984a40 100644 --- a/spec/lib/gitlab/user_access_spec.rb +++ b/spec/lib/gitlab/user_access_spec.rb @@ -142,7 +142,7 @@ describe Gitlab::UserAccess do target_project: canonical_project, source_project: project, source_branch: 'awesome-feature', - allow_maintainer_to_push: true + allow_collaboration: true ) end diff --git a/spec/lib/gitlab/utils/override_spec.rb b/spec/lib/gitlab/utils/override_spec.rb index 7c97cee982a..fc08ebcfc6d 100644 --- a/spec/lib/gitlab/utils/override_spec.rb +++ b/spec/lib/gitlab/utils/override_spec.rb @@ -1,7 +1,13 @@ -require 'spec_helper' +require 'fast_spec_helper' describe Gitlab::Utils::Override do - let(:base) { Struct.new(:good) } + let(:base) do + Struct.new(:good) do + def self.good + 0 + end + end + end let(:derived) { Class.new(base).tap { |m| m.extend described_class } } let(:extension) { Module.new.tap { |m| m.extend described_class } } @@ -9,6 +15,14 @@ describe Gitlab::Utils::Override do let(:prepending_class) { base.tap { |m| m.prepend extension } } let(:including_class) { base.tap { |m| m.include extension } } + let(:prepending_class_methods) do + base.tap { |m| m.singleton_class.prepend extension } + end + + let(:extending_class_methods) do + base.tap { |m| m.extend extension } + end + let(:klass) { subject } def good(mod) @@ -36,7 +50,7 @@ describe Gitlab::Utils::Override do shared_examples 'checking as intended' do it 'checks ok for overriding method' do good(subject) - result = klass.new(0).good + result = instance.good expect(result).to eq(1) described_class.verify! @@ -45,7 +59,25 @@ describe Gitlab::Utils::Override do it 'raises NotImplementedError when it is not overriding anything' do expect do bad(subject) - klass.new(0).bad + instance.bad + described_class.verify! + end.to raise_error(NotImplementedError) + end + end + + shared_examples 'checking as intended, nothing was overridden' do + it 'raises NotImplementedError because it is not overriding it' do + expect do + good(subject) + instance.good + described_class.verify! + end.to raise_error(NotImplementedError) + end + + it 'raises NotImplementedError when it is not overriding anything' do + expect do + bad(subject) + instance.bad described_class.verify! end.to raise_error(NotImplementedError) end @@ -54,7 +86,7 @@ describe Gitlab::Utils::Override do shared_examples 'nothing happened' do it 'does not complain when it is overriding something' do good(subject) - result = klass.new(0).good + result = instance.good expect(result).to eq(1) described_class.verify! @@ -62,7 +94,7 @@ describe Gitlab::Utils::Override do it 'does not complain when it is not overriding anything' do bad(subject) - result = klass.new(0).bad + result = instance.bad expect(result).to eq(true) described_class.verify! @@ -75,83 +107,97 @@ describe Gitlab::Utils::Override do end describe '#override' do - context 'when STATIC_VERIFICATION is set' do - before do - stub_env('STATIC_VERIFICATION', 'true') - end + context 'when instance is klass.new(0)' do + let(:instance) { klass.new(0) } - context 'when subject is a class' do - subject { derived } + context 'when STATIC_VERIFICATION is set' do + before do + stub_env('STATIC_VERIFICATION', 'true') + end - it_behaves_like 'checking as intended' - end + context 'when subject is a class' do + subject { derived } + + it_behaves_like 'checking as intended' + end + + context 'when subject is a module, and class is prepending it' do + subject { extension } + let(:klass) { prepending_class } + + it_behaves_like 'checking as intended' + end - context 'when subject is a module, and class is prepending it' do - subject { extension } - let(:klass) { prepending_class } + context 'when subject is a module, and class is including it' do + subject { extension } + let(:klass) { including_class } - it_behaves_like 'checking as intended' + it_behaves_like 'checking as intended, nothing was overridden' + end end - context 'when subject is a module, and class is including it' do - subject { extension } - let(:klass) { including_class } + context 'when STATIC_VERIFICATION is not set' do + before do + stub_env('STATIC_VERIFICATION', nil) + end - it 'raises NotImplementedError because it is not overriding it' do - expect do - good(subject) - klass.new(0).good - described_class.verify! - end.to raise_error(NotImplementedError) + context 'when subject is a class' do + subject { derived } + + it_behaves_like 'nothing happened' end - it 'raises NotImplementedError when it is not overriding anything' do - expect do - bad(subject) - klass.new(0).bad - described_class.verify! - end.to raise_error(NotImplementedError) + context 'when subject is a module, and class is prepending it' do + subject { extension } + let(:klass) { prepending_class } + + it_behaves_like 'nothing happened' end - end - end - end - context 'when STATIC_VERIFICATION is not set' do - before do - stub_env('STATIC_VERIFICATION', nil) - end + context 'when subject is a module, and class is including it' do + subject { extension } + let(:klass) { including_class } - context 'when subject is a class' do - subject { derived } + it 'does not complain when it is overriding something' do + good(subject) + result = instance.good - it_behaves_like 'nothing happened' - end + expect(result).to eq(0) + described_class.verify! + end - context 'when subject is a module, and class is prepending it' do - subject { extension } - let(:klass) { prepending_class } + it 'does not complain when it is not overriding anything' do + bad(subject) + result = instance.bad - it_behaves_like 'nothing happened' + expect(result).to eq(true) + described_class.verify! + end + end + end end - context 'when subject is a module, and class is including it' do - subject { extension } - let(:klass) { including_class } + context 'when instance is klass' do + let(:instance) { klass } - it 'does not complain when it is overriding something' do - good(subject) - result = klass.new(0).good + context 'when STATIC_VERIFICATION is set' do + before do + stub_env('STATIC_VERIFICATION', 'true') + end - expect(result).to eq(0) - described_class.verify! - end + context 'when subject is a module, and class is prepending it' do + subject { extension } + let(:klass) { prepending_class_methods } - it 'does not complain when it is not overriding anything' do - bad(subject) - result = klass.new(0).bad + it_behaves_like 'checking as intended' + end - expect(result).to eq(true) - described_class.verify! + context 'when subject is a module, and class is extending it' do + subject { extension } + let(:klass) { extending_class_methods } + + it_behaves_like 'checking as intended, nothing was overridden' + end end end end diff --git a/spec/lib/object_storage/direct_upload_spec.rb b/spec/lib/object_storage/direct_upload_spec.rb new file mode 100644 index 00000000000..e0569218d78 --- /dev/null +++ b/spec/lib/object_storage/direct_upload_spec.rb @@ -0,0 +1,168 @@ +require 'spec_helper' + +describe ObjectStorage::DirectUpload do + let(:credentials) do + { + provider: 'AWS', + aws_access_key_id: 'AWS_ACCESS_KEY_ID', + aws_secret_access_key: 'AWS_SECRET_ACCESS_KEY' + } + end + + let(:storage_url) { 'https://uploads.s3.amazonaws.com/' } + + let(:bucket_name) { 'uploads' } + let(:object_name) { 'tmp/uploads/my-file' } + let(:maximum_size) { 1.gigabyte } + + let(:direct_upload) { described_class.new(credentials, bucket_name, object_name, has_length: has_length, maximum_size: maximum_size) } + + before do + Fog.unmock! + end + + describe '#has_length' do + context 'is known' do + let(:has_length) { true } + let(:maximum_size) { nil } + + it "maximum size is not required" do + expect { direct_upload }.not_to raise_error + end + end + + context 'is unknown' do + let(:has_length) { false } + + context 'and maximum size is specified' do + let(:maximum_size) { 1.gigabyte } + + it "does not raise an error" do + expect { direct_upload }.not_to raise_error + end + end + + context 'and maximum size is not specified' do + let(:maximum_size) { nil } + + it "raises an error" do + expect { direct_upload }.to raise_error /maximum_size has to be specified if length is unknown/ + end + end + end + end + + describe '#to_hash' do + subject { direct_upload.to_hash } + + shared_examples 'a valid upload' do + it "returns valid structure" do + expect(subject).to have_key(:Timeout) + expect(subject[:GetURL]).to start_with(storage_url) + expect(subject[:StoreURL]).to start_with(storage_url) + expect(subject[:DeleteURL]).to start_with(storage_url) + end + end + + shared_examples 'a valid upload with multipart data' do + before do + stub_object_storage_multipart_init(storage_url, "myUpload") + end + + it_behaves_like 'a valid upload' + + it "returns valid structure" do + expect(subject).to have_key(:MultipartUpload) + expect(subject[:MultipartUpload]).to have_key(:PartSize) + expect(subject[:MultipartUpload][:PartURLs]).to all(start_with(storage_url)) + expect(subject[:MultipartUpload][:PartURLs]).to all(include('uploadId=myUpload')) + expect(subject[:MultipartUpload][:CompleteURL]).to start_with(storage_url) + expect(subject[:MultipartUpload][:CompleteURL]).to include('uploadId=myUpload') + expect(subject[:MultipartUpload][:AbortURL]).to start_with(storage_url) + expect(subject[:MultipartUpload][:AbortURL]).to include('uploadId=myUpload') + end + end + + shared_examples 'a valid upload without multipart data' do + it_behaves_like 'a valid upload' + + it "returns valid structure" do + expect(subject).not_to have_key(:MultipartUpload) + end + end + + context 'when AWS is used' do + context 'when length is known' do + let(:has_length) { true } + + it_behaves_like 'a valid upload without multipart data' + end + + context 'when length is unknown' do + let(:has_length) { false } + + it_behaves_like 'a valid upload with multipart data' do + context 'when maximum upload size is 10MB' do + let(:maximum_size) { 10.megabyte } + + it 'returns only 2 parts' do + expect(subject[:MultipartUpload][:PartURLs].length).to eq(2) + end + + it 'part size is mimimum, 5MB' do + expect(subject[:MultipartUpload][:PartSize]).to eq(5.megabyte) + end + end + + context 'when maximum upload size is 12MB' do + let(:maximum_size) { 12.megabyte } + + it 'returns only 3 parts' do + expect(subject[:MultipartUpload][:PartURLs].length).to eq(3) + end + + it 'part size is rounded-up to 5MB' do + expect(subject[:MultipartUpload][:PartSize]).to eq(5.megabyte) + end + end + + context 'when maximum upload size is 49GB' do + let(:maximum_size) { 49.gigabyte } + + it 'returns maximum, 100 parts' do + expect(subject[:MultipartUpload][:PartURLs].length).to eq(100) + end + + it 'part size is rounded-up to 5MB' do + expect(subject[:MultipartUpload][:PartSize]).to eq(505.megabyte) + end + end + end + end + end + + context 'when Google is used' do + let(:credentials) do + { + provider: 'Google', + google_storage_access_key_id: 'GOOGLE_ACCESS_KEY_ID', + google_storage_secret_access_key: 'GOOGLE_SECRET_ACCESS_KEY' + } + end + + let(:storage_url) { 'https://storage.googleapis.com/uploads/' } + + context 'when length is known' do + let(:has_length) { true } + + it_behaves_like 'a valid upload without multipart data' + end + + context 'when length is unknown' do + let(:has_length) { false } + + it_behaves_like 'a valid upload without multipart data' + end + end + end +end diff --git a/spec/mailers/notify_spec.rb b/spec/mailers/notify_spec.rb index 69eafbe4bbe..775ca4ba0eb 100644 --- a/spec/mailers/notify_spec.rb +++ b/spec/mailers/notify_spec.rb @@ -68,7 +68,7 @@ describe Notify do end it 'contains the description' do - is_expected.to have_html_escaped_body_text issue.description + is_expected.to have_body_text issue.description end it 'does not add a reason header' do @@ -89,7 +89,7 @@ describe Notify do end it 'contains a link to note author' do - is_expected.to have_html_escaped_body_text(issue.author_name) + is_expected.to have_body_text(issue.author_name) is_expected.to have_body_text 'created an issue:' end end @@ -115,8 +115,8 @@ describe Notify do it 'has the correct subject and body' do aggregate_failures do is_expected.to have_referable_subject(issue, reply: true) - is_expected.to have_html_escaped_body_text(previous_assignee.name) - is_expected.to have_html_escaped_body_text(assignee.name) + is_expected.to have_body_text(previous_assignee.name) + is_expected.to have_body_text(assignee.name) is_expected.to have_body_text(project_issue_path(project, issue)) end end @@ -190,7 +190,7 @@ describe Notify do aggregate_failures do is_expected.to have_referable_subject(issue, reply: true) is_expected.to have_body_text(status) - is_expected.to have_html_escaped_body_text(current_user.name) + is_expected.to have_body_text(current_user.name) is_expected.to have_body_text(project_issue_path project, issue) end end @@ -243,7 +243,7 @@ describe Notify do end it 'contains the description' do - is_expected.to have_html_escaped_body_text merge_request.description + is_expected.to have_body_text merge_request.description end context 'when sent with a reason' do @@ -260,7 +260,7 @@ describe Notify do end it 'contains a link to note author' do - is_expected.to have_html_escaped_body_text merge_request.author_name + is_expected.to have_body_text merge_request.author_name is_expected.to have_body_text 'created a merge request:' end end @@ -286,9 +286,9 @@ describe Notify do it 'has the correct subject and body' do aggregate_failures do is_expected.to have_referable_subject(merge_request, reply: true) - is_expected.to have_html_escaped_body_text(previous_assignee.name) + is_expected.to have_body_text(previous_assignee.name) is_expected.to have_body_text(project_merge_request_path(project, merge_request)) - is_expected.to have_html_escaped_body_text(assignee.name) + is_expected.to have_body_text(assignee.name) end end @@ -358,7 +358,7 @@ describe Notify do aggregate_failures do is_expected.to have_referable_subject(merge_request, reply: true) is_expected.to have_body_text(status) - is_expected.to have_html_escaped_body_text(current_user.name) + is_expected.to have_body_text(current_user.name) is_expected.to have_body_text(project_merge_request_path(project, merge_request)) end end @@ -526,7 +526,7 @@ describe Notify do it 'has the correct subject and body' do is_expected.to have_referable_subject(project_snippet, reply: true) - is_expected.to have_html_escaped_body_text project_snippet_note.note + is_expected.to have_body_text project_snippet_note.note end end @@ -539,7 +539,7 @@ describe Notify do it 'has the correct subject and body' do is_expected.to have_subject("#{project.name} | Project was moved") - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text(project.ssh_url_to_repo) end end @@ -566,7 +566,7 @@ describe Notify do expect(to_emails).to eq([recipient.notification_email]) is_expected.to have_subject "Request to join the #{project.full_name} project" - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project_project_members_url(project) is_expected.to have_body_text project_member.human_access end @@ -586,7 +586,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Access to the #{project.full_name} project was denied" - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project.web_url end end @@ -603,7 +603,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Access to the #{project.full_name} project was granted" - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project.web_url is_expected.to have_body_text project_member.human_access end @@ -633,7 +633,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Invitation to join the #{project.full_name} project" - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project.full_name is_expected.to have_body_text project_member.human_access is_expected.to have_body_text project_member.invite_token @@ -657,10 +657,10 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject 'Invitation accepted' - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project.web_url is_expected.to have_body_text project_member.invite_email - is_expected.to have_html_escaped_body_text invited_user.name + is_expected.to have_body_text invited_user.name end end @@ -680,7 +680,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject 'Invitation declined' - is_expected.to have_html_escaped_body_text project.full_name + is_expected.to have_body_text project.full_name is_expected.to have_body_text project.web_url is_expected.to have_body_text project_member.invite_email end @@ -932,7 +932,7 @@ describe Notify do end it 'contains the message from the note' do - is_expected.to have_html_escaped_body_text note.note + is_expected.to have_body_text note.note end it 'contains an introduction' do @@ -991,7 +991,7 @@ describe Notify do expect(to_emails).to eq([recipient.notification_email]) is_expected.to have_subject "Request to join the #{group.name} group" - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group_group_members_url(group) is_expected.to have_body_text group_member.human_access end @@ -1010,7 +1010,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Access to the #{group.name} group was denied" - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group.web_url end end @@ -1026,7 +1026,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Access to the #{group.name} group was granted" - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group.web_url is_expected.to have_body_text group_member.human_access end @@ -1056,7 +1056,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject "Invitation to join the #{group.name} group" - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group.web_url is_expected.to have_body_text group_member.human_access is_expected.to have_body_text group_member.invite_token @@ -1080,10 +1080,10 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject 'Invitation accepted' - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group.web_url is_expected.to have_body_text group_member.invite_email - is_expected.to have_html_escaped_body_text invited_user.name + is_expected.to have_body_text invited_user.name end end @@ -1103,7 +1103,7 @@ describe Notify do it 'contains all the useful information' do is_expected.to have_subject 'Invitation declined' - is_expected.to have_html_escaped_body_text group.name + is_expected.to have_body_text group.name is_expected.to have_body_text group.web_url is_expected.to have_body_text group_member.invite_email end @@ -1396,7 +1396,7 @@ describe Notify do it 'has the correct subject and body' do is_expected.to have_referable_subject(personal_snippet, reply: true) - is_expected.to have_html_escaped_body_text personal_snippet_note.note + is_expected.to have_body_text personal_snippet_note.note end end end diff --git a/spec/migrations/change_default_value_for_dsa_key_restriction_spec.rb b/spec/migrations/change_default_value_for_dsa_key_restriction_spec.rb new file mode 100644 index 00000000000..7e61ab9b52e --- /dev/null +++ b/spec/migrations/change_default_value_for_dsa_key_restriction_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' +require Rails.root.join('db', 'migrate', '20180531220618_change_default_value_for_dsa_key_restriction.rb') + +describe ChangeDefaultValueForDsaKeyRestriction, :migration do + let(:application_settings) { table(:application_settings) } + + before do + application_settings.create! + end + + it 'changes the default value for dsa_key_restriction' do + expect(application_settings.first.dsa_key_restriction).to eq(0) + + migrate! + + application_settings.reset_column_information + new_setting = application_settings.create! + + expect(application_settings.count).to eq(2) + expect(new_setting.dsa_key_restriction).to eq(-1) + end + + it 'changes the existing setting' do + setting = application_settings.last + + expect(setting.dsa_key_restriction).to eq(0) + + migrate! + + expect(application_settings.count).to eq(1) + expect(setting.reload.dsa_key_restriction).to eq(-1) + end +end diff --git a/spec/migrations/migrate_object_storage_upload_sidekiq_queue_spec.rb b/spec/migrations/migrate_object_storage_upload_sidekiq_queue_spec.rb new file mode 100644 index 00000000000..1ee6c440cf4 --- /dev/null +++ b/spec/migrations/migrate_object_storage_upload_sidekiq_queue_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' +require Rails.root.join('db', 'post_migrate', '20180603190921_migrate_object_storage_upload_sidekiq_queue.rb') + +describe MigrateObjectStorageUploadSidekiqQueue, :sidekiq, :redis do + include Gitlab::Database::MigrationHelpers + + context 'when there are jobs in the queue' do + it 'correctly migrates queue when migrating up' do + Sidekiq::Testing.disable! do + stubbed_worker(queue: 'object_storage_upload').perform_async('Something', [1]) + stubbed_worker(queue: 'object_storage:object_storage_background_move').perform_async('Something', [1]) + + described_class.new.up + + expect(sidekiq_queue_length('object_storage_upload')).to eq 0 + expect(sidekiq_queue_length('object_storage:object_storage_background_move')).to eq 2 + end + end + end + + context 'when there are no jobs in the queues' do + it 'does not raise error when migrating up' do + expect { described_class.new.up }.not_to raise_error + end + end + + def stubbed_worker(queue:) + Class.new do + include Sidekiq::Worker + sidekiq_options queue: queue + end + end +end diff --git a/spec/migrations/schedule_to_archive_legacy_traces_spec.rb b/spec/migrations/schedule_to_archive_legacy_traces_spec.rb new file mode 100644 index 00000000000..d3eac3c45ea --- /dev/null +++ b/spec/migrations/schedule_to_archive_legacy_traces_spec.rb @@ -0,0 +1,45 @@ +require 'spec_helper' +require Rails.root.join('db', 'post_migrate', '20180529152628_schedule_to_archive_legacy_traces') + +describe ScheduleToArchiveLegacyTraces, :migration do + include TraceHelpers + + let(:namespaces) { table(:namespaces) } + let(:projects) { table(:projects) } + let(:builds) { table(:ci_builds) } + let(:job_artifacts) { table(:ci_job_artifacts) } + + before do + namespaces.create!(id: 123, name: 'gitlab1', path: 'gitlab1') + projects.create!(id: 123, name: 'gitlab1', path: 'gitlab1', namespace_id: 123) + @build_success = builds.create!(id: 1, project_id: 123, status: 'success', type: 'Ci::Build') + @build_failed = builds.create!(id: 2, project_id: 123, status: 'failed', type: 'Ci::Build') + @builds_canceled = builds.create!(id: 3, project_id: 123, status: 'canceled', type: 'Ci::Build') + @build_running = builds.create!(id: 4, project_id: 123, status: 'running', type: 'Ci::Build') + + create_legacy_trace(@build_success, 'This job is done') + create_legacy_trace(@build_failed, 'This job is done') + create_legacy_trace(@builds_canceled, 'This job is done') + create_legacy_trace(@build_running, 'This job is not done yet') + end + + it 'correctly archive legacy traces' do + expect(job_artifacts.count).to eq(0) + expect(File.exist?(legacy_trace_path(@build_success))).to be_truthy + expect(File.exist?(legacy_trace_path(@build_failed))).to be_truthy + expect(File.exist?(legacy_trace_path(@builds_canceled))).to be_truthy + expect(File.exist?(legacy_trace_path(@build_running))).to be_truthy + + migrate! + + expect(job_artifacts.count).to eq(3) + expect(File.exist?(legacy_trace_path(@build_success))).to be_falsy + expect(File.exist?(legacy_trace_path(@build_failed))).to be_falsy + expect(File.exist?(legacy_trace_path(@builds_canceled))).to be_falsy + expect(File.exist?(legacy_trace_path(@build_running))).to be_truthy + expect(File.exist?(archived_trace_path(job_artifacts.where(job_id: @build_success.id).first))).to be_truthy + expect(File.exist?(archived_trace_path(job_artifacts.where(job_id: @build_failed.id).first))).to be_truthy + expect(File.exist?(archived_trace_path(job_artifacts.where(job_id: @builds_canceled.id).first))).to be_truthy + expect(job_artifacts.where(job_id: @build_running.id)).not_to be_exist + end +end diff --git a/spec/models/application_setting/term_spec.rb b/spec/models/application_setting/term_spec.rb index 1eddf3c56ff..aa49594f4d1 100644 --- a/spec/models/application_setting/term_spec.rb +++ b/spec/models/application_setting/term_spec.rb @@ -12,4 +12,41 @@ describe ApplicationSetting::Term do expect(described_class.latest).to eq(terms) end end + + describe '#accepted_by_user?' do + let(:user) { create(:user) } + let(:term) { create(:term) } + + it 'is true when the user accepted the terms' do + accept_terms(term, user) + + expect(term.accepted_by_user?(user)).to be(true) + end + + it 'is false when the user declined the terms' do + decline_terms(term, user) + + expect(term.accepted_by_user?(user)).to be(false) + end + + it 'does not cause a query when the user accepted the current terms' do + accept_terms(term, user) + + expect { term.accepted_by_user?(user) }.not_to exceed_query_limit(0) + end + + it 'returns false if the currently accepted terms are different' do + accept_terms(create(:term), user) + + expect(term.accepted_by_user?(user)).to be(false) + end + + def accept_terms(term, user) + Users::RespondToTermsService.new(user, term).execute(accepted: true) + end + + def decline_terms(term, user) + Users::RespondToTermsService.new(user, term).execute(accepted: false) + end + end end diff --git a/spec/models/application_setting_spec.rb b/spec/models/application_setting_spec.rb index 968267a6d24..3e6656e0f12 100644 --- a/spec/models/application_setting_spec.rb +++ b/spec/models/application_setting_spec.rb @@ -110,10 +110,9 @@ describe ApplicationSetting do # Upgraded databases will have this sort of content context 'repository_storages is a String, not an Array' do before do - setting.__send__(:raw_write_attribute, :repository_storages, 'default') + described_class.where(id: setting.id).update_all(repository_storages: 'default') end - it { expect(setting.repository_storages_before_type_cast).to eq('default') } it { expect(setting.repository_storages).to eq(['default']) } end diff --git a/spec/models/ci/build_spec.rb b/spec/models/ci/build_spec.rb index b5eac913639..0a0d7d3fea9 100644 --- a/spec/models/ci/build_spec.rb +++ b/spec/models/ci/build_spec.rb @@ -116,6 +116,26 @@ describe Ci::Build do end end + describe '.with_live_trace' do + subject { described_class.with_live_trace } + + context 'when build has live trace' do + let!(:build) { create(:ci_build, :success, :trace_live) } + + it 'selects the build' do + is_expected.to eq([build]) + end + end + + context 'when build does not have live trace' do + let!(:build) { create(:ci_build, :success, :trace_artifact) } + + it 'does not select the build' do + is_expected.to be_empty + end + end + end + describe '#actionize' do context 'when build is a created' do before do @@ -1559,6 +1579,7 @@ describe Ci::Build do { key: 'CI_PROJECT_NAMESPACE', value: project.namespace.full_path, public: true }, { key: 'CI_PROJECT_URL', value: project.web_url, public: true }, { key: 'CI_PROJECT_VISIBILITY', value: 'private', public: true }, + { key: 'CI_PIPELINE_IID', value: pipeline.iid.to_s, public: true }, { key: 'CI_CONFIG_PATH', value: pipeline.ci_yaml_file_path, public: true }, { key: 'CI_PIPELINE_SOURCE', value: pipeline.source, public: true }, { key: 'CI_COMMIT_MESSAGE', value: pipeline.git_commit_message, public: true }, @@ -2505,4 +2526,76 @@ describe Ci::Build do end end end + + describe 'pages deployments' do + set(:build) { create(:ci_build, project: project, user: user) } + + context 'when job is "pages"' do + before do + build.name = 'pages' + end + + context 'when pages are enabled' do + before do + allow(Gitlab.config.pages).to receive_messages(enabled: true) + end + + it 'is marked as pages generator' do + expect(build).to be_pages_generator + end + + context 'job succeeds' do + it "calls pages worker" do + expect(PagesWorker).to receive(:perform_async).with(:deploy, build.id) + + build.success! + end + end + + context 'job fails' do + it "does not call pages worker" do + expect(PagesWorker).not_to receive(:perform_async) + + build.drop! + end + end + end + + context 'when pages are disabled' do + before do + allow(Gitlab.config.pages).to receive_messages(enabled: false) + end + + it 'is not marked as pages generator' do + expect(build).not_to be_pages_generator + end + + context 'job succeeds' do + it "does not call pages worker" do + expect(PagesWorker).not_to receive(:perform_async) + + build.success! + end + end + end + end + + context 'when job is not "pages"' do + before do + build.name = 'other-job' + end + + it 'is not marked as pages generator' do + expect(build).not_to be_pages_generator + end + + context 'job succeeds' do + it "does not call pages worker" do + expect(PagesWorker).not_to receive(:perform_async) + + build.success + end + end + end + end end diff --git a/spec/models/ci/group_spec.rb b/spec/models/ci/group_spec.rb index 51123e73fe6..838fa63cb1f 100644 --- a/spec/models/ci/group_spec.rb +++ b/spec/models/ci/group_spec.rb @@ -41,4 +41,55 @@ describe Ci::Group do end end end + + describe '.fabricate' do + let(:pipeline) { create(:ci_empty_pipeline) } + let(:stage) { create(:ci_stage_entity, pipeline: pipeline) } + + before do + create_build(:ci_build, name: 'rspec 0 2') + create_build(:ci_build, name: 'rspec 0 1') + create_build(:ci_build, name: 'spinach 0 1') + create_build(:commit_status, name: 'aaaaa') + end + + it 'returns an array of three groups' do + expect(stage.groups).to be_a Array + expect(stage.groups).to all(be_a described_class) + expect(stage.groups.size).to eq 3 + end + + it 'returns groups with correctly ordered statuses' do + expect(stage.groups.first.jobs.map(&:name)) + .to eq ['aaaaa'] + expect(stage.groups.second.jobs.map(&:name)) + .to eq ['rspec 0 1', 'rspec 0 2'] + expect(stage.groups.third.jobs.map(&:name)) + .to eq ['spinach 0 1'] + end + + it 'returns groups with correct names' do + expect(stage.groups.map(&:name)) + .to eq %w[aaaaa rspec spinach] + end + + context 'when a name is nil on legacy pipelines' do + before do + pipeline.builds.first.update_attribute(:name, nil) + end + + it 'returns an array of three groups' do + expect(stage.groups.map(&:name)) + .to eq ['', 'aaaaa', 'rspec', 'spinach'] + end + end + + def create_build(type, status: 'success', **opts) + create(type, pipeline: pipeline, + stage: stage.name, + status: status, + stage_id: stage.id, + **opts) + end + end end diff --git a/spec/models/ci/pipeline_spec.rb b/spec/models/ci/pipeline_spec.rb index f5295bec65b..2bae98dcbb8 100644 --- a/spec/models/ci/pipeline_spec.rb +++ b/spec/models/ci/pipeline_spec.rb @@ -35,6 +35,16 @@ describe Ci::Pipeline, :mailer do end end + describe 'modules' do + it_behaves_like 'AtomicInternalId', validate_presence: false do + let(:internal_id_attribute) { :iid } + let(:instance) { build(:ci_pipeline) } + let(:scope) { :project } + let(:scope_attrs) { { project: instance.project } } + let(:usage) { :ci_pipelines } + end + end + describe '#source' do context 'when creating new pipeline' do let(:pipeline) do @@ -195,7 +205,8 @@ describe Ci::Pipeline, :mailer do it 'includes all predefined variables in a valid order' do keys = subject.map { |variable| variable[:key] } - expect(keys).to eq %w[CI_CONFIG_PATH + expect(keys).to eq %w[CI_PIPELINE_IID + CI_CONFIG_PATH CI_PIPELINE_SOURCE CI_COMMIT_MESSAGE CI_COMMIT_TITLE @@ -526,6 +537,87 @@ describe Ci::Pipeline, :mailer do end end end + + describe '#stages' do + before do + create(:ci_stage_entity, project: project, + pipeline: pipeline, + name: 'build') + end + + it 'returns persisted stages' do + expect(pipeline.stages).not_to be_empty + expect(pipeline.stages).to all(be_persisted) + end + end + + describe '#ordered_stages' do + before do + create(:ci_stage_entity, project: project, + pipeline: pipeline, + position: 4, + name: 'deploy') + + create(:ci_build, project: project, + pipeline: pipeline, + stage: 'test', + stage_idx: 3, + name: 'test') + + create(:ci_build, project: project, + pipeline: pipeline, + stage: 'build', + stage_idx: 2, + name: 'build') + + create(:ci_stage_entity, project: project, + pipeline: pipeline, + position: 1, + name: 'sanity') + + create(:ci_stage_entity, project: project, + pipeline: pipeline, + position: 5, + name: 'cleanup') + end + + subject { pipeline.ordered_stages } + + context 'when using legacy stages' do + before do + stub_feature_flags(ci_pipeline_persisted_stages: false) + end + + it 'returns legacy stages in valid order' do + expect(subject.map(&:name)).to eq %w[build test] + end + end + + context 'when using persisted stages' do + before do + stub_feature_flags(ci_pipeline_persisted_stages: true) + end + + context 'when pipelines is not complete' do + it 'still returns legacy stages' do + expect(subject).to all(be_a Ci::LegacyStage) + expect(subject.map(&:name)).to eq %w[build test] + end + end + + context 'when pipeline is complete' do + before do + pipeline.succeed! + end + + it 'returns stages in valid order' do + expect(subject).to all(be_a Ci::Stage) + expect(subject.map(&:name)) + .to eq %w[sanity build test deploy cleanup] + end + end + end + end end describe 'state machine' do @@ -1170,6 +1262,43 @@ describe Ci::Pipeline, :mailer do end end + describe '#update_status' do + context 'when pipeline is empty' do + it 'updates does not change pipeline status' do + expect(pipeline.statuses.latest.status).to be_nil + + expect { pipeline.update_status } + .to change { pipeline.reload.status }.to 'skipped' + end + end + + context 'when updating status to pending' do + before do + allow(pipeline) + .to receive_message_chain(:statuses, :latest, :status) + .and_return(:running) + end + + it 'updates pipeline status to running' do + expect { pipeline.update_status } + .to change { pipeline.reload.status }.to 'running' + end + end + + context 'when statuses status was not recognized' do + before do + allow(pipeline) + .to receive(:latest_builds_status) + .and_return(:unknown) + end + + it 'raises an exception' do + expect { pipeline.update_status } + .to raise_error(HasStatus::UnknownStatusError) + end + end + end + describe '#detailed_status' do subject { pipeline.detailed_status(user) } diff --git a/spec/models/ci/runner_spec.rb b/spec/models/ci/runner_spec.rb index 0f072aa1719..f6433234573 100644 --- a/spec/models/ci/runner_spec.rb +++ b/spec/models/ci/runner_spec.rb @@ -549,7 +549,7 @@ describe Ci::Runner do end describe '#update_cached_info' do - let(:runner) { create(:ci_runner) } + let(:runner) { create(:ci_runner, :project) } subject { runner.update_cached_info(architecture: '18-bit') } @@ -570,17 +570,22 @@ describe Ci::Runner do runner.contacted_at = 2.hours.ago end - it 'updates database' do - expect_redis_update + context 'with invalid runner' do + before do + runner.projects = [] + end + + it 'still updates redis cache and database' do + expect(runner).to be_invalid - expect { subject }.to change { runner.reload.read_attribute(:contacted_at) } - .and change { runner.reload.read_attribute(:architecture) } + expect_redis_update + does_db_update + end end - it 'updates cache' do + it 'updates redis cache and database' do expect_redis_update - - subject + does_db_update end end @@ -590,6 +595,11 @@ describe Ci::Runner do expect(redis).to receive(:set).with(redis_key, anything, any_args) end end + + def does_db_update + expect { subject }.to change { runner.reload.read_attribute(:contacted_at) } + .and change { runner.reload.read_attribute(:architecture) } + end end describe '#destroy' do diff --git a/spec/models/ci/stage_spec.rb b/spec/models/ci/stage_spec.rb index a00db1d2bfc..22a4556c10c 100644 --- a/spec/models/ci/stage_spec.rb +++ b/spec/models/ci/stage_spec.rb @@ -65,7 +65,31 @@ describe Ci::Stage, :models do end end - context 'when stage is skipped' do + context 'when stage has only created builds' do + let(:stage) { create(:ci_stage_entity, status: :created) } + + before do + create(:ci_build, :created, stage_id: stage.id) + end + + it 'updates status to skipped' do + expect(stage.reload.status).to eq 'created' + end + end + + context 'when stage is skipped because of skipped builds' do + before do + create(:ci_build, :skipped, stage_id: stage.id) + end + + it 'updates status to skipped' do + expect { stage.update_status } + .to change { stage.reload.status } + .to 'skipped' + end + end + + context 'when stage is skipped because is empty' do it 'updates status to skipped' do expect { stage.update_status } .to change { stage.reload.status } @@ -86,9 +110,85 @@ describe Ci::Stage, :models do expect(stage.reload).to be_failed end end + + context 'when statuses status was not recognized' do + before do + allow(stage) + .to receive_message_chain(:statuses, :latest, :status) + .and_return(:unknown) + end + + it 'raises an exception' do + expect { stage.update_status } + .to raise_error(HasStatus::UnknownStatusError) + end + end + end + + describe '#detailed_status' do + using RSpec::Parameterized::TableSyntax + + let(:user) { create(:user) } + let(:stage) { create(:ci_stage_entity, status: :created) } + subject { stage.detailed_status(user) } + + where(:statuses, :label) do + %w[created] | :created + %w[success] | :passed + %w[pending] | :pending + %w[skipped] | :skipped + %w[canceled] | :canceled + %w[success failed] | :failed + %w[running pending] | :running + end + + with_them do + before do + statuses.each do |status| + create(:commit_status, project: stage.project, + pipeline: stage.pipeline, + stage_id: stage.id, + status: status) + + stage.update_status + end + end + + it 'has a correct label' do + expect(subject.label).to eq label.to_s + end + end + + context 'when stage has warnings' do + before do + create(:ci_build, project: stage.project, + pipeline: stage.pipeline, + stage_id: stage.id, + status: :failed, + allow_failure: true) + + stage.update_status + end + + it 'is passed with warnings' do + expect(subject.label).to eq 'passed with warnings' + end + end end - describe '#index' do + describe '#groups' do + before do + create(:ci_build, stage_id: stage.id, name: 'rspec 0 1') + create(:ci_build, stage_id: stage.id, name: 'rspec 0 2') + end + + it 'groups stage builds by name' do + expect(stage.groups).to be_one + expect(stage.groups.first.name).to eq 'rspec' + end + end + + describe '#position' do context 'when stage has been imported and does not have position index set' do before do stage.update_column(:position, nil) @@ -119,4 +219,42 @@ describe Ci::Stage, :models do end end end + + context 'when stage has warnings' do + before do + create(:ci_build, :failed, :allowed_to_fail, stage_id: stage.id) + end + + describe '#has_warnings?' do + it 'returns true' do + expect(stage).to have_warnings + end + end + + describe '#number_of_warnings' do + it 'returns a lazy stage warnings counter' do + lazy_queries = ActiveRecord::QueryRecorder.new do + stage.number_of_warnings + end + + synced_queries = ActiveRecord::QueryRecorder.new do + stage.number_of_warnings.to_i + end + + expect(lazy_queries.count).to eq 0 + expect(synced_queries.count).to eq 1 + + expect(stage.number_of_warnings.inspect).to include 'BatchLoader' + expect(stage.number_of_warnings).to eq 1 + end + end + end + + context 'when stage does not have warnings' do + describe '#has_warnings?' do + it 'returns false' do + expect(stage).not_to have_warnings + end + end + end end diff --git a/spec/models/deployment_spec.rb b/spec/models/deployment_spec.rb index aee70bcfb29..e01906f4b6c 100644 --- a/spec/models/deployment_spec.rb +++ b/spec/models/deployment_spec.rb @@ -20,6 +20,7 @@ describe Deployment do it_behaves_like 'AtomicInternalId' do let(:internal_id_attribute) { :iid } let(:instance) { build(:deployment) } + let(:scope) { :project } let(:scope_attrs) { { project: instance.project } } let(:usage) { :deployments } end diff --git a/spec/models/issue_spec.rb b/spec/models/issue_spec.rb index 128acf83686..e818fbeb9cf 100644 --- a/spec/models/issue_spec.rb +++ b/spec/models/issue_spec.rb @@ -17,6 +17,7 @@ describe Issue do it_behaves_like 'AtomicInternalId' do let(:internal_id_attribute) { :iid } let(:instance) { build(:issue) } + let(:scope) { :project } let(:scope_attrs) { { project: instance.project } } let(:usage) { :issues } end diff --git a/spec/models/merge_request_spec.rb b/spec/models/merge_request_spec.rb index 9ffa91fc265..3f028b3bd5c 100644 --- a/spec/models/merge_request_spec.rb +++ b/spec/models/merge_request_spec.rb @@ -16,7 +16,11 @@ describe MergeRequest do describe '#squash_in_progress?' do shared_examples 'checking whether a squash is in progress' do - let(:repo_path) { subject.source_project.repository.path } + let(:repo_path) do + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + subject.source_project.repository.path + end + end let(:squash_path) { File.join(repo_path, "gitlab-worktree", "squash-#{subject.id}") } before do @@ -84,6 +88,7 @@ describe MergeRequest do it_behaves_like 'AtomicInternalId' do let(:internal_id_attribute) { :iid } let(:instance) { build(:merge_request) } + let(:scope) { :target_project } let(:scope_attrs) { { project: instance.target_project } } let(:usage) { :merge_requests } end @@ -2196,7 +2201,11 @@ describe MergeRequest do describe '#rebase_in_progress?' do shared_examples 'checking whether a rebase is in progress' do - let(:repo_path) { subject.source_project.repository.path } + let(:repo_path) do + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + subject.source_project.repository.path + end + end let(:rebase_path) { File.join(repo_path, "gitlab-worktree", "rebase-#{subject.id}") } before do @@ -2236,25 +2245,25 @@ describe MergeRequest do end end - describe '#allow_maintainer_to_push' do + describe '#allow_collaboration' do let(:merge_request) do - build(:merge_request, source_branch: 'fixes', allow_maintainer_to_push: true) + build(:merge_request, source_branch: 'fixes', allow_collaboration: true) end it 'is false when pushing by a maintainer is not possible' do - expect(merge_request).to receive(:maintainer_push_possible?) { false } + expect(merge_request).to receive(:collaborative_push_possible?) { false } - expect(merge_request.allow_maintainer_to_push).to be_falsy + expect(merge_request.allow_collaboration).to be_falsy end it 'is true when pushing by a maintainer is possible' do - expect(merge_request).to receive(:maintainer_push_possible?) { true } + expect(merge_request).to receive(:collaborative_push_possible?) { true } - expect(merge_request.allow_maintainer_to_push).to be_truthy + expect(merge_request.allow_collaboration).to be_truthy end end - describe '#maintainer_push_possible?' do + describe '#collaborative_push_possible?' do let(:merge_request) do build(:merge_request, source_branch: 'fixes') end @@ -2266,14 +2275,14 @@ describe MergeRequest do it 'does not allow maintainer to push if the source project is the same as the target' do merge_request.target_project = merge_request.source_project = create(:project, :public) - expect(merge_request.maintainer_push_possible?).to be_falsy + expect(merge_request.collaborative_push_possible?).to be_falsy end it 'allows maintainer to push when both source and target are public' do merge_request.target_project = build(:project, :public) merge_request.source_project = build(:project, :public) - expect(merge_request.maintainer_push_possible?).to be_truthy + expect(merge_request.collaborative_push_possible?).to be_truthy end it 'is not available for protected branches' do @@ -2284,11 +2293,11 @@ describe MergeRequest do .with(merge_request.source_project, 'fixes') .and_return(true) - expect(merge_request.maintainer_push_possible?).to be_falsy + expect(merge_request.collaborative_push_possible?).to be_falsy end end - describe '#can_allow_maintainer_to_push?' do + describe '#can_allow_collaboration?' do let(:target_project) { create(:project, :public) } let(:source_project) { fork_project(target_project) } let(:merge_request) do @@ -2300,17 +2309,17 @@ describe MergeRequest do let(:user) { create(:user) } before do - allow(merge_request).to receive(:maintainer_push_possible?) { true } + allow(merge_request).to receive(:collaborative_push_possible?) { true } end it 'is false if the user does not have push access to the source project' do - expect(merge_request.can_allow_maintainer_to_push?(user)).to be_falsy + expect(merge_request.can_allow_collaboration?(user)).to be_falsy end it 'is true when the user has push access to the source project' do source_project.add_developer(user) - expect(merge_request.can_allow_maintainer_to_push?(user)).to be_truthy + expect(merge_request.can_allow_collaboration?(user)).to be_truthy end end diff --git a/spec/models/milestone_spec.rb b/spec/models/milestone_spec.rb index 4bb9717d33e..204d6b47832 100644 --- a/spec/models/milestone_spec.rb +++ b/spec/models/milestone_spec.rb @@ -6,6 +6,7 @@ describe Milestone do it_behaves_like 'AtomicInternalId' do let(:internal_id_attribute) { :iid } let(:instance) { build(:milestone, project: build(:project), group: nil) } + let(:scope) { :project } let(:scope_attrs) { { project: instance.project } } let(:usage) { :milestones } end @@ -15,6 +16,7 @@ describe Milestone do it_behaves_like 'AtomicInternalId' do let(:internal_id_attribute) { :iid } let(:instance) { build(:milestone, project: nil, group: build(:group)) } + let(:scope) { :group } let(:scope_attrs) { { namespace: instance.group } } let(:usage) { :milestones } end diff --git a/spec/models/namespace_spec.rb b/spec/models/namespace_spec.rb index 6f702d8d95e..18b01c3e6b7 100644 --- a/spec/models/namespace_spec.rb +++ b/spec/models/namespace_spec.rb @@ -301,12 +301,18 @@ describe Namespace do end def project_rugged(project) - project.repository.rugged + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + project.repository.rugged + end end end describe '#rm_dir', 'callback' do - let(:repository_storage_path) { Gitlab.config.repositories.storages.default.legacy_disk_path } + let(:repository_storage_path) do + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + Gitlab.config.repositories.storages.default.legacy_disk_path + end + end let(:path_in_dir) { File.join(repository_storage_path, namespace.full_path) } let(:deleted_path) { namespace.full_path.gsub(namespace.path, "#{namespace.full_path}+#{namespace.id}+deleted") } let(:deleted_path_in_dir) { File.join(repository_storage_path, deleted_path) } diff --git a/spec/models/project_spec.rb b/spec/models/project_spec.rb index 9a76452a808..fe9d64c0e3b 100644 --- a/spec/models/project_spec.rb +++ b/spec/models/project_spec.rb @@ -3583,7 +3583,7 @@ describe Project do target_branch: 'target-branch', source_project: project, source_branch: 'awesome-feature-1', - allow_maintainer_to_push: true + allow_collaboration: true ) end @@ -3620,9 +3620,9 @@ describe Project do end end - describe '#branch_allows_maintainer_push?' do + describe '#branch_allows_collaboration_push?' do it 'allows access if the user can merge the merge request' do - expect(project.branch_allows_maintainer_push?(user, 'awesome-feature-1')) + expect(project.branch_allows_collaboration?(user, 'awesome-feature-1')) .to be_truthy end @@ -3630,7 +3630,7 @@ describe Project do guest = create(:user) target_project.add_guest(guest) - expect(project.branch_allows_maintainer_push?(guest, 'awesome-feature-1')) + expect(project.branch_allows_collaboration?(guest, 'awesome-feature-1')) .to be_falsy end @@ -3640,31 +3640,31 @@ describe Project do target_branch: 'target-branch', source_project: project, source_branch: 'rejected-feature-1', - allow_maintainer_to_push: true) + allow_collaboration: true) - expect(project.branch_allows_maintainer_push?(user, 'rejected-feature-1')) + expect(project.branch_allows_collaboration?(user, 'rejected-feature-1')) .to be_falsy end it 'does not allow access if the user cannot merge the merge request' do create(:protected_branch, :masters_can_push, project: target_project, name: 'target-branch') - expect(project.branch_allows_maintainer_push?(user, 'awesome-feature-1')) + expect(project.branch_allows_collaboration?(user, 'awesome-feature-1')) .to be_falsy end it 'caches the result' do - control = ActiveRecord::QueryRecorder.new { project.branch_allows_maintainer_push?(user, 'awesome-feature-1') } + control = ActiveRecord::QueryRecorder.new { project.branch_allows_collaboration?(user, 'awesome-feature-1') } - expect { 3.times { project.branch_allows_maintainer_push?(user, 'awesome-feature-1') } } + expect { 3.times { project.branch_allows_collaboration?(user, 'awesome-feature-1') } } .not_to exceed_query_limit(control) end context 'when the requeststore is active', :request_store do it 'only queries per project across instances' do - control = ActiveRecord::QueryRecorder.new { project.branch_allows_maintainer_push?(user, 'awesome-feature-1') } + control = ActiveRecord::QueryRecorder.new { project.branch_allows_collaboration?(user, 'awesome-feature-1') } - expect { 2.times { described_class.find(project.id).branch_allows_maintainer_push?(user, 'awesome-feature-1') } } + expect { 2.times { described_class.find(project.id).branch_allows_collaboration?(user, 'awesome-feature-1') } } .not_to exceed_query_limit(control).with_threshold(2) end end diff --git a/spec/models/project_team_spec.rb b/spec/models/project_team_spec.rb index e07c522800a..9978f3e9566 100644 --- a/spec/models/project_team_spec.rb +++ b/spec/models/project_team_spec.rb @@ -179,14 +179,14 @@ describe ProjectTeam do end describe "#human_max_access" do - it 'returns Master role' do + it 'returns Maintainer role' do user = create(:user) group = create(:group) project = create(:project, namespace: group) group.add_master(user) - expect(project.team.human_max_access(user.id)).to eq 'Master' + expect(project.team.human_max_access(user.id)).to eq 'Maintainer' end it 'returns Owner role' do diff --git a/spec/models/repository_spec.rb b/spec/models/repository_spec.rb index 0ccf55bd895..7c0a1cd967c 100644 --- a/spec/models/repository_spec.rb +++ b/spec/models/repository_spec.rb @@ -136,7 +136,10 @@ describe Repository do before do options = { message: 'test tag message\n', tagger: { name: 'John Smith', email: 'john@gmail.com' } } - repository.rugged.tags.create(annotated_tag_name, 'a48e4fc218069f68ef2e769dd8dfea3991362175', options) + + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + repository.rugged.tags.create(annotated_tag_name, 'a48e4fc218069f68ef2e769dd8dfea3991362175', options) + end double_first = double(committed_date: Time.now - 1.second) double_last = double(committed_date: Time.now) @@ -1048,6 +1051,13 @@ describe Repository do let(:target_project) { project } let(:target_repository) { target_project.repository } + around do |example| + # TODO Gitlab::Git::OperationService will be moved to gitaly-ruby and disappear from this repo + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + example.run + end + end + context 'when pre hooks were successful' do before do service = Gitlab::Git::HooksService.new @@ -1309,6 +1319,13 @@ describe Repository do end describe '#update_autocrlf_option' do + around do |example| + # TODO Gitlab::Git::OperationService will be moved to gitaly-ruby and disappear from this repo + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + example.run + end + end + describe 'when autocrlf is not already set to :input' do before do repository.raw_repository.autocrlf = true @@ -1802,7 +1819,9 @@ describe Repository do expect(repository.branch_count).to be_an(Integer) # NOTE: Until rugged goes away, make sure rugged and gitaly are in sync - rugged_count = repository.raw_repository.rugged.branches.count + rugged_count = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + repository.raw_repository.rugged.branches.count + end expect(repository.branch_count).to eq(rugged_count) end @@ -1813,7 +1832,9 @@ describe Repository do expect(repository.tag_count).to be_an(Integer) # NOTE: Until rugged goes away, make sure rugged and gitaly are in sync - rugged_count = repository.raw_repository.rugged.tags.count + rugged_count = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + repository.raw_repository.rugged.tags.count + end expect(repository.tag_count).to eq(rugged_count) end @@ -2073,7 +2094,10 @@ describe Repository do it "attempting to call keep_around on truncated ref does not fail" do repository.keep_around(sample_commit.id) ref = repository.send(:keep_around_ref_name, sample_commit.id) - path = File.join(repository.path, ref) + + path = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + File.join(repository.path, ref) + end # Corrupt the reference File.truncate(path, 0) @@ -2088,6 +2112,13 @@ describe Repository do end describe '#update_ref' do + around do |example| + # TODO Gitlab::Git::OperationService will be moved to gitaly-ruby and disappear from this repo + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + example.run + end + end + it 'can create a ref' do Gitlab::Git::OperationService.new(nil, repository.raw_repository).send(:update_ref, 'refs/heads/foobar', 'refs/heads/master', Gitlab::Git::BLANK_SHA) @@ -2372,7 +2403,9 @@ describe Repository do end def create_remote_branch(remote_name, branch_name, target) - rugged = repository.rugged + rugged = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + repository.rugged + end rugged.references.create("refs/remotes/#{remote_name}/#{branch_name}", target.id) end @@ -2410,6 +2443,32 @@ describe Repository do end end + describe '#archive_metadata' do + let(:ref) { 'master' } + let(:storage_path) { '/tmp' } + + let(:prefix) { [project.path, ref].join('-') } + let(:filename) { prefix + '.tar.gz' } + + subject(:result) { repository.archive_metadata(ref, storage_path, append_sha: false) } + + context 'with hashed storage disabled' do + let(:project) { create(:project, :repository, :legacy_storage) } + + it 'uses the project path to generate the filename' do + expect(result['ArchivePrefix']).to eq(prefix) + expect(File.basename(result['ArchivePath'])).to eq(filename) + end + end + + context 'with hashed storage enabled' do + it 'uses the project path to generate the filename' do + expect(result['ArchivePrefix']).to eq(prefix) + expect(File.basename(result['ArchivePath'])).to eq(filename) + end + end + end + describe 'commit cache' do set(:project) { create(:project, :repository) } diff --git a/spec/models/term_agreement_spec.rb b/spec/models/term_agreement_spec.rb index a59bf119692..950dfa09a6a 100644 --- a/spec/models/term_agreement_spec.rb +++ b/spec/models/term_agreement_spec.rb @@ -5,4 +5,13 @@ describe TermAgreement do it { is_expected.to validate_presence_of(:term) } it { is_expected.to validate_presence_of(:user) } end + + describe '.accepted' do + it 'only includes accepted terms' do + accepted = create(:term_agreement, :accepted) + create(:term_agreement, :declined) + + expect(described_class.accepted).to contain_exactly(accepted) + end + end end diff --git a/spec/policies/ci/build_policy_spec.rb b/spec/policies/ci/build_policy_spec.rb index 9ca156deaa0..eead55d33ca 100644 --- a/spec/policies/ci/build_policy_spec.rb +++ b/spec/policies/ci/build_policy_spec.rb @@ -101,7 +101,7 @@ describe Ci::BuildPolicy do it 'enables update_build if user is maintainer' do allow_any_instance_of(Project).to receive(:empty_repo?).and_return(false) - allow_any_instance_of(Project).to receive(:branch_allows_maintainer_push?).and_return(true) + allow_any_instance_of(Project).to receive(:branch_allows_collaboration?).and_return(true) expect(policy).to be_allowed :update_build expect(policy).to be_allowed :update_commit_status diff --git a/spec/policies/ci/pipeline_policy_spec.rb b/spec/policies/ci/pipeline_policy_spec.rb index a5e509cfa0f..bd32faf06ef 100644 --- a/spec/policies/ci/pipeline_policy_spec.rb +++ b/spec/policies/ci/pipeline_policy_spec.rb @@ -69,7 +69,7 @@ describe Ci::PipelinePolicy, :models do it 'enables update_pipeline if user is maintainer' do allow_any_instance_of(Project).to receive(:empty_repo?).and_return(false) - allow_any_instance_of(Project).to receive(:branch_allows_maintainer_push?).and_return(true) + allow_any_instance_of(Project).to receive(:branch_allows_collaboration?).and_return(true) expect(policy).to be_allowed :update_pipeline end diff --git a/spec/policies/project_policy_spec.rb b/spec/policies/project_policy_spec.rb index 6609f5f7afd..6ac151f92f3 100644 --- a/spec/policies/project_policy_spec.rb +++ b/spec/policies/project_policy_spec.rb @@ -400,7 +400,7 @@ describe ProjectPolicy do :merge_request, target_project: target_project, source_project: project, - allow_maintainer_to_push: true + allow_collaboration: true ) end let(:maintainer_abilities) do diff --git a/spec/requests/api/avatar_spec.rb b/spec/requests/api/avatar_spec.rb new file mode 100644 index 00000000000..26e0435a6d5 --- /dev/null +++ b/spec/requests/api/avatar_spec.rb @@ -0,0 +1,106 @@ +require 'spec_helper' + +describe API::Avatar do + let(:gravatar_service) { double('GravatarService') } + + describe 'GET /avatar' do + context 'avatar uploaded to GitLab' do + context 'user with matching public email address' do + let(:user) { create(:user, :with_avatar, email: 'public@example.com', public_email: 'public@example.com') } + + before do + user + end + + it 'returns the avatar url' do + get api('/avatar'), { email: 'public@example.com' } + + expect(response.status).to eq 200 + expect(json_response['avatar_url']).to eql("#{::Settings.gitlab.base_url}#{user.avatar.local_url}") + end + end + + context 'no user with matching public email address' do + before do + expect(GravatarService).to receive(:new).and_return(gravatar_service) + expect(gravatar_service).to( + receive(:execute) + .with('private@example.com', nil, 2, { username: nil }) + .and_return('https://gravatar')) + end + + it 'returns the avatar url from Gravatar' do + get api('/avatar'), { email: 'private@example.com' } + + expect(response.status).to eq 200 + expect(json_response['avatar_url']).to eq('https://gravatar') + end + end + end + + context 'avatar uploaded to Gravatar' do + context 'user with matching public email address' do + let(:user) { create(:user, email: 'public@example.com', public_email: 'public@example.com') } + + before do + user + + expect(GravatarService).to receive(:new).and_return(gravatar_service) + expect(gravatar_service).to( + receive(:execute) + .with('public@example.com', nil, 2, { username: user.username }) + .and_return('https://gravatar')) + end + + it 'returns the avatar url from Gravatar' do + get api('/avatar'), { email: 'public@example.com' } + + expect(response.status).to eq 200 + expect(json_response['avatar_url']).to eq('https://gravatar') + end + end + + context 'no user with matching public email address' do + before do + expect(GravatarService).to receive(:new).and_return(gravatar_service) + expect(gravatar_service).to( + receive(:execute) + .with('private@example.com', nil, 2, { username: nil }) + .and_return('https://gravatar')) + end + + it 'returns the avatar url from Gravatar' do + get api('/avatar'), { email: 'private@example.com' } + + expect(response.status).to eq 200 + expect(json_response['avatar_url']).to eq('https://gravatar') + end + end + + context 'public visibility level restricted' do + let(:user) { create(:user, :with_avatar, email: 'public@example.com', public_email: 'public@example.com') } + + before do + user + + stub_application_setting(restricted_visibility_levels: [Gitlab::VisibilityLevel::PUBLIC]) + end + + context 'when authenticated' do + it 'returns the avatar url' do + get api('/avatar', user), { email: 'public@example.com' } + + expect(response.status).to eq 200 + expect(json_response['avatar_url']).to eql("#{::Settings.gitlab.base_url}#{user.avatar.local_url}") + end + end + + context 'when unauthenticated' do + it_behaves_like '403 response' do + let(:request) { get api('/avatar'), { email: 'public@example.com' } } + end + end + end + end + end +end diff --git a/spec/requests/api/events_spec.rb b/spec/requests/api/events_spec.rb index 962c845f36d..e6a61fdcf39 100644 --- a/spec/requests/api/events_spec.rb +++ b/spec/requests/api/events_spec.rb @@ -176,7 +176,7 @@ describe API::Events do end it 'avoids N+1 queries' do - control_count = ActiveRecord::QueryRecorder.new do + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) do get api("/projects/#{private_project.id}/events", user), target_type: :merge_request end.count @@ -184,7 +184,7 @@ describe API::Events do expect do get api("/projects/#{private_project.id}/events", user), target_type: :merge_request - end.not_to exceed_query_limit(control_count) + end.not_to exceed_all_query_limit(control_count) expect(response).to have_gitlab_http_status(200) expect(response).to include_pagination_headers diff --git a/spec/requests/api/graphql/merge_request_query_spec.rb b/spec/requests/api/graphql/merge_request_query_spec.rb new file mode 100644 index 00000000000..12b1d5d18a2 --- /dev/null +++ b/spec/requests/api/graphql/merge_request_query_spec.rb @@ -0,0 +1,49 @@ +require 'spec_helper' + +describe 'getting merge request information' do + include GraphqlHelpers + + let(:project) { create(:project, :repository) } + let(:merge_request) { create(:merge_request, source_project: project) } + let(:current_user) { create(:user) } + + let(:query) do + attributes = { + 'fullPath' => merge_request.project.full_path, + 'iid' => merge_request.iid + } + graphql_query_for('mergeRequest', attributes) + end + + context 'when the user has access to the merge request' do + before do + project.add_developer(current_user) + post_graphql(query, current_user: current_user) + end + + it 'returns the merge request' do + expect(graphql_data['mergeRequest']).not_to be_nil + end + + # This is a field coming from the `MergeRequestPresenter` + it 'includes a web_url' do + expect(graphql_data['mergeRequest']['webUrl']).to be_present + end + + it_behaves_like 'a working graphql query' + end + + context 'when the user does not have access to the merge request' do + before do + post_graphql(query, current_user: current_user) + end + + it 'returns an empty field' do + post_graphql(query, current_user: current_user) + + expect(graphql_data['mergeRequest']).to be_nil + end + + it_behaves_like 'a working graphql query' + end +end diff --git a/spec/requests/api/graphql/project_query_spec.rb b/spec/requests/api/graphql/project_query_spec.rb new file mode 100644 index 00000000000..8196bcfa87c --- /dev/null +++ b/spec/requests/api/graphql/project_query_spec.rb @@ -0,0 +1,39 @@ +require 'spec_helper' + +describe 'getting project information' do + include GraphqlHelpers + + let(:project) { create(:project, :repository) } + let(:current_user) { create(:user) } + + let(:query) do + graphql_query_for('project', 'fullPath' => project.full_path) + end + + context 'when the user has access to the project' do + before do + project.add_developer(current_user) + post_graphql(query, current_user: current_user) + end + + it 'includes the project' do + expect(graphql_data['project']).not_to be_nil + end + + it_behaves_like 'a working graphql query' + end + + context 'when the user does not have access to the project' do + before do + post_graphql(query, current_user: current_user) + end + + it 'returns an empty field' do + post_graphql(query, current_user: current_user) + + expect(graphql_data['project']).to be_nil + end + + it_behaves_like 'a working graphql query' + end +end diff --git a/spec/requests/api/internal_spec.rb b/spec/requests/api/internal_spec.rb index 5dc3ddd4b36..bc32372d3a9 100644 --- a/spec/requests/api/internal_spec.rb +++ b/spec/requests/api/internal_spec.rb @@ -835,8 +835,7 @@ describe API::Internal do end def push(key, project, protocol = 'ssh', env: nil) - post( - api("/internal/allowed"), + params = { changes: 'd14d6c0abdd253381df51a723d58691b2ee1ab08 570e7b2abdd848b95f2f578043fc23bd6f6fd24d refs/heads/master', key_id: key.id, project: project.full_path, @@ -845,7 +844,19 @@ describe API::Internal do secret_token: secret_token, protocol: protocol, env: env - ) + } + + if Gitlab.rails5? + post( + api("/internal/allowed"), + params: params + ) + else + post( + api("/internal/allowed"), + params + ) + end end def archive(key, project) diff --git a/spec/requests/api/issues_spec.rb b/spec/requests/api/issues_spec.rb index 4181f4ebbbe..a15d60aafe0 100644 --- a/spec/requests/api/issues_spec.rb +++ b/spec/requests/api/issues_spec.rb @@ -630,15 +630,17 @@ describe API::Issues do end it 'avoids N+1 queries' do - control_count = ActiveRecord::QueryRecorder.new do + get api("/projects/#{project.id}/issues", user) + + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) do get api("/projects/#{project.id}/issues", user) end.count - create(:issue, author: user, project: project) + create_list(:issue, 3, project: project) expect do get api("/projects/#{project.id}/issues", user) - end.not_to exceed_query_limit(control_count) + end.not_to exceed_all_query_limit(control_count) end it 'returns 404 when project does not exist' do diff --git a/spec/requests/api/jobs_spec.rb b/spec/requests/api/jobs_spec.rb index 45082e644ca..50d6f4b4d99 100644 --- a/spec/requests/api/jobs_spec.rb +++ b/spec/requests/api/jobs_spec.rb @@ -177,6 +177,18 @@ describe API::Jobs do json_response.each { |job| expect(job['pipeline']['id']).to eq(pipeline.id) } end end + + it 'avoids N+1 queries' do + control_count = ActiveRecord::QueryRecorder.new(skip_cached: false) do + get api("/projects/#{project.id}/pipelines/#{pipeline.id}/jobs", api_user), query + end.count + + 3.times { create(:ci_build, :artifacts, pipeline: pipeline) } + + expect do + get api("/projects/#{project.id}/pipelines/#{pipeline.id}/jobs", api_user), query + end.not_to exceed_all_query_limit(control_count) + end end context 'unauthorized user' do diff --git a/spec/requests/api/merge_requests_spec.rb b/spec/requests/api/merge_requests_spec.rb index 605761867bf..d4ebfc3f782 100644 --- a/spec/requests/api/merge_requests_spec.rb +++ b/spec/requests/api/merge_requests_spec.rb @@ -386,12 +386,13 @@ describe API::MergeRequests do source_project: forked_project, target_project: project, source_branch: 'fixes', - allow_maintainer_to_push: true) + allow_collaboration: true) end - it 'includes the `allow_maintainer_to_push` field' do + it 'includes the `allow_collaboration` field' do get api("/projects/#{project.id}/merge_requests/#{merge_request.iid}", user) + expect(json_response['allow_collaboration']).to be_truthy expect(json_response['allow_maintainer_to_push']).to be_truthy end end @@ -654,11 +655,12 @@ describe API::MergeRequests do expect(response).to have_gitlab_http_status(400) end - it 'allows setting `allow_maintainer_to_push`' do + it 'allows setting `allow_collaboration`' do post api("/projects/#{forked_project.id}/merge_requests", user2), - title: 'Test merge_request', source_branch: "feature_conflict", target_branch: "master", - author: user2, target_project_id: project.id, allow_maintainer_to_push: true + title: 'Test merge_request', source_branch: "feature_conflict", target_branch: "master", + author: user2, target_project_id: project.id, allow_collaboration: true expect(response).to have_gitlab_http_status(201) + expect(json_response['allow_collaboration']).to be_truthy expect(json_response['allow_maintainer_to_push']).to be_truthy end diff --git a/spec/requests/api/pipelines_spec.rb b/spec/requests/api/pipelines_spec.rb index 0736329f9fd..78ea77cb3bb 100644 --- a/spec/requests/api/pipelines_spec.rb +++ b/spec/requests/api/pipelines_spec.rb @@ -285,6 +285,15 @@ describe API::Pipelines do end describe 'POST /projects/:id/pipeline ' do + def expect_variables(variables, expected_variables) + variables.each_with_index do |variable, index| + expected_variable = expected_variables[index] + + expect(variable.key).to eq(expected_variable['key']) + expect(variable.value).to eq(expected_variable['value']) + end + end + context 'authorized user' do context 'with gitlab-ci.yml' do before do @@ -294,13 +303,62 @@ describe API::Pipelines do it 'creates and returns a new pipeline' do expect do post api("/projects/#{project.id}/pipeline", user), ref: project.default_branch - end.to change { Ci::Pipeline.count }.by(1) + end.to change { project.pipelines.count }.by(1) expect(response).to have_gitlab_http_status(201) expect(json_response).to be_a Hash expect(json_response['sha']).to eq project.commit.id end + context 'variables given' do + let(:variables) { [{ 'key' => 'UPLOAD_TO_S3', 'value' => 'true' }] } + + it 'creates and returns a new pipeline using the given variables' do + expect do + post api("/projects/#{project.id}/pipeline", user), ref: project.default_branch, variables: variables + end.to change { project.pipelines.count }.by(1) + expect_variables(project.pipelines.last.variables, variables) + + expect(response).to have_gitlab_http_status(201) + expect(json_response).to be_a Hash + expect(json_response['sha']).to eq project.commit.id + expect(json_response).not_to have_key('variables') + end + end + + describe 'using variables conditions' do + let(:variables) { [{ 'key' => 'STAGING', 'value' => 'true' }] } + + before do + config = YAML.dump(test: { script: 'test', only: { variables: ['$STAGING'] } }) + stub_ci_pipeline_yaml_file(config) + end + + it 'creates and returns a new pipeline using the given variables' do + expect do + post api("/projects/#{project.id}/pipeline", user), ref: project.default_branch, variables: variables + end.to change { project.pipelines.count }.by(1) + expect_variables(project.pipelines.last.variables, variables) + + expect(response).to have_gitlab_http_status(201) + expect(json_response).to be_a Hash + expect(json_response['sha']).to eq project.commit.id + expect(json_response).not_to have_key('variables') + end + + context 'condition unmatch' do + let(:variables) { [{ 'key' => 'STAGING', 'value' => 'false' }] } + + it "doesn't create a job" do + expect do + post api("/projects/#{project.id}/pipeline", user), ref: project.default_branch + end.not_to change { project.pipelines.count } + + expect(response).to have_gitlab_http_status(400) + end + end + end + it 'fails when using an invalid ref' do post api("/projects/#{project.id}/pipeline", user), ref: 'invalid_ref' diff --git a/spec/requests/api/repositories_spec.rb b/spec/requests/api/repositories_spec.rb index 9e6d69e3874..cd135dfc32a 100644 --- a/spec/requests/api/repositories_spec.rb +++ b/spec/requests/api/repositories_spec.rb @@ -220,11 +220,10 @@ describe API::Repositories do expect(response).to have_gitlab_http_status(200) - repo_name = project.repository.name.gsub("\.git", "") type, params = workhorse_send_data expect(type).to eq('git-archive') - expect(params['ArchivePath']).to match(/#{repo_name}\-[^\.]+\.tar.gz/) + expect(params['ArchivePath']).to match(/#{project.path}\-[^\.]+\.tar.gz/) end it 'returns the repository archive archive.zip' do @@ -232,11 +231,10 @@ describe API::Repositories do expect(response).to have_gitlab_http_status(200) - repo_name = project.repository.name.gsub("\.git", "") type, params = workhorse_send_data expect(type).to eq('git-archive') - expect(params['ArchivePath']).to match(/#{repo_name}\-[^\.]+\.zip/) + expect(params['ArchivePath']).to match(/#{project.path}\-[^\.]+\.zip/) end it 'returns the repository archive archive.tar.bz2' do @@ -244,11 +242,10 @@ describe API::Repositories do expect(response).to have_gitlab_http_status(200) - repo_name = project.repository.name.gsub("\.git", "") type, params = workhorse_send_data expect(type).to eq('git-archive') - expect(params['ArchivePath']).to match(/#{repo_name}\-[^\.]+\.tar.bz2/) + expect(params['ArchivePath']).to match(/#{project.path}\-[^\.]+\.tar.bz2/) end context 'when sha does not exist' do diff --git a/spec/requests/api/runner_spec.rb b/spec/requests/api/runner_spec.rb index 319ac389083..c981a10ac38 100644 --- a/spec/requests/api/runner_spec.rb +++ b/spec/requests/api/runner_spec.rb @@ -1101,6 +1101,7 @@ describe API::Runner, :clean_gitlab_redis_shared_state do expect(json_response['RemoteObject']).to have_key('GetURL') expect(json_response['RemoteObject']).to have_key('StoreURL') expect(json_response['RemoteObject']).to have_key('DeleteURL') + expect(json_response['RemoteObject']).to have_key('MultipartUpload') end end diff --git a/spec/requests/api/settings_spec.rb b/spec/requests/api/settings_spec.rb index aead8978dd4..57adc3ca7a6 100644 --- a/spec/requests/api/settings_spec.rb +++ b/spec/requests/api/settings_spec.rb @@ -35,7 +35,9 @@ describe API::Settings, 'Settings' do context "custom repository storage type set in the config" do before do - storages = { 'custom' => 'tmp/tests/custom_repositories' } + # Add a possible storage to the config + storages = Gitlab.config.repositories.storages + .merge({ 'custom' => 'tmp/tests/custom_repositories' }) allow(Gitlab.config.repositories).to receive(:storages).and_return(storages) end diff --git a/spec/requests/api/tags_spec.rb b/spec/requests/api/tags_spec.rb index e2b19ad59f9..969710d6613 100644 --- a/spec/requests/api/tags_spec.rb +++ b/spec/requests/api/tags_spec.rb @@ -287,7 +287,10 @@ describe API::Tags do context 'annotated tag' do it 'creates a new annotated tag' do # Identity must be set in .gitconfig to create annotated tag. - repo_path = project.repository.path_to_repo + repo_path = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + project.repository.path_to_repo + end + system(*%W(#{Gitlab.config.git.bin_path} --git-dir=#{repo_path} config user.name #{user.name})) system(*%W(#{Gitlab.config.git.bin_path} --git-dir=#{repo_path} config user.email #{user.email})) diff --git a/spec/requests/lfs_http_spec.rb b/spec/requests/lfs_http_spec.rb index 79672fe1cc5..4d30b99262e 100644 --- a/spec/requests/lfs_http_spec.rb +++ b/spec/requests/lfs_http_spec.rb @@ -1021,6 +1021,7 @@ describe 'Git LFS API and storage' do expect(json_response['RemoteObject']).to have_key('GetURL') expect(json_response['RemoteObject']).to have_key('StoreURL') expect(json_response['RemoteObject']).to have_key('DeleteURL') + expect(json_response['RemoteObject']).not_to have_key('MultipartUpload') expect(json_response['LfsOid']).to eq(sample_oid) expect(json_response['LfsSize']).to eq(sample_size) end diff --git a/spec/routing/api_routing_spec.rb b/spec/routing/api_routing_spec.rb new file mode 100644 index 00000000000..5fde4bd885b --- /dev/null +++ b/spec/routing/api_routing_spec.rb @@ -0,0 +1,31 @@ +require 'spec_helper' + +describe 'api', 'routing' do + context 'when graphql is disabled' do + before do + stub_feature_flags(graphql: false) + end + + it 'does not route to the GraphqlController' do + expect(get('/api/graphql')).not_to route_to('graphql#execute') + end + + it 'does not expose graphiql' do + expect(get('/-/graphql-explorer')).not_to route_to('graphiql/rails/editors#show') + end + end + + context 'when graphql is disabled' do + before do + stub_feature_flags(graphql: true) + end + + it 'routes to the GraphqlController' do + expect(get('/api/graphql')).not_to route_to('graphql#execute') + end + + it 'exposes graphiql' do + expect(get('/-/graphql-explorer')).not_to route_to('graphiql/rails/editors#show') + end + end +end diff --git a/spec/serializers/pipeline_serializer_spec.rb b/spec/serializers/pipeline_serializer_spec.rb index b741308e2c5..e0e6eecb300 100644 --- a/spec/serializers/pipeline_serializer_spec.rb +++ b/spec/serializers/pipeline_serializer_spec.rb @@ -8,6 +8,10 @@ describe PipelineSerializer do described_class.new(current_user: user) end + before do + stub_feature_flags(ci_pipeline_persisted_stages: true) + end + subject { serializer.represent(resource) } describe '#represent' do @@ -99,7 +103,8 @@ describe PipelineSerializer do end end - context 'number of queries' do + describe 'number of queries when preloaded' do + subject { serializer.represent(resource, preload: true) } let(:resource) { Ci::Pipeline.all } before do @@ -107,7 +112,7 @@ describe PipelineSerializer do # gitaly calls in this block # Issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/37772 Gitlab::GitalyClient.allow_n_plus_1_calls do - Ci::Pipeline::AVAILABLE_STATUSES.each do |status| + Ci::Pipeline::COMPLETED_STATUSES.each do |status| create_pipeline(status) end end @@ -120,7 +125,7 @@ describe PipelineSerializer do it 'verifies number of queries', :request_store do recorded = ActiveRecord::QueryRecorder.new { subject } - expect(recorded.count).to be_within(1).of(44) + expect(recorded.count).to be_within(2).of(27) expect(recorded.cached_count).to eq(0) end end @@ -139,7 +144,7 @@ describe PipelineSerializer do # pipeline. With the same ref this check is cached but if refs are # different then there is an extra query per ref # https://gitlab.com/gitlab-org/gitlab-ce/issues/46368 - expect(recorded.count).to be_within(1).of(51) + expect(recorded.count).to be_within(2).of(30) expect(recorded.cached_count).to eq(0) end end diff --git a/spec/services/ci/retry_pipeline_service_spec.rb b/spec/services/ci/retry_pipeline_service_spec.rb index a73bd7a0268..688d3b8c038 100644 --- a/spec/services/ci/retry_pipeline_service_spec.rb +++ b/spec/services/ci/retry_pipeline_service_spec.rb @@ -280,12 +280,12 @@ describe Ci::RetryPipelineService, '#execute' do source_project: forked_project, target_project: project, source_branch: 'fixes', - allow_maintainer_to_push: true) + allow_collaboration: true) create_build('rspec 1', :failed, 1) end it 'allows to retry failed pipeline' do - allow_any_instance_of(Project).to receive(:fetch_branch_allows_maintainer_push?).and_return(true) + allow_any_instance_of(Project).to receive(:fetch_branch_allows_collaboration?).and_return(true) allow_any_instance_of(Project).to receive(:empty_repo?).and_return(false) service.execute(pipeline) diff --git a/spec/services/lfs/unlock_file_service_spec.rb b/spec/services/lfs/unlock_file_service_spec.rb index 4bea112b9c6..948401d7bdc 100644 --- a/spec/services/lfs/unlock_file_service_spec.rb +++ b/spec/services/lfs/unlock_file_service_spec.rb @@ -80,12 +80,12 @@ describe Lfs::UnlockFileService do result = subject.execute expect(result[:status]).to eq(:error) - expect(result[:message]).to match(/You must have master access/) + expect(result[:message]).to match(/You must have maintainer access/) expect(result[:http_status]).to eq(403) end end - context 'by a master user' do + context 'by a maintainer user' do let(:current_user) { master } let(:params) do { id: lock.id, diff --git a/spec/services/merge_requests/update_service_spec.rb b/spec/services/merge_requests/update_service_spec.rb index bd2e91f1f7a..443dcd92a8b 100644 --- a/spec/services/merge_requests/update_service_spec.rb +++ b/spec/services/merge_requests/update_service_spec.rb @@ -547,7 +547,7 @@ describe MergeRequests::UpdateService, :mailer do let(:closed_issuable) { create(:closed_merge_request, source_project: project) } end - context 'setting `allow_maintainer_to_push`' do + context 'setting `allow_collaboration`' do let(:target_project) { create(:project, :public) } let(:source_project) { fork_project(target_project) } let(:user) { create(:user) } @@ -562,23 +562,23 @@ describe MergeRequests::UpdateService, :mailer do allow(ProtectedBranch).to receive(:protected?).with(source_project, 'fixes') { false } end - it 'does not allow a maintainer of the target project to set `allow_maintainer_to_push`' do + it 'does not allow a maintainer of the target project to set `allow_collaboration`' do target_project.add_developer(user) - update_merge_request(allow_maintainer_to_push: true, title: 'Updated title') + update_merge_request(allow_collaboration: true, title: 'Updated title') expect(merge_request.title).to eq('Updated title') - expect(merge_request.allow_maintainer_to_push).to be_falsy + expect(merge_request.allow_collaboration).to be_falsy end it 'is allowed by a user that can push to the source and can update the merge request' do merge_request.update!(assignee: user) source_project.add_developer(user) - update_merge_request(allow_maintainer_to_push: true, title: 'Updated title') + update_merge_request(allow_collaboration: true, title: 'Updated title') expect(merge_request.title).to eq('Updated title') - expect(merge_request.allow_maintainer_to_push).to be_truthy + expect(merge_request.allow_collaboration).to be_truthy end end end diff --git a/spec/services/pages_service_spec.rb b/spec/services/pages_service_spec.rb deleted file mode 100644 index f8db6900a0a..00000000000 --- a/spec/services/pages_service_spec.rb +++ /dev/null @@ -1,53 +0,0 @@ -require 'spec_helper' - -describe PagesService do - let(:build) { create(:ci_build) } - let(:data) { Gitlab::DataBuilder::Build.build(build) } - let(:service) { described_class.new(data) } - - before do - allow(Gitlab.config.pages).to receive(:enabled).and_return(true) - end - - context 'execute asynchronously for pages job' do - before do - build.name = 'pages' - end - - context 'on success' do - before do - build.success - end - - it 'executes worker' do - expect(PagesWorker).to receive(:perform_async) - service.execute - end - end - - %w(pending running failed canceled).each do |status| - context "on #{status}" do - before do - build.status = status - end - - it 'does not execute worker' do - expect(PagesWorker).not_to receive(:perform_async) - service.execute - end - end - end - end - - context 'for other jobs' do - before do - build.name = 'other job' - build.success - end - - it 'does not execute worker' do - expect(PagesWorker).not_to receive(:perform_async) - service.execute - end - end -end diff --git a/spec/services/projects/autocomplete_service_spec.rb b/spec/services/projects/autocomplete_service_spec.rb index f7ff8b80bd7..6fd73a50511 100644 --- a/spec/services/projects/autocomplete_service_spec.rb +++ b/spec/services/projects/autocomplete_service_spec.rb @@ -115,5 +115,20 @@ describe Projects::AutocompleteService do expect(milestone_titles).to eq([group_milestone2.title, group_milestone1.title]) end + + context 'with nested groups', :nested_groups do + let(:subgroup) { create(:group, :public, parent: group) } + let!(:subgroup_milestone) { create(:milestone, group: subgroup) } + + before do + project.update(namespace: subgroup) + end + + it 'includes project milestones and all acestors milestones' do + expect(milestone_titles).to match_array( + [project_milestone.title, group_milestone2.title, group_milestone1.title, subgroup_milestone.title] + ) + end + end end end diff --git a/spec/services/projects/import_service_spec.rb b/spec/services/projects/import_service_spec.rb index 30c89ebd821..b3815045792 100644 --- a/spec/services/projects/import_service_spec.rb +++ b/spec/services/projects/import_service_spec.rb @@ -3,9 +3,17 @@ require 'spec_helper' describe Projects::ImportService do let!(:project) { create(:project) } let(:user) { project.creator } + let(:import_url) { 'http://www.gitlab.com/demo/repo.git' } + let(:oid_download_links) { { 'oid1' => "#{import_url}/gitlab-lfs/objects/oid1", 'oid2' => "#{import_url}/gitlab-lfs/objects/oid2" } } subject { described_class.new(project, user) } + before do + allow(project).to receive(:lfs_enabled?).and_return(true) + allow_any_instance_of(Projects::LfsPointers::LfsDownloadService).to receive(:execute) + allow_any_instance_of(Projects::LfsPointers::LfsImportService).to receive(:execute).and_return(oid_download_links) + end + describe '#async?' do it 'returns true for an asynchronous importer' do importer_class = double(:importer, async?: true) @@ -63,6 +71,15 @@ describe Projects::ImportService do expect(result[:status]).to eq :error expect(result[:message]).to eq "Error importing repository #{project.import_url} into #{project.full_path} - The repository could not be created." end + + context 'when repository creation succeeds' do + it 'does not download lfs files' do + expect_any_instance_of(Projects::LfsPointers::LfsImportService).not_to receive(:execute) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).not_to receive(:execute) + + subject.execute + end + end end context 'with known url' do @@ -91,6 +108,15 @@ describe Projects::ImportService do expect(result[:status]).to eq :error end + + context 'when repository import scheduled' do + it 'does not download lfs objects' do + expect_any_instance_of(Projects::LfsPointers::LfsImportService).not_to receive(:execute) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).not_to receive(:execute) + + subject.execute + end + end end context 'with a non Github repository' do @@ -99,9 +125,10 @@ describe Projects::ImportService do project.import_type = 'bitbucket' end - it 'succeeds if repository import is successfully' do + it 'succeeds if repository import is successfull' do expect_any_instance_of(Gitlab::Shell).to receive(:import_repository).and_return(true) expect_any_instance_of(Gitlab::BitbucketImport::Importer).to receive(:execute).and_return(true) + expect_any_instance_of(Projects::LfsPointers::LfsImportService).to receive(:execute).and_return({}) result = subject.execute @@ -116,6 +143,29 @@ describe Projects::ImportService do expect(result[:status]).to eq :error expect(result[:message]).to eq "Error importing repository #{project.import_url} into #{project.full_path} - Failed to import the repository" end + + context 'when repository import scheduled' do + before do + allow_any_instance_of(Gitlab::Shell).to receive(:import_repository).and_return(true) + allow(subject).to receive(:import_data) + end + + it 'downloads lfs objects if lfs_enabled is enabled for project' do + allow(project).to receive(:lfs_enabled?).and_return(true) + expect_any_instance_of(Projects::LfsPointers::LfsImportService).to receive(:execute).and_return(oid_download_links) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).to receive(:execute).twice + + subject.execute + end + + it 'does not download lfs objects if lfs_enabled is not enabled for project' do + allow(project).to receive(:lfs_enabled?).and_return(false) + expect_any_instance_of(Projects::LfsPointers::LfsImportService).not_to receive(:execute) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).not_to receive(:execute) + + subject.execute + end + end end end @@ -147,6 +197,26 @@ describe Projects::ImportService do expect(result[:status]).to eq :error end + + context 'when importer' do + it 'has a custom repository importer it does not download lfs objects' do + allow(Gitlab::GithubImport::ParallelImporter).to receive(:imports_repository?).and_return(true) + + expect_any_instance_of(Projects::LfsPointers::LfsImportService).not_to receive(:execute) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).not_to receive(:execute) + + subject.execute + end + + it 'does not have a custom repository importer downloads lfs objects' do + allow(Gitlab::GithubImport::ParallelImporter).to receive(:imports_repository?).and_return(false) + + expect_any_instance_of(Projects::LfsPointers::LfsImportService).to receive(:execute).and_return(oid_download_links) + expect_any_instance_of(Projects::LfsPointers::LfsDownloadService).to receive(:execute) + + subject.execute + end + end end context 'with blocked import_URL' do diff --git a/spec/services/projects/lfs_pointers/lfs_download_link_list_service_spec.rb b/spec/services/projects/lfs_pointers/lfs_download_link_list_service_spec.rb new file mode 100644 index 00000000000..d7a2829d5f8 --- /dev/null +++ b/spec/services/projects/lfs_pointers/lfs_download_link_list_service_spec.rb @@ -0,0 +1,102 @@ +require 'spec_helper' + +describe Projects::LfsPointers::LfsDownloadLinkListService do + let(:import_url) { 'http://www.gitlab.com/demo/repo.git' } + let(:lfs_endpoint) { "#{import_url}/info/lfs/objects/batch" } + let!(:project) { create(:project, import_url: import_url) } + let(:new_oids) { { 'oid1' => 123, 'oid2' => 125 } } + let(:remote_uri) { URI.parse(lfs_endpoint) } + + let(:objects_response) do + body = new_oids.map do |oid, size| + { + 'oid' => oid, + 'size' => size, + 'actions' => { + 'download' => { 'href' => "#{import_url}/gitlab-lfs/objects/#{oid}" } + } + } + end + + Struct.new(:success?, :objects).new(true, body) + end + + let(:invalid_object_response) do + [ + 'oid' => 'whatever', + 'size' => 123 + ] + end + + subject { described_class.new(project, remote_uri: remote_uri) } + + before do + allow(project).to receive(:lfs_enabled?).and_return(true) + allow(Gitlab::HTTP).to receive(:post).and_return(objects_response) + end + + describe '#execute' do + it 'retrieves each download link of every non existent lfs object' do + subject.execute(new_oids).each do |oid, link| + expect(link).to eq "#{import_url}/gitlab-lfs/objects/#{oid}" + end + end + + context 'credentials' do + context 'when the download link and the lfs_endpoint have the same host' do + context 'when lfs_endpoint has credentials' do + let(:import_url) { 'http://user:password@www.gitlab.com/demo/repo.git' } + + it 'adds credentials to the download_link' do + result = subject.execute(new_oids) + + result.each do |oid, link| + expect(link.starts_with?('http://user:password@')).to be_truthy + end + end + end + + context 'when lfs_endpoint does not have any credentials' do + it 'does not add any credentials' do + result = subject.execute(new_oids) + + result.each do |oid, link| + expect(link.starts_with?('http://user:password@')).to be_falsey + end + end + end + end + + context 'when the download link and the lfs_endpoint have different hosts' do + let(:import_url_with_credentials) { 'http://user:password@www.otherdomain.com/demo/repo.git' } + let(:lfs_endpoint) { "#{import_url_with_credentials}/info/lfs/objects/batch" } + + it 'downloads without any credentials' do + result = subject.execute(new_oids) + + result.each do |oid, link| + expect(link.starts_with?('http://user:password@')).to be_falsey + end + end + end + end + end + + describe '#get_download_links' do + it 'raise errorif request fails' do + allow(Gitlab::HTTP).to receive(:post).and_return(Struct.new(:success?, :message).new(false, 'Failed request')) + + expect { subject.send(:get_download_links, new_oids) }.to raise_error(described_class::DownloadLinksError) + end + end + + describe '#parse_response_links' do + it 'does not add oid entry if href not found' do + expect(Rails.logger).to receive(:error).with("Link for Lfs Object with oid whatever not found or invalid.") + + result = subject.send(:parse_response_links, invalid_object_response) + + expect(result).to be_empty + end + end +end diff --git a/spec/services/projects/lfs_pointers/lfs_download_service_spec.rb b/spec/services/projects/lfs_pointers/lfs_download_service_spec.rb new file mode 100644 index 00000000000..6af5bfc7689 --- /dev/null +++ b/spec/services/projects/lfs_pointers/lfs_download_service_spec.rb @@ -0,0 +1,69 @@ +require 'spec_helper' + +describe Projects::LfsPointers::LfsDownloadService do + let(:project) { create(:project) } + let(:oid) { '9e548e25631dd9ce6b43afd6359ab76da2819d6a5b474e66118c7819e1d8b3e8' } + let(:download_link) { "http://gitlab.com/#{oid}" } + let(:lfs_content) do + <<~HEREDOC + whatever + HEREDOC + end + + subject { described_class.new(project) } + + before do + allow(project).to receive(:lfs_enabled?).and_return(true) + WebMock.stub_request(:get, download_link).to_return(body: lfs_content) + end + + describe '#execute' do + context 'when file download succeeds' do + it 'a new lfs object is created' do + expect { subject.execute(oid, download_link) }.to change { LfsObject.count }.from(0).to(1) + end + + it 'has the same oid' do + subject.execute(oid, download_link) + + expect(LfsObject.first.oid).to eq oid + end + + it 'stores the content' do + subject.execute(oid, download_link) + + expect(File.read(LfsObject.first.file.file.file)).to eq lfs_content + end + end + + context 'when file download fails' do + it 'no lfs object is created' do + expect { subject.execute(oid, download_link) }.to change { LfsObject.count } + end + end + + context 'when credentials present' do + let(:download_link_with_credentials) { "http://user:password@gitlab.com/#{oid}" } + + before do + WebMock.stub_request(:get, download_link).with(headers: { 'Authorization' => 'Basic dXNlcjpwYXNzd29yZA==' }).to_return(body: lfs_content) + end + + it 'the request adds authorization headers' do + subject.execute(oid, download_link_with_credentials) + end + end + + context 'when an lfs object with the same oid already exists' do + before do + create(:lfs_object, oid: 'oid') + end + + it 'does not download the file' do + expect(subject).not_to receive(:download_and_save_file) + + subject.execute('oid', download_link) + end + end + end +end diff --git a/spec/services/projects/lfs_pointers/lfs_import_service_spec.rb b/spec/services/projects/lfs_pointers/lfs_import_service_spec.rb new file mode 100644 index 00000000000..5a75fb38dec --- /dev/null +++ b/spec/services/projects/lfs_pointers/lfs_import_service_spec.rb @@ -0,0 +1,146 @@ +require 'spec_helper' + +describe Projects::LfsPointers::LfsImportService do + let(:import_url) { 'http://www.gitlab.com/demo/repo.git' } + let(:default_endpoint) { "#{import_url}/info/lfs/objects/batch"} + let(:group) { create(:group, lfs_enabled: true)} + let!(:project) { create(:project, namespace: group, import_url: import_url, lfs_enabled: true) } + let!(:lfs_objects_project) { create_list(:lfs_objects_project, 2, project: project) } + let!(:existing_lfs_objects) { LfsObject.pluck(:oid, :size).to_h } + let(:oids) { { 'oid1' => 123, 'oid2' => 125 } } + let(:oid_download_links) { { 'oid1' => "#{import_url}/gitlab-lfs/objects/oid1", 'oid2' => "#{import_url}/gitlab-lfs/objects/oid2" } } + let(:all_oids) { existing_lfs_objects.merge(oids) } + let(:remote_uri) { URI.parse(lfs_endpoint) } + + subject { described_class.new(project) } + + before do + allow(project.repository).to receive(:lfsconfig_for).and_return(nil) + allow(Gitlab.config.lfs).to receive(:enabled).and_return(true) + allow_any_instance_of(Projects::LfsPointers::LfsListService).to receive(:execute).and_return(all_oids) + end + + describe '#execute' do + context 'when no lfs pointer is linked' do + before do + allow_any_instance_of(Projects::LfsPointers::LfsLinkService).to receive(:execute).and_return([]) + allow_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute).and_return(oid_download_links) + expect(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:new).with(project, remote_uri: URI.parse(default_endpoint)).and_call_original + end + + it 'retrieves all lfs pointers in the project repository' do + expect_any_instance_of(Projects::LfsPointers::LfsListService).to receive(:execute) + + subject.execute + end + + it 'links existent lfs objects to the project' do + expect_any_instance_of(Projects::LfsPointers::LfsLinkService).to receive(:execute) + + subject.execute + end + + it 'retrieves the download links of non existent objects' do + expect_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute).with(all_oids) + + subject.execute + end + end + + context 'when some lfs objects are linked' do + before do + allow_any_instance_of(Projects::LfsPointers::LfsLinkService).to receive(:execute).and_return(existing_lfs_objects.keys) + allow_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute).and_return(oid_download_links) + end + + it 'retrieves the download links of non existent objects' do + expect_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute).with(oids) + + subject.execute + end + end + + context 'when all lfs objects are linked' do + before do + allow_any_instance_of(Projects::LfsPointers::LfsLinkService).to receive(:execute).and_return(all_oids.keys) + allow_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute) + end + + it 'retrieves no download links' do + expect_any_instance_of(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:execute).with({}).and_call_original + + expect(subject.execute).to be_empty + end + end + + context 'when lfsconfig file exists' do + before do + allow(project.repository).to receive(:lfsconfig_for).and_return("[lfs]\n\turl = #{lfs_endpoint}\n") + end + + context 'when url points to the same import url host' do + let(:lfs_endpoint) { "#{import_url}/different_endpoint" } + let(:service) { double } + + before do + allow(service).to receive(:execute) + end + it 'downloads lfs object using the new endpoint' do + expect(Projects::LfsPointers::LfsDownloadLinkListService).to receive(:new).with(project, remote_uri: remote_uri).and_return(service) + + subject.execute + end + + context 'when import url has credentials' do + let(:import_url) { 'http://user:password@www.gitlab.com/demo/repo.git'} + + it 'adds the credentials to the new endpoint' do + expect(Projects::LfsPointers::LfsDownloadLinkListService) + .to receive(:new).with(project, remote_uri: URI.parse("http://user:password@www.gitlab.com/demo/repo.git/different_endpoint")) + .and_return(service) + + subject.execute + end + + context 'when url has its own credentials' do + let(:lfs_endpoint) { "http://user1:password1@www.gitlab.com/demo/repo.git/different_endpoint" } + + it 'does not add the import url credentials' do + expect(Projects::LfsPointers::LfsDownloadLinkListService) + .to receive(:new).with(project, remote_uri: remote_uri) + .and_return(service) + + subject.execute + end + end + end + end + + context 'when url points to a third party service' do + let(:lfs_endpoint) { 'http://third_party_service.com/info/lfs/objects/' } + + it 'disables lfs from the project' do + expect(project.lfs_enabled?).to be_truthy + + subject.execute + + expect(project.lfs_enabled?).to be_falsey + end + + it 'does not download anything' do + expect_any_instance_of(Projects::LfsPointers::LfsListService).not_to receive(:execute) + + subject.execute + end + end + end + end + + describe '#default_endpoint_uri' do + let(:import_url) { 'http://www.gitlab.com/demo/repo' } + + it 'adds suffix .git if the url does not have it' do + expect(subject.send(:default_endpoint_uri).path).to match(/repo.git/) + end + end +end diff --git a/spec/services/projects/lfs_pointers/lfs_link_service_spec.rb b/spec/services/projects/lfs_pointers/lfs_link_service_spec.rb new file mode 100644 index 00000000000..b7b153655db --- /dev/null +++ b/spec/services/projects/lfs_pointers/lfs_link_service_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe Projects::LfsPointers::LfsLinkService do + let!(:project) { create(:project, lfs_enabled: true) } + let!(:lfs_objects_project) { create_list(:lfs_objects_project, 2, project: project) } + let(:new_oids) { { 'oid1' => 123, 'oid2' => 125 } } + let(:all_oids) { LfsObject.pluck(:oid, :size).to_h.merge(new_oids) } + let(:new_lfs_object) { create(:lfs_object) } + let(:new_oid_list) { all_oids.merge(new_lfs_object.oid => new_lfs_object.size) } + + subject { described_class.new(project) } + + before do + allow(project).to receive(:lfs_enabled?).and_return(true) + end + + describe '#execute' do + it 'links existing lfs objects to the project' do + expect(project.all_lfs_objects.count).to eq 2 + + linked = subject.execute(new_oid_list.keys) + + expect(project.all_lfs_objects.count).to eq 3 + expect(linked.size).to eq 3 + end + + it 'returns linked oids' do + linked = lfs_objects_project.map(&:lfs_object).map(&:oid) << new_lfs_object.oid + + expect(subject.execute(new_oid_list.keys)).to eq linked + end + end +end diff --git a/spec/services/projects/transfer_service_spec.rb b/spec/services/projects/transfer_service_spec.rb index 3e6483d7e28..5100987c2fe 100644 --- a/spec/services/projects/transfer_service_spec.rb +++ b/spec/services/projects/transfer_service_spec.rb @@ -64,7 +64,7 @@ describe Projects::TransferService do it 'updates project full path in .git/config' do transfer_project(project, user, group) - expect(project.repository.rugged.config['gitlab.fullpath']).to eq "#{group.full_path}/#{project.path}" + expect(rugged_config['gitlab.fullpath']).to eq "#{group.full_path}/#{project.path}" end end @@ -84,7 +84,9 @@ describe Projects::TransferService do end def project_path(project) - project.repository.path_to_repo + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + project.repository.path_to_repo + end end def current_path @@ -101,7 +103,7 @@ describe Projects::TransferService do it 'rolls back project full path in .git/config' do attempt_project_transfer - expect(project.repository.rugged.config['gitlab.fullpath']).to eq project.full_path + expect(rugged_config['gitlab.fullpath']).to eq project.full_path end it "doesn't send move notifications" do @@ -264,4 +266,10 @@ describe Projects::TransferService do transfer_project(project, owner, group) end end + + def rugged_config + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + project.repository.rugged.config + end + end end diff --git a/spec/services/projects/update_service_spec.rb b/spec/services/projects/update_service_spec.rb index 3e6073b9861..ecf1ba05618 100644 --- a/spec/services/projects/update_service_spec.rb +++ b/spec/services/projects/update_service_spec.rb @@ -125,7 +125,7 @@ describe Projects::UpdateService do context 'when we update project but not enabling a wiki' do it 'does not try to create an empty wiki' do - FileUtils.rm_rf(project.wiki.repository.path) + Gitlab::Shell.new.rm_directory(project.repository_storage, project.wiki.path) result = update_project(project, user, { name: 'test1' }) @@ -146,7 +146,7 @@ describe Projects::UpdateService do context 'when enabling a wiki' do it 'creates a wiki' do project.project_feature.update(wiki_access_level: ProjectFeature::DISABLED) - FileUtils.rm_rf(project.wiki.repository.path) + Gitlab::Shell.new.rm_directory(project.repository_storage, project.wiki.path) result = update_project(project, user, project_feature_attributes: { wiki_access_level: ProjectFeature::ENABLED }) @@ -275,6 +275,10 @@ describe Projects::UpdateService do it { is_expected.to eq(false) } end + context 'when auto devops is nil' do + it { is_expected.to eq(false) } + end + context 'when auto devops is explicitly enabled' do before do project.create_auto_devops!(enabled: true) diff --git a/spec/services/test_hooks/project_service_spec.rb b/spec/services/test_hooks/project_service_spec.rb index 962b9f40c4f..19e1c5ff3b2 100644 --- a/spec/services/test_hooks/project_service_spec.rb +++ b/spec/services/test_hooks/project_service_spec.rb @@ -6,13 +6,19 @@ describe TestHooks::ProjectService do describe '#execute' do let(:project) { create(:project, :repository) } let(:hook) { create(:project_hook, project: project) } + let(:trigger) { 'not_implemented_events' } let(:service) { described_class.new(hook, current_user, trigger) } let(:sample_data) { { data: 'sample' } } let(:success_result) { { status: :success, http_status: 200, message: 'ok' } } - context 'hook with not implemented test' do - let(:trigger) { 'not_implemented_events' } + it 'allows to set a custom project' do + project = double + service.project = project + + expect(service.project).to eq(project) + end + context 'hook with not implemented test' do it 'returns error message' do expect(hook).not_to receive(:execute) expect(service.execute).to include({ status: :error, message: 'Testing not available for this hook' }) diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index d3de2331244..e093444121a 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -133,6 +133,10 @@ RSpec.configure do |config| RequestStore.clear! end + config.after(:example) do + Fog.unmock! if Fog.mock? + end + config.before(:example, :mailer) do reset_delivered_emails! end diff --git a/spec/support/controllers/githubish_import_controller_shared_examples.rb b/spec/support/controllers/githubish_import_controller_shared_examples.rb index 368439aa5b0..1c1b68c12a2 100644 --- a/spec/support/controllers/githubish_import_controller_shared_examples.rb +++ b/spec/support/controllers/githubish_import_controller_shared_examples.rb @@ -118,14 +118,19 @@ shared_examples 'a GitHub-ish import controller: POST create' do expect(response).to have_gitlab_http_status(200) end - it 'returns 422 response when the project could not be imported' do + it 'returns 422 response with the base error when the project could not be imported' do + project = build(:project) + project.errors.add(:name, 'is invalid') + project.errors.add(:path, 'is old') + allow(Gitlab::LegacyGithubImport::ProjectCreator) .to receive(:new).with(provider_repo, provider_repo.name, user.namespace, user, access_params, type: provider) - .and_return(double(execute: build(:project))) + .and_return(double(execute: project)) post :create, format: :json expect(response).to have_gitlab_http_status(422) + expect(json_response['errors']).to eq('Name is invalid, Path is old') end context "when the repository owner is the provider user" do diff --git a/spec/support/helpers/cycle_analytics_helpers.rb b/spec/support/helpers/cycle_analytics_helpers.rb index 06a76d53354..32d9807f06a 100644 --- a/spec/support/helpers/cycle_analytics_helpers.rb +++ b/spec/support/helpers/cycle_analytics_helpers.rb @@ -123,7 +123,11 @@ module CycleAnalyticsHelpers if branch_update.newrev _, opts = args - commit = raw_repository.commit(branch_update.newrev).rugged_commit + + commit = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + raw_repository.commit(branch_update.newrev).rugged_commit + end + branch_update.newrev = commit.amend( update_ref: "#{Gitlab::Git::BRANCH_REF_PREFIX}#{opts[:branch_name]}", author: commit.author.merge(time: new_date), diff --git a/spec/support/helpers/graphql_helpers.rb b/spec/support/helpers/graphql_helpers.rb new file mode 100644 index 00000000000..30ff9a1196a --- /dev/null +++ b/spec/support/helpers/graphql_helpers.rb @@ -0,0 +1,90 @@ +module GraphqlHelpers + # makes an underscored string look like a fieldname + # "merge_request" => "mergeRequest" + def self.fieldnamerize(underscored_field_name) + graphql_field_name = underscored_field_name.to_s.camelize + graphql_field_name[0] = graphql_field_name[0].downcase + + graphql_field_name + end + + # Run a loader's named resolver + def resolve(resolver_class, obj: nil, args: {}, ctx: {}) + resolver_class.new(object: obj, context: ctx).resolve(args) + end + + # Runs a block inside a BatchLoader::Executor wrapper + def batch(max_queries: nil, &blk) + wrapper = proc do + begin + BatchLoader::Executor.ensure_current + yield + ensure + BatchLoader::Executor.clear_current + end + end + + if max_queries + result = nil + expect { result = wrapper.call }.not_to exceed_query_limit(max_queries) + result + else + wrapper.call + end + end + + def graphql_query_for(name, attributes = {}, fields = nil) + fields ||= all_graphql_fields_for(name.classify) + attributes = attributes_to_graphql(attributes) + <<~QUERY + { + #{name}(#{attributes}) { + #{fields} + } + } + QUERY + end + + def all_graphql_fields_for(class_name) + type = GitlabSchema.types[class_name.to_s] + return "" unless type + + type.fields.map do |name, field| + if scalar?(field) + name + else + "#{name} { #{all_graphql_fields_for(field_type(field))} }" + end + end.join("\n") + end + + def attributes_to_graphql(attributes) + attributes.map do |name, value| + "#{GraphqlHelpers.fieldnamerize(name.to_s)}: \"#{value}\"" + end.join(", ") + end + + def post_graphql(query, current_user: nil) + post api('/', current_user, version: 'graphql'), query: query + end + + def graphql_data + json_response['data'] + end + + def graphql_errors + json_response['data'] + end + + def scalar?(field) + field_type(field).kind.scalar? + end + + def field_type(field) + if field.type.respond_to?(:of_type) + field.type.of_type + else + field.type + end + end +end diff --git a/spec/support/helpers/query_recorder.rb b/spec/support/helpers/query_recorder.rb index 28536bbef5e..7ce63375d34 100644 --- a/spec/support/helpers/query_recorder.rb +++ b/spec/support/helpers/query_recorder.rb @@ -1,10 +1,11 @@ module ActiveRecord class QueryRecorder - attr_reader :log, :cached + attr_reader :log, :skip_cached, :cached - def initialize(&block) + def initialize(skip_cached: true, &block) @log = [] @cached = [] + @skip_cached = skip_cached ActiveSupport::Notifications.subscribed(method(:callback), 'sql.active_record', &block) end @@ -16,7 +17,7 @@ module ActiveRecord def callback(name, start, finish, message_id, values) show_backtrace(values) if ENV['QUERY_RECORDER_DEBUG'] - if values[:name]&.include?("CACHE") + if values[:name]&.include?("CACHE") && skip_cached @cached << values[:sql] elsif !values[:name]&.include?("SCHEMA") @log << values[:sql] diff --git a/spec/support/helpers/stub_object_storage.rb b/spec/support/helpers/stub_object_storage.rb index 19d744b959a..bceaf8277ee 100644 --- a/spec/support/helpers/stub_object_storage.rb +++ b/spec/support/helpers/stub_object_storage.rb @@ -45,4 +45,16 @@ module StubObjectStorage remote_directory: 'uploads', **params) end + + def stub_object_storage_multipart_init(endpoint, upload_id = "upload_id") + stub_request(:post, %r{\A#{endpoint}tmp/uploads/[a-z0-9-]*\?uploads\z}) + .to_return status: 200, body: <<-EOS.strip_heredoc + <?xml version="1.0" encoding="UTF-8"?> + <InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> + <Bucket>example-bucket</Bucket> + <Key>example-object</Key> + <UploadId>#{upload_id}</UploadId> + </InitiateMultipartUploadResult> + EOS + end end diff --git a/spec/support/matchers/email_matchers.rb b/spec/support/matchers/email_matchers.rb deleted file mode 100644 index d9d59ec12ec..00000000000 --- a/spec/support/matchers/email_matchers.rb +++ /dev/null @@ -1,5 +0,0 @@ -RSpec::Matchers.define :have_html_escaped_body_text do |expected| - match do |actual| - expect(actual).to have_body_text(ERB::Util.html_escape(expected)) - end -end diff --git a/spec/support/matchers/exceed_query_limit.rb b/spec/support/matchers/exceed_query_limit.rb index 88d22a3ddd9..cd042401f3a 100644 --- a/spec/support/matchers/exceed_query_limit.rb +++ b/spec/support/matchers/exceed_query_limit.rb @@ -1,17 +1,4 @@ -RSpec::Matchers.define :exceed_query_limit do |expected| - supports_block_expectations - - match do |block| - @subject_block = block - actual_count > expected_count + threshold - end - - failure_message_when_negated do |actual| - threshold_message = threshold > 0 ? " (+#{@threshold})" : '' - counts = "#{expected_count}#{threshold_message}" - "Expected a maximum of #{counts} queries, got #{actual_count}:\n\n#{log_message}" - end - +module ExceedQueryLimitHelpers def with_threshold(threshold) @threshold = threshold self @@ -43,7 +30,7 @@ RSpec::Matchers.define :exceed_query_limit do |expected| end def recorder - @recorder ||= ActiveRecord::QueryRecorder.new(&@subject_block) + @recorder ||= ActiveRecord::QueryRecorder.new(skip_cached: skip_cached, &@subject_block) end def count_queries(queries) @@ -61,4 +48,52 @@ RSpec::Matchers.define :exceed_query_limit do |expected| @recorder.log_message end end + + def skip_cached + true + end + + def verify_count(&block) + @subject_block = block + actual_count > expected_count + threshold + end + + def failure_message + threshold_message = threshold > 0 ? " (+#{@threshold})" : '' + counts = "#{expected_count}#{threshold_message}" + "Expected a maximum of #{counts} queries, got #{actual_count}:\n\n#{log_message}" + end +end + +RSpec::Matchers.define :exceed_all_query_limit do |expected| + supports_block_expectations + + include ExceedQueryLimitHelpers + + match do |block| + verify_count(&block) + end + + failure_message_when_negated do |actual| + failure_message + end + + def skip_cached + false + end +end + +# Excludes cached methods from the query count +RSpec::Matchers.define :exceed_query_limit do |expected| + supports_block_expectations + + include ExceedQueryLimitHelpers + + match do |block| + verify_count(&block) + end + + failure_message_when_negated do |actual| + failure_message + end end diff --git a/spec/support/matchers/graphql_matchers.rb b/spec/support/matchers/graphql_matchers.rb new file mode 100644 index 00000000000..ba7a1c8cde0 --- /dev/null +++ b/spec/support/matchers/graphql_matchers.rb @@ -0,0 +1,40 @@ +RSpec::Matchers.define :require_graphql_authorizations do |*expected| + match do |field| + field_definition = field.metadata[:type_class] + expect(field_definition).to respond_to(:required_permissions) + expect(field_definition.required_permissions).to contain_exactly(*expected) + end +end + +RSpec::Matchers.define :have_graphql_fields do |*expected| + match do |kls| + field_names = expected.map { |name| GraphqlHelpers.fieldnamerize(name) } + expect(kls.fields.keys).to contain_exactly(*field_names) + end +end + +RSpec::Matchers.define :have_graphql_arguments do |*expected| + include GraphqlHelpers + + match do |field| + argument_names = expected.map { |name| GraphqlHelpers.fieldnamerize(name) } + expect(field.arguments.keys).to contain_exactly(*argument_names) + end +end + +RSpec::Matchers.define :have_graphql_type do |expected| + match do |field| + expect(field.type).to eq(expected.to_graphql) + end +end + +RSpec::Matchers.define :have_graphql_resolver do |expected| + match do |field| + case expected + when Method + expect(field.metadata[:type_class].resolve_proc).to eq(expected) + else + expect(field.metadata[:type_class].resolver).to eq(expected) + end + end +end diff --git a/spec/support/shared_examples/ci_trace_shared_examples.rb b/spec/support/shared_examples/ci_trace_shared_examples.rb index 21c6f3c829f..6dbe0f6f980 100644 --- a/spec/support/shared_examples/ci_trace_shared_examples.rb +++ b/spec/support/shared_examples/ci_trace_shared_examples.rb @@ -227,6 +227,42 @@ shared_examples_for 'common trace features' do end end end + + describe '#archive!' do + subject { trace.archive! } + + context 'when build status is success' do + let!(:build) { create(:ci_build, :success, :trace_live) } + + it 'does not have an archived trace yet' do + expect(build.job_artifacts_trace).to be_nil + end + + context 'when archives' do + it 'has an archived trace' do + subject + + build.reload + expect(build.job_artifacts_trace).to be_exist + end + + context 'when another process has already been archiving', :clean_gitlab_redis_shared_state do + before do + Gitlab::ExclusiveLease.new("trace:archive:#{trace.job.id}", timeout: 1.hour).try_obtain + end + + it 'blocks concurrent archiving' do + expect(Rails.logger).to receive(:error).with('Cannot obtain an exclusive lease. There must be another instance already in execution.') + + subject + + build.reload + expect(build.job_artifacts_trace).to be_nil + end + end + end + end + end end shared_examples_for 'trace with disabled live trace feature' do diff --git a/spec/support/shared_examples/models/atomic_internal_id_spec.rb b/spec/support/shared_examples/models/atomic_internal_id_spec.rb index 6a6e13418a9..7ab1041d17c 100644 --- a/spec/support/shared_examples/models/atomic_internal_id_spec.rb +++ b/spec/support/shared_examples/models/atomic_internal_id_spec.rb @@ -1,6 +1,6 @@ require 'spec_helper' -shared_examples_for 'AtomicInternalId' do +shared_examples_for 'AtomicInternalId' do |validate_presence: true| describe '.has_internal_id' do describe 'Module inclusion' do subject { described_class } @@ -9,14 +9,31 @@ shared_examples_for 'AtomicInternalId' do end describe 'Validation' do - subject { instance } - before do - allow(InternalId).to receive(:generate_next).and_return(nil) + allow_any_instance_of(described_class).to receive(:"ensure_#{scope}_#{internal_id_attribute}!") + + instance.valid? end - it { is_expected.to validate_presence_of(internal_id_attribute) } - it { is_expected.to validate_numericality_of(internal_id_attribute) } + context 'when presence validation is required' do + before do + skip unless validate_presence + end + + it 'validates presence' do + expect(instance.errors[internal_id_attribute]).to include("can't be blank") + end + end + + context 'when presence validation is not required' do + before do + skip if validate_presence + end + + it 'does not validate presence' do + expect(instance.errors[internal_id_attribute]).to be_empty + end + end end describe 'Creating an instance' do diff --git a/spec/support/shared_examples/notify_shared_examples.rb b/spec/support/shared_examples/notify_shared_examples.rb index 43fdaddf545..d176d3fa425 100644 --- a/spec/support/shared_examples/notify_shared_examples.rb +++ b/spec/support/shared_examples/notify_shared_examples.rb @@ -212,7 +212,7 @@ shared_examples 'a note email' do end it 'contains the message from the note' do - is_expected.to have_html_escaped_body_text note.note + is_expected.to have_body_text note.note end it 'does not contain note author' do @@ -225,7 +225,7 @@ shared_examples 'a note email' do end it 'contains a link to note author' do - is_expected.to have_html_escaped_body_text note.author_name + is_expected.to have_body_text note.author_name end end end diff --git a/spec/support/shared_examples/requests/graphql_shared_examples.rb b/spec/support/shared_examples/requests/graphql_shared_examples.rb new file mode 100644 index 00000000000..9b2b74593a5 --- /dev/null +++ b/spec/support/shared_examples/requests/graphql_shared_examples.rb @@ -0,0 +1,11 @@ +require 'spec_helper' + +shared_examples 'a working graphql query' do + include GraphqlHelpers + + it 'is returns a successfull response', :aggregate_failures do + expect(response).to be_success + expect(graphql_errors['errors']).to be_nil + expect(json_response.keys).to include('data') + end +end diff --git a/spec/support/trace/trace_helpers.rb b/spec/support/trace/trace_helpers.rb new file mode 100644 index 00000000000..c7802bbcb94 --- /dev/null +++ b/spec/support/trace/trace_helpers.rb @@ -0,0 +1,27 @@ +module TraceHelpers + def create_legacy_trace(build, content) + File.open(legacy_trace_path(build), 'wb') { |stream| stream.write(content) } + end + + def create_legacy_trace_in_db(build, content) + build.update_column(:trace, content) + end + + def legacy_trace_path(build) + legacy_trace_dir = File.join(Settings.gitlab_ci.builds_path, + build.created_at.utc.strftime("%Y_%m"), + build.project_id.to_s) + + FileUtils.mkdir_p(legacy_trace_dir) + + File.join(legacy_trace_dir, "#{build.id}.log") + end + + def archived_trace_path(job_artifact) + disk_hash = Digest::SHA2.hexdigest(job_artifact.project_id.to_s) + creation_date = job_artifact.created_at.utc.strftime('%Y_%m_%d') + + File.join(Gitlab.config.artifacts.path, disk_hash[0..1], disk_hash[2..3], disk_hash, + creation_date, job_artifact.job_id.to_s, job_artifact.id.to_s, 'job.log') + end +end diff --git a/spec/tasks/gitlab/backup_rake_spec.rb b/spec/tasks/gitlab/backup_rake_spec.rb index c9252bebb2e..93a436cb2b5 100644 --- a/spec/tasks/gitlab/backup_rake_spec.rb +++ b/spec/tasks/gitlab/backup_rake_spec.rb @@ -101,7 +101,9 @@ describe 'gitlab:app namespace rake task' do before do stub_env('SKIP', 'db') - path = File.join(project.repository.path_to_repo, 'custom_hooks') + path = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + File.join(project.repository.path_to_repo, 'custom_hooks') + end FileUtils.mkdir_p(path) FileUtils.touch(File.join(path, "dummy.txt")) end @@ -122,7 +124,10 @@ describe 'gitlab:app namespace rake task' do expect { run_rake_task('gitlab:backup:create') }.to output.to_stdout expect { run_rake_task('gitlab:backup:restore') }.to output.to_stdout - expect(Dir.entries(File.join(project.repository.path, 'custom_hooks'))).to include("dummy.txt") + repo_path = Gitlab::GitalyClient::StorageSettings.allow_disk_access do + project.repository.path + end + expect(Dir.entries(File.join(repo_path, 'custom_hooks'))).to include("dummy.txt") end end @@ -243,10 +248,12 @@ describe 'gitlab:app namespace rake task' do FileUtils.mkdir_p(b_storage_dir) # Even when overriding the storage, we have to move it there, so it exists - FileUtils.mv( - File.join(Settings.absolute(storages['default'].legacy_disk_path), project_b.repository.disk_path + '.git'), - Rails.root.join(storages['test_second_storage'].legacy_disk_path, project_b.repository.disk_path + '.git') - ) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + FileUtils.mv( + File.join(Settings.absolute(storages['default'].legacy_disk_path), project_b.repository.disk_path + '.git'), + Rails.root.join(storages['test_second_storage'].legacy_disk_path, project_b.repository.disk_path + '.git') + ) + end expect { run_rake_task('gitlab:backup:create') }.to output.to_stdout diff --git a/spec/uploaders/object_storage_spec.rb b/spec/uploaders/object_storage_spec.rb index 2dd0925a8e6..4503288e410 100644 --- a/spec/uploaders/object_storage_spec.rb +++ b/spec/uploaders/object_storage_spec.rb @@ -355,14 +355,10 @@ describe ObjectStorage do end describe '.workhorse_authorize' do - subject { uploader_class.workhorse_authorize } + let(:has_length) { true } + let(:maximum_size) { nil } - before do - # ensure that we use regular Fog libraries - # other tests might call `Fog.mock!` and - # it will make tests to fail - Fog.unmock! - end + subject { uploader_class.workhorse_authorize(has_length: has_length, maximum_size: maximum_size) } shared_examples 'uses local storage' do it "returns temporary path" do @@ -371,10 +367,6 @@ describe ObjectStorage do expect(subject[:TempPath]).to start_with(uploader_class.root) expect(subject[:TempPath]).to include(described_class::TMP_UPLOAD_PATH) end - - it "does not return remote store" do - is_expected.not_to have_key('RemoteObject') - end end shared_examples 'uses remote storage' do @@ -383,7 +375,7 @@ describe ObjectStorage do expect(subject[:RemoteObject]).to have_key(:ID) expect(subject[:RemoteObject]).to include(Timeout: a_kind_of(Integer)) - expect(subject[:RemoteObject][:Timeout]).to be(ObjectStorage::DIRECT_UPLOAD_TIMEOUT) + expect(subject[:RemoteObject][:Timeout]).to be(ObjectStorage::DirectUpload::TIMEOUT) expect(subject[:RemoteObject]).to have_key(:GetURL) expect(subject[:RemoteObject]).to have_key(:DeleteURL) expect(subject[:RemoteObject]).to have_key(:StoreURL) @@ -391,9 +383,31 @@ describe ObjectStorage do expect(subject[:RemoteObject][:DeleteURL]).to include(described_class::TMP_UPLOAD_PATH) expect(subject[:RemoteObject][:StoreURL]).to include(described_class::TMP_UPLOAD_PATH) end + end - it "does not return local store" do - is_expected.not_to have_key('TempPath') + shared_examples 'uses remote storage with multipart uploads' do + it_behaves_like 'uses remote storage' do + it "returns multipart upload" do + is_expected.to have_key(:RemoteObject) + + expect(subject[:RemoteObject]).to have_key(:MultipartUpload) + expect(subject[:RemoteObject][:MultipartUpload]).to have_key(:PartSize) + expect(subject[:RemoteObject][:MultipartUpload]).to have_key(:PartURLs) + expect(subject[:RemoteObject][:MultipartUpload]).to have_key(:CompleteURL) + expect(subject[:RemoteObject][:MultipartUpload]).to have_key(:AbortURL) + expect(subject[:RemoteObject][:MultipartUpload][:PartURLs]).to all(include(described_class::TMP_UPLOAD_PATH)) + expect(subject[:RemoteObject][:MultipartUpload][:CompleteURL]).to include(described_class::TMP_UPLOAD_PATH) + expect(subject[:RemoteObject][:MultipartUpload][:AbortURL]).to include(described_class::TMP_UPLOAD_PATH) + end + end + end + + shared_examples 'uses remote storage without multipart uploads' do + it_behaves_like 'uses remote storage' do + it "does not return multipart upload" do + is_expected.to have_key(:RemoteObject) + expect(subject[:RemoteObject]).not_to have_key(:MultipartUpload) + end end end @@ -416,6 +430,8 @@ describe ObjectStorage do end context 'uses AWS' do + let(:storage_url) { "https://uploads.s3-eu-central-1.amazonaws.com/" } + before do expect(uploader_class).to receive(:object_store_credentials) do { provider: "AWS", @@ -425,18 +441,40 @@ describe ObjectStorage do end end - it_behaves_like 'uses remote storage' do - let(:storage_url) { "https://uploads.s3-eu-central-1.amazonaws.com/" } + context 'for known length' do + it_behaves_like 'uses remote storage without multipart uploads' do + it 'returns links for S3' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + end + end + end - it 'returns links for S3' do - expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + context 'for unknown length' do + let(:has_length) { false } + let(:maximum_size) { 1.gigabyte } + + before do + stub_object_storage_multipart_init(storage_url) + end + + it_behaves_like 'uses remote storage with multipart uploads' do + it 'returns links for S3' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:MultipartUpload][:PartURLs]).to all(start_with(storage_url)) + expect(subject[:RemoteObject][:MultipartUpload][:CompleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:MultipartUpload][:AbortURL]).to start_with(storage_url) + end end end end context 'uses Google' do + let(:storage_url) { "https://storage.googleapis.com/uploads/" } + before do expect(uploader_class).to receive(:object_store_credentials) do { provider: "Google", @@ -445,36 +483,71 @@ describe ObjectStorage do end end - it_behaves_like 'uses remote storage' do - let(:storage_url) { "https://storage.googleapis.com/uploads/" } + context 'for known length' do + it_behaves_like 'uses remote storage without multipart uploads' do + it 'returns links for Google Cloud' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + end + end + end + + context 'for unknown length' do + let(:has_length) { false } + let(:maximum_size) { 1.gigabyte } - it 'returns links for Google Cloud' do - expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + it_behaves_like 'uses remote storage without multipart uploads' do + it 'returns links for Google Cloud' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + end end end end context 'uses GDK/minio' do + let(:storage_url) { "http://minio:9000/uploads/" } + before do expect(uploader_class).to receive(:object_store_credentials) do { provider: "AWS", aws_access_key_id: "AWS_ACCESS_KEY_ID", aws_secret_access_key: "AWS_SECRET_ACCESS_KEY", - endpoint: 'http://127.0.0.1:9000', + endpoint: 'http://minio:9000', path_style: true, region: "gdk" } end end - it_behaves_like 'uses remote storage' do - let(:storage_url) { "http://127.0.0.1:9000/uploads/" } + context 'for known length' do + it_behaves_like 'uses remote storage without multipart uploads' do + it 'returns links for S3' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + end + end + end + + context 'for unknown length' do + let(:has_length) { false } + let(:maximum_size) { 1.gigabyte } - it 'returns links for S3' do - expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) - expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + before do + stub_object_storage_multipart_init(storage_url) + end + + it_behaves_like 'uses remote storage with multipart uploads' do + it 'returns links for S3' do + expect(subject[:RemoteObject][:GetURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:DeleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:StoreURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:MultipartUpload][:PartURLs]).to all(start_with(storage_url)) + expect(subject[:RemoteObject][:MultipartUpload][:CompleteURL]).to start_with(storage_url) + expect(subject[:RemoteObject][:MultipartUpload][:AbortURL]).to start_with(storage_url) + end end end end @@ -659,4 +732,26 @@ describe ObjectStorage do end end end + + describe '#retrieve_from_store!' do + [:group, :project, :user].each do |model| + context "for #{model}s" do + let(:models) { create_list(model, 3, :with_avatar).map(&:reload) } + let(:avatars) { models.map(&:avatar) } + + it 'batches fetching uploads from the database' do + # Ensure that these are all created and fully loaded before we start + # running queries for avatars + models + + expect { avatars }.not_to exceed_query_limit(1) + end + + it 'fetches a unique upload for each model' do + expect(avatars.map(&:url).uniq).to eq(avatars.map(&:url)) + expect(avatars.map(&:upload).uniq).to eq(avatars.map(&:upload)) + end + end + end + end end diff --git a/spec/uploaders/workers/object_storage/background_move_worker_spec.rb b/spec/uploaders/workers/object_storage/background_move_worker_spec.rb index b34f427fd8a..95813d15e52 100644 --- a/spec/uploaders/workers/object_storage/background_move_worker_spec.rb +++ b/spec/uploaders/workers/object_storage/background_move_worker_spec.rb @@ -125,8 +125,10 @@ describe ObjectStorage::BackgroundMoveWorker do it "migrates file to remote storage" do perform + project.reload + BatchLoader::Executor.clear_current - expect(project.reload.avatar.file_storage?).to be_falsey + expect(project.avatar).not_to be_file_storage end end @@ -137,7 +139,7 @@ describe ObjectStorage::BackgroundMoveWorker do it "migrates file to remote storage" do perform - expect(project.reload.avatar.file_storage?).to be_falsey + expect(project.reload.avatar).not_to be_file_storage end end end diff --git a/spec/workers/ci/archive_traces_cron_worker_spec.rb b/spec/workers/ci/archive_traces_cron_worker_spec.rb new file mode 100644 index 00000000000..9af51b7d4d8 --- /dev/null +++ b/spec/workers/ci/archive_traces_cron_worker_spec.rb @@ -0,0 +1,59 @@ +require 'spec_helper' + +describe Ci::ArchiveTracesCronWorker do + subject { described_class.new.perform } + + before do + stub_feature_flags(ci_enable_live_trace: true) + end + + shared_examples_for 'archives trace' do + it do + subject + + build.reload + expect(build.job_artifacts_trace).to be_exist + end + end + + shared_examples_for 'does not archive trace' do + it do + subject + + build.reload + expect(build.job_artifacts_trace).to be_nil + end + end + + context 'when a job was succeeded' do + let!(:build) { create(:ci_build, :success, :trace_live) } + + it_behaves_like 'archives trace' + + context 'when archive raised an exception' do + let!(:build) { create(:ci_build, :success, :trace_artifact, :trace_live) } + let!(:build2) { create(:ci_build, :success, :trace_live) } + + it 'archives valid targets' do + expect(Rails.logger).to receive(:error).with("Failed to archive stale live trace. id: #{build.id} message: Already archived") + + subject + + build2.reload + expect(build2.job_artifacts_trace).to be_exist + end + end + end + + context 'when a job was cancelled' do + let!(:build) { create(:ci_build, :canceled, :trace_live) } + + it_behaves_like 'archives trace' + end + + context 'when a job is running' do + let!(:build) { create(:ci_build, :running, :trace_live) } + + it_behaves_like 'does not archive trace' + end +end diff --git a/spec/workers/git_garbage_collect_worker_spec.rb b/spec/workers/git_garbage_collect_worker_spec.rb index 74539a7e493..f44b4edc305 100644 --- a/spec/workers/git_garbage_collect_worker_spec.rb +++ b/spec/workers/git_garbage_collect_worker_spec.rb @@ -104,7 +104,7 @@ describe GitGarbageCollectWorker do it_should_behave_like 'flushing ref caches', true end - context "with Gitaly turned off", :skip_gitaly_mock do + context "with Gitaly turned off", :disable_gitaly do it_should_behave_like 'flushing ref caches', false end diff --git a/spec/workers/gitlab/github_import/stage/import_lfs_objects_worker_spec.rb b/spec/workers/gitlab/github_import/stage/import_lfs_objects_worker_spec.rb new file mode 100644 index 00000000000..b19884d7991 --- /dev/null +++ b/spec/workers/gitlab/github_import/stage/import_lfs_objects_worker_spec.rb @@ -0,0 +1,28 @@ +require 'spec_helper' + +describe Gitlab::GithubImport::Stage::ImportLfsObjectsWorker do + let(:project) { create(:project) } + let(:worker) { described_class.new } + + describe '#import' do + it 'imports all the lfs objects' do + importer = double(:importer) + waiter = Gitlab::JobWaiter.new(2, '123') + + expect(Gitlab::GithubImport::Importer::LfsObjectsImporter) + .to receive(:new) + .with(project, nil) + .and_return(importer) + + expect(importer) + .to receive(:execute) + .and_return(waiter) + + expect(Gitlab::GithubImport::AdvanceStageWorker) + .to receive(:perform_async) + .with(project.id, { '123' => 2 }, :finish) + + worker.import(project) + end + end +end diff --git a/spec/workers/gitlab/github_import/stage/import_notes_worker_spec.rb b/spec/workers/gitlab/github_import/stage/import_notes_worker_spec.rb index 098d2d55386..94cff9e4e80 100644 --- a/spec/workers/gitlab/github_import/stage/import_notes_worker_spec.rb +++ b/spec/workers/gitlab/github_import/stage/import_notes_worker_spec.rb @@ -21,7 +21,7 @@ describe Gitlab::GithubImport::Stage::ImportNotesWorker do expect(Gitlab::GithubImport::AdvanceStageWorker) .to receive(:perform_async) - .with(project.id, { '123' => 2 }, :finish) + .with(project.id, { '123' => 2 }, :lfs_objects) worker.import(client, project) end diff --git a/spec/workers/repository_check/single_repository_worker_spec.rb b/spec/workers/repository_check/single_repository_worker_spec.rb index a021235aed6..22fc64c1536 100644 --- a/spec/workers/repository_check/single_repository_worker_spec.rb +++ b/spec/workers/repository_check/single_repository_worker_spec.rb @@ -88,7 +88,9 @@ describe RepositoryCheck::SingleRepositoryWorker do end def break_wiki(project) - break_repo(wiki_path(project)) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + break_repo(wiki_path(project)) + end end def wiki_path(project) @@ -96,7 +98,9 @@ describe RepositoryCheck::SingleRepositoryWorker do end def break_project(project) - break_repo(project.repository.path_to_repo) + Gitlab::GitalyClient::StorageSettings.allow_disk_access do + break_repo(project.repository.path_to_repo) + end end def break_repo(repo) diff --git a/spec/workers/stuck_ci_jobs_worker_spec.rb b/spec/workers/stuck_ci_jobs_worker_spec.rb index bdc64c6785b..2605c14334f 100644 --- a/spec/workers/stuck_ci_jobs_worker_spec.rb +++ b/spec/workers/stuck_ci_jobs_worker_spec.rb @@ -132,8 +132,10 @@ describe StuckCiJobsWorker do end it 'cancels exclusive lease after worker perform' do - expect(Gitlab::ExclusiveLease).to receive(:cancel).with(described_class::EXCLUSIVE_LEASE_KEY, exclusive_lease_uuid) worker.perform + + expect(Gitlab::ExclusiveLease.new(described_class::EXCLUSIVE_LEASE_KEY, timeout: 1.hour)) + .not_to be_exists end end end diff --git a/vendor/gitlab-ci-yml/Auto-DevOps.gitlab-ci.yml b/vendor/gitlab-ci-yml/Auto-DevOps.gitlab-ci.yml index 589ebcf1414..27e8245185e 100644 --- a/vendor/gitlab-ci-yml/Auto-DevOps.gitlab-ci.yml +++ b/vendor/gitlab-ci-yml/Auto-DevOps.gitlab-ci.yml @@ -7,6 +7,17 @@ # * creating a review app for each topic branch, # * and continuous deployment to production # +# Test jobs may be disabled by setting environment variables: +# * test: TEST_DISABLED +# * code_quality: CODE_QUALITY_DISABLED +# * performance: PERFORMANCE_DISABLED +# * sast: SAST_DISABLED +# * dependency_scanning: DEPENDENCY_SCANNING_DISABLED +# * container_scanning: CONTAINER_SCANNING_DISABLED +# * dast: DAST_DISABLED +# * review: REVIEW_DISABLED +# * stop_review: REVIEW_DISABLED +# # In order to deploy, you must have a Kubernetes cluster configured either # via a project integration, or via group/project variables. # AUTO_DEVOPS_DOMAIN must also be set as a variable at the group or project @@ -15,7 +26,7 @@ # Continuous deployment to production is enabled by default. # If you want to deploy to staging first, or enable incremental rollouts, # set STAGING_ENABLED or INCREMENTAL_ROLLOUT_ENABLED environment variables. -# If you want to use canary deployments, uncomment the canary job. +# If you want to use canary deployments, set CANARY_ENABLED environment variable. # # If Auto DevOps fails to detect the proper buildpack, or if you want to # specify a custom buildpack, set a project variable `BUILDPACK_URL` to the @@ -76,8 +87,12 @@ test: - /bin/herokuish buildpack test only: - branches + except: + variables: + - $TEST_DISABLED code_quality: + stage: test image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -89,6 +104,9 @@ code_quality: - code_quality artifacts: paths: [gl-code-quality-report.json] + except: + variables: + - $CODE_QUALITY_DISABLED performance: stage: performance @@ -109,8 +127,12 @@ performance: refs: - branches kubernetes: active + except: + variables: + - $PERFORMANCE_DISABLED sast: + stage: test image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -122,8 +144,12 @@ sast: - sast artifacts: paths: [gl-sast-report.json] + except: + variables: + - $SAST_DISABLED dependency_scanning: + stage: test image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -135,8 +161,12 @@ dependency_scanning: - dependency_scanning artifacts: paths: [gl-dependency-scanning-report.json] + except: + variables: + - $DEPENDENCY_SCANNING_DISABLED container_scanning: + stage: test image: docker:stable variables: DOCKER_DRIVER: overlay2 @@ -148,6 +178,9 @@ container_scanning: - container_scanning artifacts: paths: [gl-container-scanning-report.json] + except: + variables: + - $CONTAINER_SCANNING_DISABLED dast: stage: dast @@ -164,7 +197,10 @@ dast: - branches kubernetes: active except: - - master + refs: + - master + variables: + - $DAST_DISABLED review: stage: review @@ -188,7 +224,10 @@ review: - branches kubernetes: active except: - - master + refs: + - master + variables: + - $REVIEW_DISABLED stop_review: stage: cleanup @@ -207,7 +246,10 @@ stop_review: - branches kubernetes: active except: - - master + refs: + - master + variables: + - $REVIEW_DISABLED # Keys that start with a dot (.) will not be processed by GitLab CI. # Staging and canary jobs are disabled by default, to enable them @@ -240,10 +282,11 @@ staging: variables: - $STAGING_ENABLED -# Canaries are disabled by default, but if you want them, -# and know what the downsides are, enable this job by removing the dot (.). +# Canaries are also disabled by default, but if you want them, +# and know what the downsides are, you can enable this by setting +# CANARY_ENABLED. -.canary: +canary: stage: canary script: - check_kube_domain @@ -261,6 +304,8 @@ staging: refs: - master kubernetes: active + variables: + - $CANARY_ENABLED .production: &production_template stage: production @@ -290,6 +335,7 @@ production: except: variables: - $STAGING_ENABLED + - $CANARY_ENABLED - $INCREMENTAL_ROLLOUT_ENABLED production_manual: @@ -615,7 +661,7 @@ rollout 100%: function check_kube_domain() { if [ -z ${AUTO_DEVOPS_DOMAIN+x} ]; then echo "In order to deploy or use Review Apps, AUTO_DEVOPS_DOMAIN variable must be set" - echo "You can do it in Auto DevOps project settings or defining a secret variable at group or project level" + echo "You can do it in Auto DevOps project settings or defining a variable at group or project level" echo "You can also manually add it in .gitlab-ci.yml" false else @@ -624,7 +670,6 @@ rollout 100%: } function build() { - if [[ -n "$CI_REGISTRY_USER" ]]; then echo "Logging to GitLab Container Registry with CI credentials..." docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY" @@ -636,7 +681,7 @@ rollout 100%: docker build -t "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG" . else echo "Building Heroku-based application using gliderlabs/herokuish docker image..." - docker run -i --name="$CI_CONTAINER_NAME" -v "$(pwd):/tmp/app:ro" gliderlabs/herokuish /bin/herokuish buildpack build + docker run -i -e BUILDPACK_URL --name="$CI_CONTAINER_NAME" -v "$(pwd):/tmp/app:ro" gliderlabs/herokuish /bin/herokuish buildpack build docker commit "$CI_CONTAINER_NAME" "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG" docker rm "$CI_CONTAINER_NAME" >/dev/null echo "" |
