diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-03-29 21:08:53 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-03-29 21:08:53 +0000 |
commit | 31664a1a5ac22e8c56a471d3afab26e661efcc0e (patch) | |
tree | a300c578ef9877df4fdbe28774b509172d474ae0 | |
parent | 511cd681d4ab0d4263df538b1800058edc07230e (diff) | |
download | gitlab-ce-31664a1a5ac22e8c56a471d3afab26e661efcc0e.tar.gz |
Add latest changes from gitlab-org/gitlab@master
69 files changed, 709 insertions, 429 deletions
diff --git a/.gitlab/issue_templates/Feature Flag Roll Out.md b/.gitlab/issue_templates/Feature Flag Roll Out.md index 5791eca11ff..5efc9304a4e 100644 --- a/.gitlab/issue_templates/Feature Flag Roll Out.md +++ b/.gitlab/issue_templates/Feature Flag Roll Out.md @@ -43,9 +43,9 @@ Are there any other stages or teams involved that need to be kept in the loop? <!-- What are the settings we need to configure in order to have this feature viable? --> -<!-- +<!-- Example below: - + 1. Enable service ping collection `ApplicationSetting.first.update(usage_ping_enabled: true)` --> @@ -57,7 +57,7 @@ Example below: ### What can we monitor to detect problems with this? <!-- Which dashboards from https://dashboards.gitlab.net are most relevant? --> -_Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects +_Consider mentioning checks for 5xx errors or other anomalies like an increase in redirects (302 HTTP response status)_ ### What can we check for monitoring production after rollouts? @@ -66,7 +66,7 @@ _Consider adding links to check for Sentry errors, Production logs for 5xx, 302s ## Rollout Steps -Note: Please make sure to run the chatops commands in the slack channel that gets impacted by the command. +Note: Please make sure to run the chatops commands in the slack channel that gets impacted by the command. ### Rollout on non-production environments @@ -75,11 +75,15 @@ Note: Please make sure to run the chatops commands in the slack channel that get - [ ] `/chatops run auto_deploy status <merge-commit-of-your-feature>` - [ ] Enable the feature globally on non-production environments. - [ ] `/chatops run feature set <feature-flag-name> true --dev --staging --staging-ref` + - If the feature flag causes QA end-to-end tests to fail: + - [ ] Disable the feature flag on staging to avoid blocking [deployments](https://about.gitlab.com/handbook/engineering/deployments-and-releases/deployments/). - [ ] Verify that the feature works as expected. Posting the QA result in this issue is preferable. The best environment to validate the feature in is [staging-canary](https://about.gitlab.com/handbook/engineering/infrastructure/environments/#staging-canary) as this is the first environment deployed to. Note you will need to make sure you are configured to use canary as outlined [here](https://about.gitlab.com/handbook/engineering/infrastructure/environments/canary-stage/) when accessing the staging environment in order to make sure you are testing appropriately. +For assistance with QA end-to-end test failures, please reach out via the `#quality` Slack channel. Note that QA test failures on staging-ref [don't block deployments](https://about.gitlab.com/handbook/engineering/infrastructure/environments/staging-ref/#how-to-use-staging-ref). + ### Specific rollout on production For visibility, all `/chatops` commands that target production should be executed in the `#production` slack channel and cross-posted (with the command results) to the responsible team's slack channel (`#g_TEAM_NAME`). @@ -104,7 +108,7 @@ For visibility, all `/chatops` commands that target production should be execute - [ ] Ensure that you or a representative in development can be available for at least 2 hours after feature flag updates in production. If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the `@sre-oncall` Slack alias. - [ ] Ensure that documentation has been updated ([More info](https://docs.gitlab.com/ee/development/documentation/feature_flags.html#features-that-became-enabled-by-default)). -- [ ] Leave a comment on [the feature issue][main-issue] announcing estimated time when this feature flag will be enabled on GitLab.com. +- [ ] Leave a comment on [the feature issue][main-issue] announcing estimated time when this feature flag will be enabled on GitLab.com. - [ ] Ensure that any breaking changes have been announced following the [release post process](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes) to ensure GitLab customers are aware. - [ ] Notify `#support_gitlab-com` and your team channel ([more guidance when this is necessary in the dev docs](https://docs.gitlab.com/ee/development/feature_flags/controls.html#communicate-the-change)). - [ ] Ensure that the feature flag rollout plan is reviewed by another developer familiar with the domain. diff --git a/app/assets/javascripts/admin/abuse_reports/components/abuse_report_details.vue b/app/assets/javascripts/admin/abuse_reports/components/abuse_report_details.vue new file mode 100644 index 00000000000..f49411604f1 --- /dev/null +++ b/app/assets/javascripts/admin/abuse_reports/components/abuse_report_details.vue @@ -0,0 +1,66 @@ +<script> +import { uniqueId } from 'lodash'; +import { GlButton, GlCollapse } from '@gitlab/ui'; +import { getTimeago } from '~/lib/utils/datetime_utility'; +import { __, sprintf } from '~/locale'; +import SafeHtml from '~/vue_shared/directives/safe_html'; + +export default { + components: { + GlButton, + GlCollapse, + }, + directives: { SafeHtml }, + props: { + report: { + type: Object, + required: true, + }, + }, + data() { + return { + isVisible: false, + collapseId: uniqueId('abuse-report-detail-'), + }; + }, + computed: { + toggleText() { + return this.isVisible ? __('Hide details') : __('Show details'); + }, + reportedUserCreatedAt() { + const { reportedUser } = this.report; + return sprintf(__('User joined %{timeAgo}'), { + timeAgo: getTimeago().format(reportedUser.createdAt), + }); + }, + }, + methods: { + toggleCollapse() { + this.isVisible = !this.isVisible; + }, + }, +}; +</script> + +<template> + <div class="gl-display-flex gl-flex-direction-column"> + <gl-collapse :id="collapseId" v-model="isVisible"> + <dl class="gl-mb-2"> + <dd>{{ reportedUserCreatedAt }}</dd> + + <dt>{{ __('Message') }}</dt> + <dd v-safe-html="report.message"></dd> + </dl> + </gl-collapse> + <div> + <gl-button + :aria-expanded="`${isVisible}`" + :aria-controls="collapseId" + size="small" + variant="link" + @click="toggleCollapse" + >{{ toggleText }} + </gl-button> + </div> + </div> +</template> diff --git a/app/assets/javascripts/admin/abuse_reports/components/abuse_report_row.vue b/app/assets/javascripts/admin/abuse_reports/components/abuse_report_row.vue index f3cbf975aab..a9fe59a7b85 100644 --- a/app/assets/javascripts/admin/abuse_reports/components/abuse_report_row.vue +++ b/app/assets/javascripts/admin/abuse_reports/components/abuse_report_row.vue @@ -1,13 +1,17 @@ <script> import { GlSprintf, GlLink } from '@gitlab/ui'; import { getTimeago } from '~/lib/utils/datetime_utility'; +import { queryToObject } from '~/lib/utils/url_utility'; import { __, sprintf } from '~/locale'; import ListItem from '~/vue_shared/components/registry/list_item.vue'; +import { SORT_UPDATED_AT } from '../constants'; import AbuseReportActions from './abuse_report_actions.vue'; +import AbuseReportDetails from './abuse_report_details.vue'; export default { name: 'AbuseReportRow', components: { + AbuseReportDetails, GlLink, GlSprintf, AbuseReportActions, @@ -20,9 +24,14 @@ export default { }, }, computed: { - updatedAt() { - const template = __('Updated %{timeAgo}'); - return sprintf(template, { timeAgo: getTimeago().format(this.report.updatedAt) }); + displayDate() { + const { sort } = queryToObject(window.location.search); + const { createdAt, updatedAt } = this.report; + const { template, timeAgo } = Object.values(SORT_UPDATED_AT.sortDirection).includes(sort) + ? { template: __('Updated %{timeAgo}'), timeAgo: updatedAt } + : { template: __('Created %{timeAgo}'), timeAgo: createdAt }; + + return sprintf(template, { timeAgo: getTimeago().format(timeAgo) }); }, reported() { const { reportedUser } = this.report; @@ -48,7 +57,7 @@ export default { <template> <list-item data-testid="abuse-report-row"> <template #left-primary> - <div class="gl-font-weight-normal" data-testid="title"> + <div class="gl-font-weight-normal gl-mb-2" data-testid="title"> <gl-sprintf :message="title"> <template #userLink="{ content }"> <gl-link :href="report.reportedUserPath">{{ content }}</gl-link> @@ -60,9 +69,12 @@ export default { </div> </template> - <template #right-secondary> - <div data-testid="updated-at">{{ updatedAt }}</div> + <template #left-secondary> + <abuse-report-details :report="report" /> + </template> + <template #right-secondary> + <div data-testid="abuse-report-date">{{ displayDate }}</div> <abuse-report-actions :report="report" /> </template> </list-item> diff --git a/app/assets/javascripts/admin/abuse_reports/constants.js b/app/assets/javascripts/admin/abuse_reports/constants.js index 8b1045fd531..ee002f269ac 100644 --- a/app/assets/javascripts/admin/abuse_reports/constants.js +++ b/app/assets/javascripts/admin/abuse_reports/constants.js @@ -40,25 +40,24 @@ export const FILTERED_SEARCH_TOKEN_STATUS = { }; export const DEFAULT_SORT = 'created_at_desc'; - -export const SORT_OPTIONS = [ - { - id: 10, - title: __('Created date'), - sortDirection: { - descending: DEFAULT_SORT, - ascending: 'created_at_asc', - }, +export const SORT_UPDATED_AT = Object.freeze({ + id: 20, + title: __('Updated date'), + sortDirection: { + descending: 'updated_at_desc', + ascending: 'updated_at_asc', }, - { - id: 20, - title: __('Updated date'), - sortDirection: { - descending: 'updated_at_desc', - ascending: 'updated_at_asc', - }, +}); +const SORT_CREATED_AT = Object.freeze({ + id: 10, + title: __('Created date'), + sortDirection: { + descending: DEFAULT_SORT, + ascending: 'created_at_asc', }, -]; +}); + +export const SORT_OPTIONS = [SORT_CREATED_AT, SORT_UPDATED_AT]; export const isValidSortKey = (key) => SORT_OPTIONS.some( diff --git a/app/assets/javascripts/content_editor/components/wrappers/reference.vue b/app/assets/javascripts/content_editor/components/wrappers/reference.vue new file mode 100644 index 00000000000..bd16773d291 --- /dev/null +++ b/app/assets/javascripts/content_editor/components/wrappers/reference.vue @@ -0,0 +1,22 @@ +<script> +import { NodeViewWrapper } from '@tiptap/vue-2'; + +export default { + name: 'DetailsWrapper', + components: { + NodeViewWrapper, + }, + props: { + node: { + type: Object, + required: true, + }, + }, +}; +</script> +<template> + <node-view-wrapper class="gl-display-inline-block"> + <span v-if="node.attrs.referenceType === 'command'">{{ node.attrs.text }}</span> + <a v-else href="#" @click.prevent.stop>{{ node.attrs.text }}</a> + </node-view-wrapper> +</template> diff --git a/app/assets/javascripts/content_editor/components/wrappers/label.vue b/app/assets/javascripts/content_editor/components/wrappers/reference_label.vue index 4206c866032..4206c866032 100644 --- a/app/assets/javascripts/content_editor/components/wrappers/label.vue +++ b/app/assets/javascripts/content_editor/components/wrappers/reference_label.vue diff --git a/app/assets/javascripts/content_editor/extensions/reference.js b/app/assets/javascripts/content_editor/extensions/reference.js index 707beaf1231..b56aa8596a0 100644 --- a/app/assets/javascripts/content_editor/extensions/reference.js +++ b/app/assets/javascripts/content_editor/extensions/reference.js @@ -1,4 +1,6 @@ import { Node } from '@tiptap/core'; +import { VueNodeViewRenderer } from '@tiptap/vue-2'; +import ReferenceWrapper from '../components/wrappers/reference.vue'; import { PARSE_HTML_PRIORITY_HIGHEST } from '../constants'; const getAnchor = (element) => { @@ -49,7 +51,7 @@ export default Node.create({ ]; }, - renderHTML({ node }) { - return ['a', { href: '#' }, node.attrs.text]; + addNodeView() { + return new VueNodeViewRenderer(ReferenceWrapper); }, }); diff --git a/app/assets/javascripts/content_editor/extensions/reference_label.js b/app/assets/javascripts/content_editor/extensions/reference_label.js index 9dff0b7a689..0441f8ef8d2 100644 --- a/app/assets/javascripts/content_editor/extensions/reference_label.js +++ b/app/assets/javascripts/content_editor/extensions/reference_label.js @@ -1,6 +1,6 @@ import { VueNodeViewRenderer } from '@tiptap/vue-2'; import { SCOPED_LABEL_DELIMITER } from '~/sidebar/components/labels/labels_select_widget/constants'; -import LabelWrapper from '../components/wrappers/label.vue'; +import LabelWrapper from '../components/wrappers/reference_label.vue'; import Reference from './reference'; export default Reference.extend({ diff --git a/app/assets/javascripts/editor/schema/ci.json b/app/assets/javascripts/editor/schema/ci.json index a5080332b78..44944a4a205 100644 --- a/app/assets/javascripts/editor/schema/ci.json +++ b/app/assets/javascripts/editor/schema/ci.json @@ -1894,6 +1894,10 @@ } }, "additionalProperties": false + }, + "publish": { + "description": "A path to a directory that contains the files to be published with Pages", + "type": "string" } }, "oneOf": [ diff --git a/app/models/ci/pipeline.rb b/app/models/ci/pipeline.rb index 7c4da0bc01e..a1db808feb5 100644 --- a/app/models/ci/pipeline.rb +++ b/app/models/ci/pipeline.rb @@ -325,12 +325,6 @@ module Ci end end - after_transition running: ::Ci::Pipeline.completed_statuses + [:manual] do |pipeline| - pipeline.run_after_commit do - ::Ci::UnlockRefArtifactsOnPipelineStopWorker.perform_async(pipeline.id) - end - end - after_transition any => [:success, :failed] do |pipeline| ref_status = pipeline.ci_ref&.update_status_by!(pipeline) diff --git a/app/models/ci/ref.rb b/app/models/ci/ref.rb index 32ea63b63d1..199e1cd07e7 100644 --- a/app/models/ci/ref.rb +++ b/app/models/ci/ref.rb @@ -30,6 +30,15 @@ module Ci state :fixed, value: 3 state :broken, value: 4 state :still_failing, value: 5 + + after_transition any => [:fixed, :success] do |ci_ref| + # Do not try to unlock if no artifacts are locked + next unless ci_ref.artifacts_locked? + + ci_ref.run_after_commit do + Ci::PipelineSuccessUnlockArtifactsWorker.perform_async(ci_ref.last_finished_pipeline_id) + end + end end class << self @@ -46,10 +55,6 @@ module Ci Ci::Pipeline.last_finished_for_ref_id(self.id)&.id end - def last_successful_pipeline - pipelines.ci_sources.success.order(id: :desc).take - end - def artifacts_locked? self.pipelines.where(locked: :artifacts_locked).exists? end diff --git a/app/models/concerns/enums/internal_id.rb b/app/models/concerns/enums/internal_id.rb index a8227363a22..8e161c1513f 100644 --- a/app/models/concerns/enums/internal_id.rb +++ b/app/models/concerns/enums/internal_id.rb @@ -17,7 +17,8 @@ module Enums sprints: 9, # iterations design_management_designs: 10, incident_management_oncall_schedules: 11, - ml_experiments: 12 + ml_experiments: 12, + ml_candidates: 13 } end end diff --git a/app/models/ml/candidate.rb b/app/models/ml/candidate.rb index 02da09e6b51..c1409da05ec 100644 --- a/app/models/ml/candidate.rb +++ b/app/models/ml/candidate.rb @@ -3,7 +3,9 @@ module Ml class Candidate < ApplicationRecord include Sortable + include AtomicInternalId include IgnorableColumns + ignore_column :iid, remove_with: '16.0', remove_after: '2023-05-01' PACKAGE_PREFIX = 'ml_candidate_' @@ -16,6 +18,7 @@ module Ml belongs_to :experiment, class_name: 'Ml::Experiment' belongs_to :user belongs_to :package, class_name: 'Packages::Package' + belongs_to :project has_many :metrics, class_name: 'Ml::CandidateMetric' has_many :params, class_name: 'Ml::CandidateParam' has_many :metadata, class_name: 'Ml::CandidateMetadata' @@ -23,6 +26,10 @@ module Ml attribute :eid, default: -> { SecureRandom.uuid } + has_internal_id :internal_id, + scope: :project, + init: AtomicInternalId.project_init(self, :internal_id) + scope :including_relationships, -> { includes(:latest_metrics, :params, :user, :package) } scope :by_name, ->(name) { where("ml_candidates.name LIKE ?", "%#{sanitize_sql_like(name)}%") } # rubocop:disable GitlabSecurity/SqlInjection scope :order_by_metric, ->(metric, direction) do @@ -49,8 +56,6 @@ module Ml ) end - delegate :project_id, :project, to: :experiment - alias_attribute :artifact, :package # Remove alias after https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115401 diff --git a/app/serializers/admin/abuse_report_entity.rb b/app/serializers/admin/abuse_report_entity.rb index 106467d4018..54916d02ecb 100644 --- a/app/serializers/admin/abuse_report_entity.rb +++ b/app/serializers/admin/abuse_report_entity.rb @@ -3,12 +3,14 @@ module Admin class AbuseReportEntity < Grape::Entity include RequestAwareEntity + include MarkupHelper expose :category + expose :created_at expose :updated_at expose :reported_user do |report| - UserEntity.represent(report.user, only: [:name]) + UserEntity.represent(report.user, only: [:name, :created_at]) end expose :reporter do |report| @@ -38,5 +40,9 @@ module Admin expose :remove_user_and_report_path do |report| admin_abuse_report_path(report, remove_user: true) end + + expose :message do |report| + markdown_field(report, :message) + end end end diff --git a/app/services/ci/unlock_artifacts_service.rb b/app/services/ci/unlock_artifacts_service.rb index 7d944132998..237f1997edb 100644 --- a/app/services/ci/unlock_artifacts_service.rb +++ b/app/services/ci/unlock_artifacts_service.rb @@ -4,11 +4,6 @@ module Ci class UnlockArtifactsService < ::BaseService BATCH_SIZE = 100 - # This service performs either one of the following, - # depending on whether `before_pipeline` is given. - # 1. Without `before_pipeline`, it unlocks all the pipelines belonging to the given `ci_ref` - # 2. With `before_pipeline`, it unlocks all the pipelines in the `ci_ref` that was created - # before the given `before_pipeline`, with the exception of the last successful pipeline. def execute(ci_ref, before_pipeline = nil) results = { unlocked_pipelines: 0, @@ -56,15 +51,15 @@ module Ci def unlock_pipelines_query(ci_ref, before_pipeline) ci_pipelines = ::Ci::Pipeline.arel_table - pipelines_to_unlock = ci_ref.pipelines.artifacts_locked - pipelines_to_unlock = exclude_last_successful_pipeline(pipelines_to_unlock, ci_ref, before_pipeline) - pipelines_to_unlock = pipelines_to_unlock.select(:id).limit(BATCH_SIZE).lock('FOR UPDATE SKIP LOCKED') + pipelines_scope = ci_ref.pipelines.artifacts_locked + pipelines_scope = pipelines_scope.before_pipeline(before_pipeline) if before_pipeline + pipelines_scope = pipelines_scope.select(:id).limit(BATCH_SIZE).lock('FOR UPDATE SKIP LOCKED') returning = Arel::Nodes::Grouping.new(ci_pipelines[:id]) Arel::UpdateManager.new .table(ci_pipelines) - .where(ci_pipelines[:id].in(Arel.sql(pipelines_to_unlock.to_sql))) + .where(ci_pipelines[:id].in(Arel.sql(pipelines_scope.to_sql))) .set([[ci_pipelines[:locked], ::Ci::Pipeline.lockeds[:unlocked]]]) .to_sql + " RETURNING #{returning.to_sql}" end @@ -72,22 +67,6 @@ module Ci private - # rubocop:disable CodeReuse/ActiveRecord - def exclude_last_successful_pipeline(pipelines_to_unlock, ci_ref, before_pipeline) - return pipelines_to_unlock if before_pipeline.nil? - - pipelines_to_unlock = pipelines_to_unlock.before_pipeline(before_pipeline) - - last_successful_pipeline = ci_ref.last_successful_pipeline - - if last_successful_pipeline.present? - pipelines_to_unlock = pipelines_to_unlock.outside_pipeline_family(last_successful_pipeline) - end - - pipelines_to_unlock - end - # rubocop:enable CodeReuse/ActiveRecord - def unlock_job_artifacts(pipelines) return if pipelines.empty? diff --git a/app/services/ml/experiment_tracking/candidate_repository.rb b/app/services/ml/experiment_tracking/candidate_repository.rb index f1fd93d7816..818cac7efbe 100644 --- a/app/services/ml/experiment_tracking/candidate_repository.rb +++ b/app/services/ml/experiment_tracking/candidate_repository.rb @@ -18,6 +18,7 @@ module Ml candidate = experiment.candidates.create!( user: user, name: candidate_name(name, tags), + project: project, start_time: start_time || 0 ) diff --git a/app/views/admin/applications/_form.html.haml b/app/views/admin/applications/_form.html.haml index 83347034cc5..da5a253041a 100644 --- a/app/views/admin/applications/_form.html.haml +++ b/app/views/admin/applications/_form.html.haml @@ -2,39 +2,34 @@ = form_errors(application) = content_tag :div, class: 'form-group row' do - .col-sm-2.col-form-label + .col-12 = f.label :name - .col-sm-10 = f.text_field :name, class: 'form-control gl-form-input' = doorkeeper_errors_for application, :name = content_tag :div, class: 'form-group row' do - .col-sm-2.col-form-label + .col-12 = f.label :redirect_uri - .col-sm-10 = f.text_area :redirect_uri, class: 'form-control gl-form-input' = doorkeeper_errors_for application, :redirect_uri %span.form-text.text-muted Use one line per URI = content_tag :div, class: 'form-group row' do - .col-sm-2.col-form-label.pt-0 + .col-12 = f.label :trusted - .col-sm-10 = f.gitlab_ui_checkbox_component :trusted, _('Trusted applications are automatically authorized on GitLab OAuth flow. It\'s highly recommended for the security of users that trusted applications have the confidential setting set to true.') = content_tag :div, class: 'form-group row' do - .col-sm-2.col-form-label.pt-0 + .col-12 = f.label :confidential - .col-sm-10 = f.gitlab_ui_checkbox_component :confidential, _('The application will be used where the client secret can be kept confidential. Native mobile apps and Single Page Apps are considered non-confidential.') .form-group.row - .col-sm-2.col-form-label.pt-0 + .col-12 = f.label :scopes - .col-sm-10 = render 'shared/tokens/scopes_form', prefix: 'doorkeeper_application', token: application, scopes: @scopes, f: f - .form-actions + .gl-mt-5 = f.submit _('Save application'), pajamas_button: true = link_to _('Cancel'), admin_applications_path, class: "gl-button btn btn-default btn-cancel" diff --git a/app/workers/all_queues.yml b/app/workers/all_queues.yml index 8e328e8babb..6f86c3b939e 100644 --- a/app/workers/all_queues.yml +++ b/app/workers/all_queues.yml @@ -1866,15 +1866,6 @@ :weight: 1 :idempotent: true :tags: [] -- :name: pipeline_background:ci_unlock_ref_artifacts_on_pipeline_stop - :worker_name: Ci::UnlockRefArtifactsOnPipelineStopWorker - :feature_category: :continuous_integration - :has_external_dependencies: false - :urgency: :low - :resource_boundary: :unknown - :weight: 1 - :idempotent: true - :tags: [] - :name: pipeline_creation:ci_external_pull_requests_create_pipeline :worker_name: Ci::ExternalPullRequests::CreatePipelineWorker :feature_category: :continuous_integration diff --git a/app/workers/ci/pipeline_success_unlock_artifacts_worker.rb b/app/workers/ci/pipeline_success_unlock_artifacts_worker.rb index a329ca0f577..2a1f492cacb 100644 --- a/app/workers/ci/pipeline_success_unlock_artifacts_worker.rb +++ b/app/workers/ci/pipeline_success_unlock_artifacts_worker.rb @@ -1,11 +1,6 @@ # frozen_string_literal: true module Ci - # TODO: Clean up this worker in a subsequent release. - # The process to unlock job artifacts have been moved to - # be triggered by the pipeline state transitions and - # to use UnlockRefArtifactsOnPipelineStopWorker. - # https://gitlab.com/gitlab-org/gitlab/-/issues/397491 class PipelineSuccessUnlockArtifactsWorker include ApplicationWorker diff --git a/app/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker.rb b/app/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker.rb deleted file mode 100644 index 3299cab0f6f..00000000000 --- a/app/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker.rb +++ /dev/null @@ -1,33 +0,0 @@ -# frozen_string_literal: true - -module Ci - # This worker is triggered when the pipeline state - # changes into one of the stopped statuses - # `Ci::Pipeline.stopped_statuses`. - # It unlocks the previous pipelines on the same ref - # as the pipeline that has just completed - # using `Ci::UnlockArtifactsService`. - class UnlockRefArtifactsOnPipelineStopWorker - include ApplicationWorker - - data_consistency :always - - include PipelineBackgroundQueue - - idempotent! - - def perform(pipeline_id) - pipeline = ::Ci::Pipeline.find_by_id(pipeline_id) - - return if pipeline.nil? - return if pipeline.ci_ref.nil? - - results = ::Ci::UnlockArtifactsService - .new(pipeline.project, pipeline.user) - .execute(pipeline.ci_ref, pipeline) - - log_extra_metadata_on_done(:unlocked_pipelines, results[:unlocked_pipelines]) - log_extra_metadata_on_done(:unlocked_job_artifacts, results[:unlocked_job_artifacts]) - end - end -end diff --git a/data/deprecations/14-5-certificate-based-integration-with-kubernetes-saas.yml b/data/deprecations/14-5-certificate-based-integration-with-kubernetes-saas.yml index 216568b6d19..6f6cfd70fed 100644 --- a/data/deprecations/14-5-certificate-based-integration-with-kubernetes-saas.yml +++ b/data/deprecations/14-5-certificate-based-integration-with-kubernetes-saas.yml @@ -8,6 +8,9 @@ For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) + Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. + For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). + For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). GitLab self-managed customers can still use the feature [with a feature flag](https://docs.gitlab.com/ee/update/deprecations.html#self-managed-certificate-based-integration-with-kubernetes). diff --git a/data/deprecations/14-5-certificate-based-integration-with-kubernetes.yml b/data/deprecations/14-5-certificate-based-integration-with-kubernetes.yml index 85b006e6768..a94d2fc7f7d 100644 --- a/data/deprecations/14-5-certificate-based-integration-with-kubernetes.yml +++ b/data/deprecations/14-5-certificate-based-integration-with-kubernetes.yml @@ -12,6 +12,9 @@ For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) + Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. + For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). + For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). stage: Configure tiers: [Core, Premium, Ultimate] diff --git a/db/migrate/20230321162810_add_project_id_to_ml_candidates.rb b/db/migrate/20230321162810_add_project_id_to_ml_candidates.rb new file mode 100644 index 00000000000..a8121f197d9 --- /dev/null +++ b/db/migrate/20230321162810_add_project_id_to_ml_candidates.rb @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +class AddProjectIdToMlCandidates < Gitlab::Database::Migration[2.1] + enable_lock_retries! + + def change + add_column :ml_candidates, :project_id, :bigint, null: true + end +end diff --git a/db/migrate/20230321162902_add_index_on_project_id_on_ml_candidates.rb b/db/migrate/20230321162902_add_index_on_project_id_on_ml_candidates.rb new file mode 100644 index 00000000000..e6c08468c0c --- /dev/null +++ b/db/migrate/20230321162902_add_index_on_project_id_on_ml_candidates.rb @@ -0,0 +1,15 @@ +# frozen_string_literal: true + +class AddIndexOnProjectIdOnMlCandidates < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + INDEX_NAME = 'index_ml_candidates_on_project_id' + + def up + add_concurrent_index :ml_candidates, :project_id, name: INDEX_NAME + end + + def down + remove_concurrent_index_by_name :ml_candidates, INDEX_NAME + end +end diff --git a/db/migrate/20230321163051_add_project_id_foreign_key_to_ml_candidates.rb b/db/migrate/20230321163051_add_project_id_foreign_key_to_ml_candidates.rb new file mode 100644 index 00000000000..3e43a160306 --- /dev/null +++ b/db/migrate/20230321163051_add_project_id_foreign_key_to_ml_candidates.rb @@ -0,0 +1,15 @@ +# frozen_string_literal: true + +class AddProjectIdForeignKeyToMlCandidates < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + def up + add_concurrent_foreign_key :ml_candidates, :projects, column: :project_id, on_delete: :cascade + end + + def down + with_lock_retries do + remove_foreign_key_if_exists :ml_candidates, column: :project_id + end + end +end diff --git a/db/migrate/20230321170734_add_internal_id_to_ml_candidates.rb b/db/migrate/20230321170734_add_internal_id_to_ml_candidates.rb new file mode 100644 index 00000000000..f6ced91c0a8 --- /dev/null +++ b/db/migrate/20230321170734_add_internal_id_to_ml_candidates.rb @@ -0,0 +1,7 @@ +# frozen_string_literal: true + +class AddInternalIdToMlCandidates < Gitlab::Database::Migration[2.1] + def change + add_column :ml_candidates, :internal_id, :bigint, null: true + end +end diff --git a/db/migrate/20230321170803_add_index_on_project_id_on_internal_id_to_ml_candidates.rb b/db/migrate/20230321170803_add_index_on_project_id_on_internal_id_to_ml_candidates.rb new file mode 100644 index 00000000000..4c295972106 --- /dev/null +++ b/db/migrate/20230321170803_add_index_on_project_id_on_internal_id_to_ml_candidates.rb @@ -0,0 +1,15 @@ +# frozen_string_literal: true + +class AddIndexOnProjectIdOnInternalIdToMlCandidates < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + INDEX_NAME = 'index_ml_candidates_on_project_id_on_internal_id' + + def up + add_concurrent_index :ml_candidates, [:project_id, :internal_id], name: INDEX_NAME + end + + def down + remove_concurrent_index_by_name :ml_candidates, INDEX_NAME + end +end diff --git a/db/schema_migrations/20230321162810 b/db/schema_migrations/20230321162810 new file mode 100644 index 00000000000..ef81f6f1549 --- /dev/null +++ b/db/schema_migrations/20230321162810 @@ -0,0 +1 @@ +f393893085e2a7faf43668589ce707dc27c61f8ea0dc8a3632503a39de673134
\ No newline at end of file diff --git a/db/schema_migrations/20230321162902 b/db/schema_migrations/20230321162902 new file mode 100644 index 00000000000..54e447494d3 --- /dev/null +++ b/db/schema_migrations/20230321162902 @@ -0,0 +1 @@ +2d00140af48ff5137f2c8df0b03fdebbc08abd0d448b967fdc1fb8781ab0841f
\ No newline at end of file diff --git a/db/schema_migrations/20230321163051 b/db/schema_migrations/20230321163051 new file mode 100644 index 00000000000..a4d26398090 --- /dev/null +++ b/db/schema_migrations/20230321163051 @@ -0,0 +1 @@ +e172b6f87e3f06e3c2a7f64b0d7d9eae797802a4dd77b86a989ab4eb6ec5e626
\ No newline at end of file diff --git a/db/schema_migrations/20230321170734 b/db/schema_migrations/20230321170734 new file mode 100644 index 00000000000..b6653465894 --- /dev/null +++ b/db/schema_migrations/20230321170734 @@ -0,0 +1 @@ +e60dc9b8f28fdbbc84ed808edc98fb8d640ec5a53b21363a59f375a0a3fe5bfd
\ No newline at end of file diff --git a/db/schema_migrations/20230321170803 b/db/schema_migrations/20230321170803 new file mode 100644 index 00000000000..85d557d7681 --- /dev/null +++ b/db/schema_migrations/20230321170803 @@ -0,0 +1 @@ +c08a4f0873dfbc3ce2d6c1cc9224b2427ec41de482da85768f7cf08409ec8a54
\ No newline at end of file diff --git a/db/structure.sql b/db/structure.sql index c151fa8cb44..229ce106caa 100644 --- a/db/structure.sql +++ b/db/structure.sql @@ -18420,6 +18420,8 @@ CREATE TABLE ml_candidates ( name text, package_id bigint, eid uuid, + project_id bigint, + internal_id bigint, CONSTRAINT check_25e6c65051 CHECK ((char_length(name) <= 255)), CONSTRAINT check_cd160587d4 CHECK ((eid IS NOT NULL)) ); @@ -31085,6 +31087,10 @@ CREATE UNIQUE INDEX index_ml_candidates_on_experiment_id_and_eid ON ml_candidate CREATE INDEX index_ml_candidates_on_package_id ON ml_candidates USING btree (package_id); +CREATE INDEX index_ml_candidates_on_project_id ON ml_candidates USING btree (project_id); + +CREATE INDEX index_ml_candidates_on_project_id_on_internal_id ON ml_candidates USING btree (project_id, internal_id); + CREATE INDEX index_ml_candidates_on_user_id ON ml_candidates USING btree (user_id); CREATE UNIQUE INDEX index_ml_experiment_metadata_on_experiment_id_and_name ON ml_experiment_metadata USING btree (experiment_id, name); @@ -34348,6 +34354,9 @@ ALTER TABLE ONLY merge_requests_compliance_violations ALTER TABLE ONLY coverage_fuzzing_corpuses ADD CONSTRAINT fk_29f6f15f82 FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE; +ALTER TABLE ONLY ml_candidates + ADD CONSTRAINT fk_2a0421d824 FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE; + ALTER TABLE ONLY agent_group_authorizations ADD CONSTRAINT fk_2c9f941965 FOREIGN KEY (group_id) REFERENCES namespaces(id) ON DELETE CASCADE; diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 6a9dd34e703..e4768bdb97f 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -136,9 +136,9 @@ This [documentation](merge_request_workflow.md) outlines the current merge reque ## Getting an Enterprise Edition License If you need a license for contributing to an EE-feature, see -[relevant information](https://about.gitlab.com/handbook/marketing/community-relations/code-contributor-program/operations/#contributing-to-the-gitlab-enterprise-edition-ee). +[relevant information](https://about.gitlab.com/handbook/marketing/community-relations/contributor-success/community-contributors-workflows.html#contributing-to-the-gitlab-enterprise-edition-ee). ## Finding help - [Get help](https://about.gitlab.com/get-help/). -- Join the community-run [Discord server](https://discord.com/invite/gitlab) and find other contributors in the `#contribute` channel. +- Join the community-run [Discord server](https://discord.gg/gitlab) and find other contributors in the `#contribute` channel. diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index 4f644dd018e..f1342d24fb4 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -1290,6 +1290,7 @@ This sensitive data must be handled carefully to avoid leaks which could lead to - Credentials must be encrypted while at rest (database or file) with `attr_encrypted`. See [issue #26243](https://gitlab.com/gitlab-org/gitlab/-/issues/26243) before using `attr_encrypted`. - Store the encryption keys separately from the encrypted credentials with proper access control. For instance, store the keys in a vault, KMS, or file. Here is an [example](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/user.rb#L70-74) use of `attr_encrypted` for encryption with keys stored in separate access controlled file. - When the intention is to only compare secrets, store only the salted hash of the secret instead of the encrypted value. +- Salted hashes should be used to store any sensitive value where the plaintext value itself does not need to be retrieved. - Never commit credentials to repositories. - The [Gitleaks Git hook](https://gitlab.com/gitlab-com/gl-security/security-research/gitleaks-endpoint-installer) is recommended for preventing credentials from being committed. - Never log credentials under any circumstance. Issue [#353857](https://gitlab.com/gitlab-org/gitlab/-/issues/353857) is an example of credential leaks through log file. @@ -1306,6 +1307,32 @@ This sensitive data must be handled carefully to avoid leaks which could lead to In the event of credential leak through an MR, issue, or any other medium, [reach out to SIRT team](https://about.gitlab.com/handbook/security/security-operations/sirt/#-engaging-sirt). +### Examples + +Encrypting a token with `attr_encrypted` so that the plaintext can be retrieved and used later: + +```ruby +module AlertManagement + class HttpIntegration < ApplicationRecord + + attr_encrypted :token, + mode: :per_attribute_iv, + key: Settings.attr_encrypted_db_key_base_32, + algorithm: 'aes-256-gcm' +``` + +Hashing a sensitive value with `CryptoHelper` so that it can be compared in future, but the plaintext is irretrievable: + +```ruby +class WebHookLog < ApplicationRecord + before_save :set_url_hash, if: -> { interpolated_url.present? } + + def set_url_hash + self.url_hash = Gitlab::CryptoHelper.sha256(interpolated_url) + end +end +``` + ## Serialization Serialization of active record models can leak sensitive attributes if they are not protected. diff --git a/doc/tutorials/index.md b/doc/tutorials/index.md index 3df89a23379..d81b667e700 100644 --- a/doc/tutorials/index.md +++ b/doc/tutorials/index.md @@ -87,6 +87,7 @@ GitLab can check your application for security vulnerabilities. |-------|-------------|--------------------| | [Set up dependency scanning](https://about.gitlab.com/blog/2021/01/14/try-dependency-scanning/) | Try out dependency scanning, which checks for known vulnerabilities in dependencies. | **{star}** | | [Get started with GitLab application security](../user/application_security/get-started-security.md) | Follow recommended steps to set up security tools. | | +| [GitLab Security Essentials](https://levelup.gitlab.com/courses/security-essentials) | Learn about the essential security capabilities of GitLab in this self-paced course. | | ## Work with a self-managed instance diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md index cbefaf9ba77..ce5eae9d48d 100644 --- a/doc/update/deprecations.md +++ b/doc/update/deprecations.md @@ -3415,6 +3415,9 @@ The certificate-based integration with Kubernetes will be [deprecated and remove For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) +Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. +For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). + For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). GitLab self-managed customers can still use the feature [with a feature flag](https://docs.gitlab.com/ee/update/deprecations.html#self-managed-certificate-based-integration-with-kubernetes). @@ -3440,6 +3443,9 @@ In GitLab 17.0 we will remove both the feature and its related code. Until the f For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. [How do I migrate?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) +Although an explicit removal date is set, we don't plan to remove this feature until the new solution has feature parity. +For more information about the blockers to removal, see [this issue](https://gitlab.com/gitlab-org/configure/general/-/issues/199). + For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). </div> diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index f7e6ea610fe..1af95b06aa8 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -210,17 +210,6 @@ To migrate users to a new email domain, tell users to: If the **NameID** is configured with the email address, [change the **NameID** for users](#change-nameid-for-one-or-more-users). -### User attributes - -To create users with the correct information for improved [user access and management](#user-access-and-management), -the user's details must be passed to GitLab as attributes in the SAML assertion. At a minimum, the user's email address -must be specified as an attribute named `email` or `mail`. - -You can configure the following attributes with GitLab.com Group SAML: - -- `username` or `nickname`. We recommend you configure only one of these. -- The [attributes available](../../../integration/saml.md#configure-assertions) to self-managed GitLab instances. - ## Configure GitLab After you set up your identity provider to work with GitLab, you must configure GitLab to use it for authentication: @@ -337,6 +326,16 @@ When a user tries to sign in with Group SSO, GitLab attempts to find or create a - Create a new account with another email address. - Sign-in to their existing account to link the SAML identity. +### User attributes + +You can pass user information to GitLab as attributes in the SAML assertion. + +- The user's email address can be an **email** or **mail** attribute. +- The username can be either a **username** or **nickname** attribute. You should specify only + one of these. + +For more information, see the [attributes available for self-managed GitLab instances](../../../integration/saml.md#configure-assertions). + ### Linking SAML to your existing GitLab.com account > **Remember me** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/121569) in GitLab 15.7. diff --git a/lib/gitlab/ci/config/entry/job.rb b/lib/gitlab/ci/config/entry/job.rb index 2390ba05916..d31d1b366c3 100644 --- a/lib/gitlab/ci/config/entry/job.rb +++ b/lib/gitlab/ci/config/entry/job.rb @@ -14,7 +14,7 @@ module Gitlab ALLOWED_KEYS = %i[tags script image services start_in artifacts cache dependencies before_script after_script hooks environment coverage retry parallel interruptible timeout - release id_tokens].freeze + release id_tokens publish].freeze validations do validates :config, allowed_keys: Gitlab::Ci::Config::Entry::Job.allowed_keys + PROCESSABLE_ALLOWED_KEYS @@ -45,6 +45,8 @@ module Gitlab errors.add(:dependencies, "the #{missing_needs.join(", ")} should be part of needs") if missing_needs.any? end end + + validates :publish, absence: { message: "can only be used within a `pages` job" }, unless: -> { pages_job? } end entry :before_script, Entry::Commands, @@ -125,10 +127,14 @@ module Gitlab inherit: false, metadata: { composable_class: ::Gitlab::Ci::Config::Entry::IdToken } + entry :publish, Entry::Publish, + description: 'Path to be published with Pages', + inherit: false + attributes :script, :tags, :when, :dependencies, :needs, :retry, :parallel, :start_in, :interruptible, :timeout, - :release, :allow_failure + :release, :allow_failure, :publish def self.matching?(name, config) !name.to_s.start_with?('.') && @@ -169,7 +175,8 @@ module Gitlab allow_failure_criteria: allow_failure_criteria, needs: needs_defined? ? needs_value : nil, scheduling_type: needs_defined? ? :dag : :stage, - id_tokens: id_tokens_value + id_tokens: id_tokens_value, + publish: publish ).compact end @@ -177,6 +184,10 @@ module Gitlab allow_failure_defined? ? static_allow_failure : manual_action? end + def pages_job? + name == :pages + end + def self.allowed_keys ALLOWED_KEYS end diff --git a/lib/gitlab/ci/config/entry/publish.rb b/lib/gitlab/ci/config/entry/publish.rb new file mode 100644 index 00000000000..52a2487009e --- /dev/null +++ b/lib/gitlab/ci/config/entry/publish.rb @@ -0,0 +1,24 @@ +# frozen_string_literal: true + +module Gitlab + module Ci + class Config + module Entry + ## + # Entry that represents the path to be published with Pages. + # + class Publish < ::Gitlab::Config::Entry::Node + include ::Gitlab::Config::Entry::Validatable + + validations do + validates :config, type: String + end + + def self.default + 'public' + end + end + end + end + end +end diff --git a/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml b/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml index 7f8e2150c71..0cb3f85ba40 100644 --- a/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml +++ b/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml @@ -40,7 +40,8 @@ container_scanning: reports: container_scanning: gl-container-scanning-report.json dependency_scanning: gl-dependency-scanning-report.json - paths: [gl-container-scanning-report.json, gl-dependency-scanning-report.json] + cyclonedx: "**/gl-sbom-*.cdx.json" + paths: [gl-container-scanning-report.json, gl-dependency-scanning-report.json, "**/gl-sbom-*.cdx.json"] dependencies: [] script: - gtcs scan diff --git a/lib/gitlab/ci/templates/Jobs/Container-Scanning.latest.gitlab-ci.yml b/lib/gitlab/ci/templates/Jobs/Container-Scanning.latest.gitlab-ci.yml index 15688da71ab..bed0900ae4f 100644 --- a/lib/gitlab/ci/templates/Jobs/Container-Scanning.latest.gitlab-ci.yml +++ b/lib/gitlab/ci/templates/Jobs/Container-Scanning.latest.gitlab-ci.yml @@ -40,7 +40,8 @@ container_scanning: reports: container_scanning: gl-container-scanning-report.json dependency_scanning: gl-dependency-scanning-report.json - paths: [gl-container-scanning-report.json, gl-dependency-scanning-report.json] + cyclonedx: "**/gl-sbom-*.cdx.json" + paths: [gl-container-scanning-report.json, gl-dependency-scanning-report.json, "**/gl-sbom-*.cdx.json"] dependencies: [] script: - gtcs scan diff --git a/lib/gitlab/ci/templates/Python.gitlab-ci.yml b/lib/gitlab/ci/templates/Python.gitlab-ci.yml index febbb36d834..5797bcbaca9 100644 --- a/lib/gitlab/ci/templates/Python.gitlab-ci.yml +++ b/lib/gitlab/ci/templates/Python.gitlab-ci.yml @@ -32,7 +32,7 @@ test: script: - python setup.py test - pip install tox flake8 # you can also use tox - - tox -e py36,flake8 + - tox -e py,flake8 run: script: diff --git a/lib/gitlab/ci/yaml_processor/result.rb b/lib/gitlab/ci/yaml_processor/result.rb index d867439b10b..6207b595fc6 100644 --- a/lib/gitlab/ci/yaml_processor/result.rb +++ b/lib/gitlab/ci/yaml_processor/result.rb @@ -123,7 +123,8 @@ module Gitlab start_in: job[:start_in], trigger: job[:trigger], bridge_needs: job.dig(:needs, :bridge)&.first, - release: job[:release] + release: job[:release], + publish: job[:publish] }.compact }.compact end diff --git a/lib/sidebars/projects/menus/issues_menu.rb b/lib/sidebars/projects/menus/issues_menu.rb index 6904dc129b7..a7f9ddde247 100644 --- a/lib/sidebars/projects/menus/issues_menu.rb +++ b/lib/sidebars/projects/menus/issues_menu.rb @@ -85,6 +85,10 @@ module Sidebars can?(context.current_user, :read_issue, context.project) end + def multi_issue_boards? + context.project.multiple_issue_boards_available? + end + def list_menu_item ::Sidebars::MenuItem.new( title: _('List'), @@ -97,7 +101,11 @@ module Sidebars end def boards_menu_item - title = context.project.multiple_issue_boards_available? ? s_('IssueBoards|Boards') : s_('IssueBoards|Board') + title = if context.is_super_sidebar + multi_issue_boards? ? s_('Issue boards') : s_('Issue board') + else + multi_issue_boards? ? s_('IssueBoards|Boards') : s_('IssueBoards|Board') + end ::Sidebars::MenuItem.new( title: title, @@ -122,8 +130,7 @@ module Sidebars ::Sidebars::MenuItem.new( title: _('Milestones'), link: project_milestones_path(context.project), - super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::PlanMenu, - super_sidebar_before: :service_desk, + super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::ManageMenu, active_routes: { controller: :milestones }, item_id: :milestones ) diff --git a/lib/sidebars/projects/menus/project_information_menu.rb b/lib/sidebars/projects/menus/project_information_menu.rb index 020de2ff65f..6ab7e00dad3 100644 --- a/lib/sidebars/projects/menus/project_information_menu.rb +++ b/lib/sidebars/projects/menus/project_information_menu.rb @@ -44,7 +44,7 @@ module Sidebars ::Sidebars::MenuItem.new( title: _('Activity'), link: activity_project_path(context.project), - super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::PlanMenu, + super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::ManageMenu, active_routes: { path: 'projects#activity' }, item_id: :activity, container_html_options: { class: 'shortcuts-project-activity' } @@ -59,8 +59,7 @@ module Sidebars ::Sidebars::MenuItem.new( title: _('Labels'), link: project_labels_path(context.project), - super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::PlanMenu, - super_sidebar_before: :activity, + super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::ManageMenu, active_routes: { controller: :labels }, item_id: :labels ) @@ -74,8 +73,7 @@ module Sidebars ::Sidebars::MenuItem.new( title: _('Members'), link: project_project_members_path(context.project), - super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::PlanMenu, - super_sidebar_before: :labels, + super_sidebar_parent: ::Sidebars::Projects::SuperSidebarMenus::ManageMenu, active_routes: { controller: :project_members }, item_id: :members, container_html_options: { diff --git a/lib/sidebars/projects/super_sidebar_menus/manage_menu.rb b/lib/sidebars/projects/super_sidebar_menus/manage_menu.rb index 72743910411..faf9708604d 100644 --- a/lib/sidebars/projects/super_sidebar_menus/manage_menu.rb +++ b/lib/sidebars/projects/super_sidebar_menus/manage_menu.rb @@ -13,6 +13,17 @@ module Sidebars def sprite_icon 'users' end + + override :configure_menu_items + def configure_menu_items + [ + :activity, + :members, + :labels, + :milestones, + :iterations + ].each { |id| add_item(::Sidebars::NilMenuItem.new(item_id: id)) } + end end end end diff --git a/lib/sidebars/projects/super_sidebar_menus/plan_menu.rb b/lib/sidebars/projects/super_sidebar_menus/plan_menu.rb index 787d096cabf..38b30949bfa 100644 --- a/lib/sidebars/projects/super_sidebar_menus/plan_menu.rb +++ b/lib/sidebars/projects/super_sidebar_menus/plan_menu.rb @@ -13,6 +13,16 @@ module Sidebars def sprite_icon 'planning' end + + override :configure_menu_items + def configure_menu_items + [ + :boards, + :project_wiki, + :service_desk, + :requirements + ].each { |id| add_item(::Sidebars::NilMenuItem.new(item_id: id)) } + end end end end diff --git a/locale/gitlab.pot b/locale/gitlab.pot index 2d07aa3f1ec..79623c31b34 100644 --- a/locale/gitlab.pot +++ b/locale/gitlab.pot @@ -47380,6 +47380,9 @@ msgstr "" msgid "User is not allowed to resolve thread" msgstr "" +msgid "User joined %{timeAgo}" +msgstr "" + msgid "User key" msgstr "" diff --git a/package.json b/package.json index 8d0957be2af..dc95e9b3294 100644 --- a/package.json +++ b/package.json @@ -56,7 +56,7 @@ "@gitlab/favicon-overlay": "2.0.0", "@gitlab/fonts": "^1.2.0", "@gitlab/svgs": "3.31.0", - "@gitlab/ui": "58.5.0", + "@gitlab/ui": "58.6.0", "@gitlab/visual-review-tools": "1.7.3", "@gitlab/web-ide": "0.0.1-dev-20230323132525", "@mattiasbuelens/web-streams-adapter": "^0.1.0", diff --git a/spec/db/schema_spec.rb b/spec/db/schema_spec.rb index 45a914439a9..ca0d54abca7 100644 --- a/spec/db/schema_spec.rb +++ b/spec/db/schema_spec.rb @@ -121,7 +121,9 @@ RSpec.describe 'Database schema', feature_category: :database do vulnerability_reads: %w[cluster_agent_id], # See: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87584 # Fixes performance issues with the deletion of web-hooks with many log entries - web_hook_logs: %w[web_hook_id] + web_hook_logs: %w[web_hook_id], + ml_candidates: %w[internal_id] + }.with_indifferent_access.freeze context 'for table' do diff --git a/spec/factories/ml/candidates.rb b/spec/factories/ml/candidates.rb index bcf1f25e19b..9d049987cfd 100644 --- a/spec/factories/ml/candidates.rb +++ b/spec/factories/ml/candidates.rb @@ -1,9 +1,11 @@ # frozen_string_literal: true FactoryBot.define do factory :ml_candidates, class: '::Ml::Candidate' do - association :experiment, factory: :ml_experiments + association :project, factory: :project association :user + experiment { association :ml_experiments, project_id: project.id } + trait :with_metrics_and_params do after(:create) do |candidate| candidate.metrics = FactoryBot.create_list(:ml_candidate_metrics, 2, candidate: candidate ) diff --git a/spec/frontend/admin/abuse_reports/components/abuse_report_details_spec.js b/spec/frontend/admin/abuse_reports/components/abuse_report_details_spec.js new file mode 100644 index 00000000000..b89bbac0196 --- /dev/null +++ b/spec/frontend/admin/abuse_reports/components/abuse_report_details_spec.js @@ -0,0 +1,53 @@ +import { GlButton, GlCollapse } from '@gitlab/ui'; +import { nextTick } from 'vue'; +import { shallowMount } from '@vue/test-utils'; +import AbuseReportDetails from '~/admin/abuse_reports/components/abuse_report_details.vue'; +import { getTimeago } from '~/lib/utils/datetime_utility'; +import { mockAbuseReports } from '../mock_data'; + +describe('AbuseReportDetails', () => { + let wrapper; + const report = mockAbuseReports[0]; + + const findToggleButton = () => wrapper.findComponent(GlButton); + const findCollapsible = () => wrapper.findComponent(GlCollapse); + + const createComponent = () => { + wrapper = shallowMount(AbuseReportDetails, { + propsData: { + report, + }, + }); + }; + + describe('default', () => { + beforeEach(() => { + createComponent(); + }); + + it('renders toggle button with the correct text', () => { + expect(findToggleButton().text()).toEqual('Show details'); + }); + + it('renders collapsed GlCollapse containing the report details', () => { + const collapsible = findCollapsible(); + expect(collapsible.attributes('visible')).toBeUndefined(); + + const userJoinedText = `User joined ${getTimeago().format(report.reportedUser.createdAt)}`; + expect(collapsible.text()).toMatch(userJoinedText); + expect(collapsible.text()).toMatch(report.message); + }); + }); + + describe('when toggled', () => { + it('expands GlCollapse and updates toggle text', async () => { + createComponent(); + + findToggleButton().vm.$emit('click'); + await nextTick(); + + expect(findToggleButton().text()).toEqual('Hide details'); + expect(findCollapsible().attributes('visible')).toBe('true'); + }); + }); +}); diff --git a/spec/frontend/admin/abuse_reports/components/abuse_report_row_spec.js b/spec/frontend/admin/abuse_reports/components/abuse_report_row_spec.js index 599d52227c1..9876ee70e5e 100644 --- a/spec/frontend/admin/abuse_reports/components/abuse_report_row_spec.js +++ b/spec/frontend/admin/abuse_reports/components/abuse_report_row_spec.js @@ -1,9 +1,12 @@ import { GlSprintf, GlLink } from '@gitlab/ui'; import { shallowMountExtended } from 'helpers/vue_test_utils_helper'; +import setWindowLocation from 'helpers/set_window_location_helper'; +import AbuseReportDetails from '~/admin/abuse_reports/components/abuse_report_details.vue'; import AbuseReportRow from '~/admin/abuse_reports/components/abuse_report_row.vue'; import AbuseReportActions from '~/admin/abuse_reports/components/abuse_report_actions.vue'; import ListItem from '~/vue_shared/components/registry/list_item.vue'; import { getTimeago } from '~/lib/utils/datetime_utility'; +import { SORT_UPDATED_AT } from '~/admin/abuse_reports/constants'; import { mockAbuseReports } from '../mock_data'; describe('AbuseReportRow', () => { @@ -14,7 +17,8 @@ describe('AbuseReportRow', () => { const findAbuseReportActions = () => wrapper.findComponent(AbuseReportActions); const findListItem = () => wrapper.findComponent(ListItem); const findTitle = () => wrapper.findByTestId('title'); - const findUpdatedAt = () => wrapper.findByTestId('updated-at'); + const findDisplayedDate = () => wrapper.findByTestId('abuse-report-date'); + const findAbuseReportDetails = () => wrapper.findComponent(AbuseReportDetails); const createComponent = () => { wrapper = shallowMountExtended(AbuseReportRow, { @@ -48,10 +52,29 @@ describe('AbuseReportRow', () => { expect(reporterLink.attributes('href')).toEqual(reporterPath); }); - it('displays correctly formatted updated at', () => { - expect(findUpdatedAt().text()).toMatchInterpolatedText( - `Updated ${getTimeago().format(mockAbuseReport.updatedAt)}`, - ); + describe('displayed date', () => { + it('displays correctly formatted created at', () => { + expect(findDisplayedDate().text()).toMatchInterpolatedText( + `Created ${getTimeago().format(mockAbuseReport.createdAt)}`, + ); + }); + + describe('when sorted by updated_at', () => { + it('displays correctly formatted updated at', () => { + setWindowLocation(`?sort=${SORT_UPDATED_AT.sortDirection.ascending}`); + + createComponent(); + + expect(findDisplayedDate().text()).toMatchInterpolatedText( + `Updated ${getTimeago().format(mockAbuseReport.updatedAt)}`, + ); + }); + }); + }); + + it('renders AbuseReportDetails', () => { + expect(findAbuseReportDetails().exists()).toBe(true); + expect(findAbuseReportDetails().props('report')).toEqual(mockAbuseReport); }); it('renders AbuseReportRowActions with the correct props', () => { diff --git a/spec/frontend/admin/abuse_reports/mock_data.js b/spec/frontend/admin/abuse_reports/mock_data.js index 92e50ad6c7e..90289757a74 100644 --- a/spec/frontend/admin/abuse_reports/mock_data.js +++ b/spec/frontend/admin/abuse_reports/mock_data.js @@ -1,26 +1,30 @@ export const mockAbuseReports = [ { category: 'spam', + createdAt: '2018-10-03T05:46:38.977Z', updatedAt: '2022-12-07T06:45:39.977Z', reporter: { name: 'Ms. Admin' }, - reportedUser: { name: 'Mr. Abuser' }, + reportedUser: { name: 'Mr. Abuser', createdAt: '2017-09-01T05:46:38.977Z' }, reportedUserPath: '/mr_abuser', reporterPath: '/admin', userBlocked: false, blockUserPath: '/block/user/mr_abuser/path', removeUserAndReportPath: '/remove/user/mr_abuser/and/report/path', removeReportPath: '/remove/report/path', + message: 'message 1', }, { category: 'phishing', + createdAt: '2018-10-03T05:46:38.977Z', updatedAt: '2022-12-07T06:45:39.977Z', reporter: { name: 'Ms. Reporter' }, - reportedUser: { name: 'Mr. Phisher' }, + reportedUser: { name: 'Mr. Phisher', createdAt: '2016-09-01T05:46:38.977Z' }, reportedUserPath: '/mr_phisher', reporterPath: '/admin', userBlocked: false, blockUserPath: '/block/user/mr_phisher/path', removeUserAndReportPath: '/remove/user/mr_phisher/and/report/path', removeReportPath: '/remove/report/path', + message: 'message 2', }, ]; diff --git a/spec/frontend/content_editor/components/wrappers/label_spec.js b/spec/frontend/content_editor/components/wrappers/reference_label_spec.js index fa32b746142..ac3b0730223 100644 --- a/spec/frontend/content_editor/components/wrappers/label_spec.js +++ b/spec/frontend/content_editor/components/wrappers/reference_label_spec.js @@ -1,12 +1,12 @@ import { GlLabel } from '@gitlab/ui'; import { shallowMountExtended } from 'helpers/vue_test_utils_helper'; -import LabelWrapper from '~/content_editor/components/wrappers/label.vue'; +import ReferenceLabelWrapper from '~/content_editor/components/wrappers/reference_label.vue'; -describe('content/components/wrappers/label', () => { +describe('content/components/wrappers/reference_label', () => { let wrapper; const createWrapper = async (node = {}) => { - wrapper = shallowMountExtended(LabelWrapper, { + wrapper = shallowMountExtended(ReferenceLabelWrapper, { propsData: { node }, }); }; diff --git a/spec/frontend/content_editor/components/wrappers/reference_spec.js b/spec/frontend/content_editor/components/wrappers/reference_spec.js new file mode 100644 index 00000000000..4f9f2e3f800 --- /dev/null +++ b/spec/frontend/content_editor/components/wrappers/reference_spec.js @@ -0,0 +1,28 @@ +import { shallowMountExtended } from 'helpers/vue_test_utils_helper'; +import ReferenceWrapper from '~/content_editor/components/wrappers/reference.vue'; + +describe('content/components/wrappers/reference', () => { + let wrapper; + + const createWrapper = async (node = {}) => { + wrapper = shallowMountExtended(ReferenceWrapper, { + propsData: { node }, + }); + }; + + it('renders a span for comamnds', () => { + createWrapper({ attrs: { referenceType: 'command', text: '/assign' } }); + + expect(wrapper.html()).toMatchInlineSnapshot( + `"<node-view-wrapper-stub as=\\"div\\" class=\\"gl-display-inline-block\\"><span>/assign</span></node-view-wrapper-stub>"`, + ); + }); + + it('renders an anchor for everything else', () => { + createWrapper({ attrs: { referenceType: 'issue', text: '#252522' } }); + + expect(wrapper.html()).toMatchInlineSnapshot( + `"<node-view-wrapper-stub as=\\"div\\" class=\\"gl-display-inline-block\\"><a href=\\"#\\">#252522</a></node-view-wrapper-stub>"`, + ); + }); +}); diff --git a/spec/helpers/projects/ml/experiments_helper_spec.rb b/spec/helpers/projects/ml/experiments_helper_spec.rb index 8ef81c49fa7..0354ce5d9c4 100644 --- a/spec/helpers/projects/ml/experiments_helper_spec.rb +++ b/spec/helpers/projects/ml/experiments_helper_spec.rb @@ -9,7 +9,7 @@ RSpec.describe Projects::Ml::ExperimentsHelper, feature_category: :mlops do let_it_be(:project) { create(:project, :private) } let_it_be(:experiment) { create(:ml_experiments, user_id: project.creator, project: project) } let_it_be(:candidate0) do - create(:ml_candidates, :with_artifact, experiment: experiment, user: project.creator).tap do |c| + create(:ml_candidates, :with_artifact, experiment: experiment, user: project.creator, project: project).tap do |c| c.params.build([{ name: 'param1', value: 'p1' }, { name: 'param2', value: 'p2' }]) c.metrics.create!( [{ name: 'metric1', value: 0.1 }, { name: 'metric2', value: 0.2 }, { name: 'metric3', value: 0.3 }] @@ -18,7 +18,8 @@ RSpec.describe Projects::Ml::ExperimentsHelper, feature_category: :mlops do end let_it_be(:candidate1) do - create(:ml_candidates, experiment: experiment, user: project.creator, name: 'candidate1').tap do |c| + create(:ml_candidates, experiment: experiment, user: project.creator, name: 'candidate1', + project: project).tap do |c| c.params.build([{ name: 'param2', value: 'p3' }, { name: 'param3', value: 'p4' }]) c.metrics.create!(name: 'metric3', value: 0.4) end diff --git a/spec/lib/gitlab/ci/config/entry/job_spec.rb b/spec/lib/gitlab/ci/config/entry/job_spec.rb index c8b4a8b8a0e..39a88fc7721 100644 --- a/spec/lib/gitlab/ci/config/entry/job_spec.rb +++ b/spec/lib/gitlab/ci/config/entry/job_spec.rb @@ -595,6 +595,39 @@ RSpec.describe Gitlab::Ci::Config::Entry::Job, feature_category: :pipeline_compo end end end + + context 'when job is not a pages job' do + let(:name) { :rspec } + + context 'if the config contains a publish entry' do + let(:entry) { described_class.new({ script: 'echo', publish: 'foo' }, name: name) } + + it 'is invalid' do + expect(entry).not_to be_valid + expect(entry.errors).to include /job publish can only be used within a `pages` job/ + end + end + end + + context 'when job is a pages job' do + let(:name) { :pages } + + context 'when it does not have a publish entry' do + let(:entry) { described_class.new({ script: 'echo' }, name: name) } + + it 'is valid' do + expect(entry).to be_valid + end + end + + context 'when it has a publish entry' do + let(:entry) { described_class.new({ script: 'echo', publish: 'foo' }, name: name) } + + it 'is valid' do + expect(entry).to be_valid + end + end + end end describe '#relevant?' do diff --git a/spec/lib/gitlab/ci/config/entry/publish_spec.rb b/spec/lib/gitlab/ci/config/entry/publish_spec.rb new file mode 100644 index 00000000000..53ad868a05e --- /dev/null +++ b/spec/lib/gitlab/ci/config/entry/publish_spec.rb @@ -0,0 +1,40 @@ +# frozen_string_literal: true + +require 'spec_helper' + +RSpec.describe Gitlab::Ci::Config::Entry::Publish, feature_category: :pages do + let(:publish) { described_class.new(config) } + + describe 'validations' do + context 'when publish config value is correct' do + let(:config) { 'dist/static' } + + describe '#config' do + it 'returns the publish directory' do + expect(publish.config).to eq config + end + end + + describe '#valid?' do + it 'is valid' do + expect(publish).to be_valid + end + end + end + + context 'when the value has a wrong type' do + let(:config) { { test: true } } + + it 'reports an error' do + expect(publish.errors) + .to include 'publish config should be a string' + end + end + end + + describe '.default' do + it 'returns the default value' do + expect(described_class.default).to eq 'public' + end + end +end diff --git a/spec/lib/sidebars/projects/super_sidebar_menus/manage_menu_spec.rb b/spec/lib/sidebars/projects/super_sidebar_menus/manage_menu_spec.rb index 8c5f7aaaf36..6a6d61496ea 100644 --- a/spec/lib/sidebars/projects/super_sidebar_menus/manage_menu_spec.rb +++ b/spec/lib/sidebars/projects/super_sidebar_menus/manage_menu_spec.rb @@ -5,8 +5,21 @@ require 'spec_helper' RSpec.describe Sidebars::Projects::SuperSidebarMenus::ManageMenu, feature_category: :navigation do subject { described_class.new({}) } + let(:items) { subject.instance_variable_get(:@items) } + it 'has title and sprite_icon' do expect(subject.title).to eq(s_("Navigation|Manage")) expect(subject.sprite_icon).to eq("users") end + + it 'defines list of NilMenuItem placeholders' do + expect(items.map(&:class).uniq).to eq([Sidebars::NilMenuItem]) + expect(items.map(&:item_id)).to eq([ + :activity, + :members, + :labels, + :milestones, + :iterations + ]) + end end diff --git a/spec/lib/sidebars/projects/super_sidebar_menus/plan_menu_spec.rb b/spec/lib/sidebars/projects/super_sidebar_menus/plan_menu_spec.rb index 5f6f6e4f6c2..9f3aa62a364 100644 --- a/spec/lib/sidebars/projects/super_sidebar_menus/plan_menu_spec.rb +++ b/spec/lib/sidebars/projects/super_sidebar_menus/plan_menu_spec.rb @@ -5,8 +5,20 @@ require 'spec_helper' RSpec.describe Sidebars::Projects::SuperSidebarMenus::PlanMenu, feature_category: :navigation do subject { described_class.new({}) } + let(:items) { subject.instance_variable_get(:@items) } + it 'has title and sprite_icon' do expect(subject.title).to eq(s_("Navigation|Plan")) expect(subject.sprite_icon).to eq("planning") end + + it 'defines list of NilMenuItem placeholders' do + expect(items.map(&:class).uniq).to eq([Sidebars::NilMenuItem]) + expect(items.map(&:item_id)).to eq([ + :boards, + :project_wiki, + :service_desk, + :requirements + ]) + end end diff --git a/spec/models/ci/pipeline_spec.rb b/spec/models/ci/pipeline_spec.rb index 45768ed28f0..263db8e58c7 100644 --- a/spec/models/ci/pipeline_spec.rb +++ b/spec/models/ci/pipeline_spec.rb @@ -153,38 +153,6 @@ RSpec.describe Ci::Pipeline, :mailer, factory_default: :keep, feature_category: pipeline.succeed! end end - - describe 'unlocking artifacts on after a running pipeline succeeds, skipped, canceled, failed or blocked' do - shared_examples 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker' do |event:| - let(:pipeline) { create(:ci_pipeline, :running) } - - it 'schedules Ci::UnlockRefArtifactsOnPipelineStopWorker' do - expect(Ci::UnlockRefArtifactsOnPipelineStopWorker).to receive(:perform_async).with(pipeline.id) - - pipeline.fire_status_event(event) - end - end - - context 'when running pipeline is successful' do - it_behaves_like 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker', event: :succeed - end - - context 'when running pipeline is skipped' do - it_behaves_like 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker', event: :skip - end - - context 'when running pipeline is canceled' do - it_behaves_like 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker', event: :cancel - end - - context 'when running pipeline failed' do - it_behaves_like 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker', event: :drop - end - - context 'when running pipeline is blocked' do - it_behaves_like 'scheduling Ci::UnlockRefArtifactsOnPipelineStopWorker', event: :block - end - end end describe 'pipeline age metric' do diff --git a/spec/models/ci/ref_spec.rb b/spec/models/ci/ref_spec.rb index 9a0df98d8cf..eab5a40bc30 100644 --- a/spec/models/ci/ref_spec.rb +++ b/spec/models/ci/ref_spec.rb @@ -7,6 +7,61 @@ RSpec.describe Ci::Ref do it { is_expected.to belong_to(:project) } + describe 'state machine transitions' do + context 'unlock artifacts transition' do + let(:ci_ref) { create(:ci_ref) } + let(:unlock_artifacts_worker_spy) { class_spy(::Ci::PipelineSuccessUnlockArtifactsWorker) } + + before do + stub_const('Ci::PipelineSuccessUnlockArtifactsWorker', unlock_artifacts_worker_spy) + end + + context 'pipline is locked' do + let!(:pipeline) { create(:ci_pipeline, ci_ref_id: ci_ref.id, locked: :artifacts_locked) } + + where(:initial_state, :action, :count) do + :unknown | :succeed! | 1 + :unknown | :do_fail! | 0 + :success | :succeed! | 1 + :success | :do_fail! | 0 + :failed | :succeed! | 1 + :failed | :do_fail! | 0 + :fixed | :succeed! | 1 + :fixed | :do_fail! | 0 + :broken | :succeed! | 1 + :broken | :do_fail! | 0 + :still_failing | :succeed | 1 + :still_failing | :do_fail | 0 + end + + with_them do + context "when transitioning states" do + before do + status_value = Ci::Ref.state_machines[:status].states[initial_state].value + ci_ref.update!(status: status_value) + end + + it 'calls unlock artifacts service' do + ci_ref.send(action) + + expect(unlock_artifacts_worker_spy).to have_received(:perform_async).exactly(count).times + end + end + end + end + + context 'pipeline is unlocked' do + let!(:pipeline) { create(:ci_pipeline, ci_ref_id: ci_ref.id, locked: :unlocked) } + + it 'does not call unlock artifacts service' do + ci_ref.succeed! + + expect(unlock_artifacts_worker_spy).not_to have_received(:perform_async) + end + end + end + end + describe '.ensure_for' do let_it_be(:project) { create(:project, :repository) } @@ -86,7 +141,7 @@ RSpec.describe Ci::Ref do expect(ci_ref.last_finished_pipeline_id).to eq(pipeline.id) end - context 'when the pipeline is a dangling pipeline' do + context 'when the pipeline a dangling pipeline' do let(:pipeline_source) { Enums::Ci::Pipeline.sources[:ondemand_dast_scan] } it 'returns nil' do @@ -96,47 +151,6 @@ RSpec.describe Ci::Ref do end end - describe '#last_successful_pipeline' do - let_it_be(:ci_ref) { create(:ci_ref) } - - let(:pipeline_source) { Enums::Ci::Pipeline.sources[:push] } - - context 'when there are no successful pipelines' do - let!(:pipeline) { create(:ci_pipeline, :running, ci_ref: ci_ref, source: pipeline_source) } - - it 'returns nil' do - expect(ci_ref.last_successful_pipeline).to be_nil - end - end - - context 'when there are successful pipelines' do - let!(:successful_pipeline) { create(:ci_pipeline, :success, ci_ref: ci_ref, source: pipeline_source) } - let!(:last_successful_pipeline) { create(:ci_pipeline, :success, ci_ref: ci_ref, source: pipeline_source) } - - it 'returns the latest successful pipeline id' do - expect(ci_ref.last_successful_pipeline).to eq(last_successful_pipeline) - end - end - - context 'when there are non-successful pipelines' do - let!(:last_successful_pipeline) { create(:ci_pipeline, :success, ci_ref: ci_ref, source: pipeline_source) } - let!(:failed_pipeline) { create(:ci_pipeline, :failed, ci_ref: ci_ref, source: pipeline_source) } - - it 'returns the latest successful pipeline id' do - expect(ci_ref.last_successful_pipeline).to eq(last_successful_pipeline) - end - end - - context 'when the pipeline is a dangling pipeline' do - let(:pipeline_source) { Enums::Ci::Pipeline.sources[:ondemand_dast_scan] } - let!(:pipeline) { create(:ci_pipeline, :running, ci_ref: ci_ref, source: pipeline_source) } - - it 'returns nil' do - expect(ci_ref.last_finished_pipeline_id).to be_nil - end - end - end - describe '#update_status_by!' do subject { ci_ref.update_status_by!(pipeline) } diff --git a/spec/models/ml/candidate_spec.rb b/spec/models/ml/candidate_spec.rb index 2c823bb875b..7bab49d39df 100644 --- a/spec/models/ml/candidate_spec.rb +++ b/spec/models/ml/candidate_spec.rb @@ -12,6 +12,7 @@ RSpec.describe Ml::Candidate, factory_default: :keep, feature_category: :mlops d describe 'associations' do it { is_expected.to belong_to(:experiment) } + it { is_expected.to belong_to(:project) } it { is_expected.to belong_to(:user) } it { is_expected.to belong_to(:package) } it { is_expected.to have_many(:params) } diff --git a/spec/serializers/admin/abuse_report_entity_spec.rb b/spec/serializers/admin/abuse_report_entity_spec.rb index 7972dbde4e9..760c12d3cf9 100644 --- a/spec/serializers/admin/abuse_report_entity_spec.rb +++ b/spec/serializers/admin/abuse_report_entity_spec.rb @@ -11,12 +11,19 @@ RSpec.describe Admin::AbuseReportEntity, feature_category: :insider_threat do described_class.new(abuse_report) end + before do + allow_next_instance_of(described_class) do |instance| + allow(instance).to receive(:markdown_field).with(abuse_report, :message).and_return(abuse_report.message) + end + end + describe '#as_json' do subject(:entity_hash) { entity.as_json } it 'exposes correct attributes' do expect(entity_hash.keys).to include( :category, + :created_at, :updated_at, :reported_user, :reporter, @@ -25,12 +32,13 @@ RSpec.describe Admin::AbuseReportEntity, feature_category: :insider_threat do :user_blocked, :block_user_path, :remove_report_path, - :remove_user_and_report_path + :remove_user_and_report_path, + :message ) end it 'correctly exposes `reported user`' do - expect(entity_hash[:reported_user].keys).to match_array([:name]) + expect(entity_hash[:reported_user].keys).to match_array([:name, :created_at]) end it 'correctly exposes `reporter`' do @@ -76,5 +84,9 @@ RSpec.describe Admin::AbuseReportEntity, feature_category: :insider_threat do it 'correctly exposes :remove_user_and_report_path' do expect(entity_hash[:remove_user_and_report_path]).to eq admin_abuse_report_path(abuse_report, remove_user: true) end + + it 'correctly exposes :message' do + expect(entity_hash[:message]).to eq(abuse_report.message) + end end end diff --git a/spec/services/ci/unlock_artifacts_service_spec.rb b/spec/services/ci/unlock_artifacts_service_spec.rb index 64b2e66e819..1921ea4bdba 100644 --- a/spec/services/ci/unlock_artifacts_service_spec.rb +++ b/spec/services/ci/unlock_artifacts_service_spec.rb @@ -3,16 +3,6 @@ require 'spec_helper' RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integration do - let_it_be(:ref) { 'master' } - let_it_be(:project) { create(:project) } - let_it_be(:tag_ref_path) { "#{::Gitlab::Git::TAG_REF_PREFIX}#{ref}" } - let_it_be(:ci_ref_tag) { create(:ci_ref, ref_path: tag_ref_path, project: project) } - let_it_be(:branch_ref_path) { "#{::Gitlab::Git::BRANCH_REF_PREFIX}#{ref}" } - let_it_be(:ci_ref_branch) { create(:ci_ref, ref_path: branch_ref_path, project: project) } - let_it_be(:new_ref) { 'new_ref' } - let_it_be(:new_ref_path) { "#{::Gitlab::Git::BRANCH_REF_PREFIX}#{new_ref}" } - let_it_be(:new_ci_ref) { create(:ci_ref, ref_path: new_ref_path, project: project) } - using RSpec::Parameterized::TableSyntax where(:tag) do @@ -23,31 +13,31 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end with_them do - let(:target_ref) { tag ? ci_ref_tag : ci_ref_branch } + let(:ref) { 'master' } + let(:ref_path) { tag ? "#{::Gitlab::Git::TAG_REF_PREFIX}#{ref}" : "#{::Gitlab::Git::BRANCH_REF_PREFIX}#{ref}" } + let(:ci_ref) { create(:ci_ref, ref_path: ref_path) } + let(:project) { ci_ref.project } let(:source_job) { create(:ci_build, pipeline: pipeline) } let!(:old_unlocked_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :unlocked) } let!(:older_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } let!(:older_ambiguous_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: !tag, project: project, locked: :artifacts_locked) } let!(:code_coverage_pipeline) { create(:ci_pipeline, :with_coverage_report_artifact, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } - let!(:successful_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } - let!(:child_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, child_of: successful_pipeline, project: project, locked: :artifacts_locked) } - let!(:last_successful_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } - let!(:last_successful_child_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, child_of: last_successful_pipeline, project: project, locked: :artifacts_locked) } - let!(:older_failed_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, status: :failed, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } - let!(:latest_failed_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, status: :failed, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } - let!(:blocked_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, status: :manual, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } + let!(:pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } + let!(:child_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, child_of: pipeline, project: project, locked: :artifacts_locked) } + let!(:newer_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: ref, tag: tag, project: project, locked: :artifacts_locked) } let!(:other_ref_pipeline) { create(:ci_pipeline, :with_persisted_artifacts, ref: 'other_ref', tag: tag, project: project, locked: :artifacts_locked) } + let!(:sources_pipeline) { create(:ci_sources_pipeline, source_job: source_job, source_project: project, pipeline: child_pipeline, project: project) } before do stub_const("#{described_class}::BATCH_SIZE", 1) end describe '#execute' do - subject(:execute) { described_class.new(successful_pipeline.project, successful_pipeline.user).execute(target_ref, before_pipeline) } + subject(:execute) { described_class.new(pipeline.project, pipeline.user).execute(ci_ref, before_pipeline) } context 'when running on a ref before a pipeline' do - let(:before_pipeline) { successful_pipeline } + let(:before_pipeline) { pipeline } it 'unlocks artifacts from older pipelines' do expect { execute }.to change { older_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') @@ -58,15 +48,15 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end it 'does not unlock artifacts from newer pipelines' do - expect { execute }.not_to change { last_successful_pipeline.reload.locked }.from('artifacts_locked') + expect { execute }.not_to change { newer_pipeline.reload.locked }.from('artifacts_locked') end it 'does not lock artifacts from old unlocked pipelines' do expect { execute }.not_to change { old_unlocked_pipeline.reload.locked }.from('unlocked') end - it 'does not unlock artifacts from the successful pipeline' do - expect { execute }.not_to change { successful_pipeline.reload.locked }.from('artifacts_locked') + it 'does not unlock artifacts from the same pipeline' do + expect { execute }.not_to change { pipeline.reload.locked }.from('artifacts_locked') end it 'does not unlock artifacts for other refs' do @@ -84,60 +74,6 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra it 'unlocks pipeline artifact records' do expect { execute }.to change { ::Ci::PipelineArtifact.artifact_unlocked.count }.from(0).to(1) end - - context 'when before_pipeline is a failed pipeline' do - let(:before_pipeline) { latest_failed_pipeline } - - it 'unlocks artifacts from older failed pipeline' do - expect { execute }.to change { older_failed_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') - end - - it 'does not unlock artifact from the latest failed pipeline' do - expect { execute }.not_to change { latest_failed_pipeline.reload.locked }.from('artifacts_locked') - end - - it 'does not unlock artifacts from the last successful pipeline' do - expect { execute }.not_to change { last_successful_pipeline.reload.locked }.from('artifacts_locked') - end - - it 'does not unlock artifacts from the child of last successful pipeline' do - expect { execute }.not_to change { last_successful_child_pipeline.reload.locked }.from('artifacts_locked') - end - end - - context 'when before_pipeline is a blocked pipeline' do - let(:before_pipeline) { blocked_pipeline } - - it 'unlocks artifacts from failed pipeline' do - expect { execute }.to change { latest_failed_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') - end - - it 'does not unlock artifact from the latest blocked pipeline' do - expect { execute }.not_to change { blocked_pipeline.reload.locked }.from('artifacts_locked') - end - - it 'does not unlock artifacts from the last successful pipeline' do - expect { execute }.not_to change { last_successful_pipeline.reload.locked }.from('artifacts_locked') - end - end - - # rubocop:todo RSpec/MultipleMemoizedHelpers - context 'when the ref has no successful pipeline' do - let!(:target_ref) { new_ci_ref } - let!(:failed_pipeline_1) { create(:ci_pipeline, :with_persisted_artifacts, status: :failed, ref: new_ref, project: project, locked: :artifacts_locked) } - let!(:failed_pipeline_2) { create(:ci_pipeline, :with_persisted_artifacts, status: :failed, ref: new_ref, project: project, locked: :artifacts_locked) } - - let(:before_pipeline) { failed_pipeline_2 } - - it 'unlocks earliest failed pipeline' do - expect { execute }.to change { failed_pipeline_1.reload.locked }.from('artifacts_locked').to('unlocked') - end - - it 'does not unlock latest failed pipeline' do - expect { execute }.not_to change { failed_pipeline_2.reload.locked }.from('artifacts_locked') - end - end - # rubocop:enable RSpec/MultipleMemoizedHelpers end context 'when running on just the ref' do @@ -148,11 +84,11 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end it 'unlocks artifacts from newer pipelines' do - expect { execute }.to change { last_successful_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') + expect { execute }.to change { newer_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') end - it 'unlocks artifacts from the successful pipeline' do - expect { execute }.to change { successful_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') + it 'unlocks artifacts from the same pipeline' do + expect { execute }.to change { pipeline.reload.locked }.from('artifacts_locked').to('unlocked') end it 'does not unlock artifacts for tag or branch with same name as ref' do @@ -168,7 +104,7 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end it 'unlocks job artifact records' do - expect { execute }.to change { ::Ci::JobArtifact.artifact_unlocked.count }.from(0).to(16) + expect { execute }.to change { ::Ci::JobArtifact.artifact_unlocked.count }.from(0).to(8) end it 'unlocks pipeline artifact records' do @@ -178,10 +114,10 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end describe '#unlock_pipelines_query' do - subject { described_class.new(successful_pipeline.project, successful_pipeline.user).unlock_pipelines_query(target_ref, before_pipeline) } + subject { described_class.new(pipeline.project, pipeline.user).unlock_pipelines_query(ci_ref, before_pipeline) } context 'when running on a ref before a pipeline' do - let(:before_pipeline) { successful_pipeline } + let(:before_pipeline) { pipeline } it 'produces the expected SQL string' do expect(subject.squish).to eq <<~SQL.squish @@ -196,7 +132,7 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra FROM "ci_pipelines" WHERE - "ci_pipelines"."ci_ref_id" = #{target_ref.id} + "ci_pipelines"."ci_ref_id" = #{ci_ref.id} AND "ci_pipelines"."locked" = 1 AND "ci_pipelines"."id" < #{before_pipeline.id} AND "ci_pipelines"."id" NOT IN @@ -226,33 +162,6 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra "base_and_descendants" AS "ci_pipelines") - AND "ci_pipelines"."id" NOT IN - (WITH RECURSIVE - "base_and_descendants" - AS - ((SELECT - "ci_pipelines".* - FROM - "ci_pipelines" - WHERE - "ci_pipelines"."id" = #{target_ref.last_successful_pipeline.id}) - UNION - (SELECT - "ci_pipelines".* - FROM - "ci_pipelines", - "base_and_descendants", - "ci_sources_pipelines" - WHERE - "ci_sources_pipelines"."pipeline_id" = "ci_pipelines"."id" - AND "ci_sources_pipelines"."source_pipeline_id" = "base_and_descendants"."id" - AND "ci_sources_pipelines"."source_project_id" = "ci_sources_pipelines"."project_id")) - SELECT - "id" - FROM - "base_and_descendants" - AS - "ci_pipelines") LIMIT 1 FOR UPDATE SKIP LOCKED) @@ -277,7 +186,7 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra FROM "ci_pipelines" WHERE - "ci_pipelines"."ci_ref_id" = #{target_ref.id} + "ci_pipelines"."ci_ref_id" = #{ci_ref.id} AND "ci_pipelines"."locked" = 1 LIMIT 1 FOR UPDATE @@ -290,7 +199,7 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end describe '#unlock_job_artifacts_query' do - subject { described_class.new(successful_pipeline.project, successful_pipeline.user).unlock_job_artifacts_query(pipeline_ids) } + subject { described_class.new(pipeline.project, pipeline.user).unlock_job_artifacts_query(pipeline_ids) } context 'when given a single pipeline ID' do let(:pipeline_ids) { [older_pipeline.id] } @@ -317,7 +226,7 @@ RSpec.describe Ci::UnlockArtifactsService, feature_category: :continuous_integra end context 'when given multiple pipeline IDs' do - let(:pipeline_ids) { [older_pipeline.id, last_successful_pipeline.id, successful_pipeline.id] } + let(:pipeline_ids) { [older_pipeline.id, newer_pipeline.id, pipeline.id] } it 'produces the expected SQL string' do expect(subject.squish).to eq <<~SQL.squish diff --git a/spec/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker_spec.rb b/spec/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker_spec.rb deleted file mode 100644 index a9f0e40e466..00000000000 --- a/spec/workers/ci/unlock_ref_artifacts_on_pipeline_stop_worker_spec.rb +++ /dev/null @@ -1,64 +0,0 @@ -# frozen_string_literal: true - -require 'spec_helper' - -RSpec.describe Ci::UnlockRefArtifactsOnPipelineStopWorker, feature_category: :build_artifacts do - describe '#perform' do - subject(:perform) { described_class.new.perform(pipeline_id) } - - include_examples 'an idempotent worker' do - subject(:idempotent_perform) { perform_multiple(pipeline.id, exec_times: 2) } - - let!(:older_pipeline) do - create(:ci_pipeline, :success, :with_job, locked: :artifacts_locked).tap do |pipeline| - create(:ci_job_artifact, job: pipeline.builds.first) - end - end - - let!(:pipeline) do - create(:ci_pipeline, :success, :with_job, ref: older_pipeline.ref, tag: older_pipeline.tag, - project: older_pipeline.project, locked: :unlocked).tap do |pipeline| - create(:ci_job_artifact, job: pipeline.builds.first) - end - end - - it 'unlocks the artifacts from older pipelines' do - expect { idempotent_perform }.to change { older_pipeline.reload.locked }.from('artifacts_locked').to('unlocked') - end - end - - context 'when pipeline exists' do - let(:pipeline) { create(:ci_pipeline, :success, :with_job) } - let(:pipeline_id) { pipeline.id } - - it 'calls the Ci::UnlockArtifactsService with the ref and pipeline' do - expect_next_instance_of(Ci::UnlockArtifactsService) do |service| - expect(service).to receive(:execute).with(pipeline.ci_ref, pipeline).and_call_original - end - - perform - end - end - - context 'when pipeline does not exist' do - let(:pipeline_id) { non_existing_record_id } - - it 'does not call the service' do - expect(Ci::UnlockArtifactsService).not_to receive(:new) - - perform - end - end - - context 'when the ref no longer exists' do - let(:pipeline) { create(:ci_pipeline, :success, :with_job, ci_ref_presence: false) } - let(:pipeline_id) { pipeline.id } - - it 'does not call the service' do - expect(Ci::UnlockArtifactsService).not_to receive(:new) - - perform - end - end - end -end diff --git a/yarn.lock b/yarn.lock index 3c40e01e823..9a36f701ca0 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1127,10 +1127,10 @@ resolved "https://registry.yarnpkg.com/@gitlab/svgs/-/svgs-3.31.0.tgz#37e0a189def22400758267a84b61840a74aaf937" integrity sha512-VzbMlj7TSroWvHDBMvCF4EDOnozFah5wPSyI+YJ+eefQoX0Fzu6RIZ9h8+lhnRzffygcValdVNdnuzMbXB+Q/g== -"@gitlab/ui@58.5.0": - version "58.5.0" - resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-58.5.0.tgz#d82eae865db2af8e5f899273260b4b349b45ab47" - integrity sha512-doPDyhnBfkQIEz6aO9sKmc40TUCBNz4ghZXwCubfg6LzOc5dh+pa3tu3+Lghy1j9+ONzJYOpLWx4HknbEkgo7w== +"@gitlab/ui@58.6.0": + version "58.6.0" + resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-58.6.0.tgz#4f49ca6374fa376a53e5bad866155620bdaac45b" + integrity sha512-OGkk5nxECUZ1vZEvar+49xz/PGdJoKzy9ZOIDF4cXTkRGtxXJApqglFH0Uy39l3mzjBhHMHZuLd0122wWj0XJA== dependencies: "@popperjs/core" "^2.11.2" bootstrap-vue "2.23.1" |