Backport the EE schema and migrations to CE

This backports all EE schema changes to CE, including EE migrations,
ensuring both use the same schema.

== Updated tests

A spec related to ghost and support bot users had to be modified to make
it pass. The spec in question assumes that the "support_bot" column
exists when defining the spec. In the single codebase setup this is not
the case, as the column is backported in a later migration. Any attempt
to use a different schema version or use of "around" blocks to
conditionally disable specs won't help, as reverting the backport
migration would also drop the "support_bot" column. Removing the
"support_bot" tests entirely appears to be the only solution.

We also need to update some foreign key tests now that we have
backported the EE columns. Fortunately, these changes are very minor.

== Backporting migrations

This commit moves EE specific migrations (except those for the Geo
tracking database) and related files to CE, and also removes any traces
of the ee/db directory.

Some migrations had to be modified or removed, as they no longer work
with the schema being backported. These migrations were all quite old,
so we opted for removing them where modifying them would take too much
time and effort.

Some old migrations were modified in EE, while also existing in CE. In
these cases we took the EE code, and in one case removed them entirely.
It's not worth spending time trying to merge these changes somehow as we
plan to remove old migrations around the release of 12.0, see
https://gitlab.com/gitlab-org/gitlab-ce/issues/59177 for more details.

16 files changed, 152 insertions, 770 deletions

diff --git a/lib/gitlab/background_migration/create_gpg_key_subkeys_from_gpg_keys.rb b/lib/gitlab/background_migration/create_gpg_key_subkeys_from_gpg_keys.rb
deleted file mode 100644
index da8265a3a5f..00000000000
--- a/lib/gitlab/background_migration/create_gpg_key_subkeys_from_gpg_keys.rb
+++ /dev/null
@@ -1,56 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-class Gitlab::BackgroundMigration::CreateGpgKeySubkeysFromGpgKeys
-  class GpgKey < ActiveRecord::Base
-    self.table_name = 'gpg_keys'
-
-    include EachBatch
-    include ShaAttribute
-
-    sha_attribute :primary_keyid
-    sha_attribute :fingerprint
-
-    has_many :subkeys, class_name: 'GpgKeySubkey'
-  end
-
-  class GpgKeySubkey < ActiveRecord::Base
-    self.table_name = 'gpg_key_subkeys'
-
-    include ShaAttribute
-
-    sha_attribute :keyid
-    sha_attribute :fingerprint
-  end
-
-  def perform(gpg_key_id)
-    gpg_key = GpgKey.find_by(id: gpg_key_id)
-
-    return if gpg_key.nil?
-    return if gpg_key.subkeys.any?
-
-    create_subkeys(gpg_key)
-    update_signatures(gpg_key)
-  end
-
-  private
-
-  def create_subkeys(gpg_key)
-    gpg_subkeys = Gitlab::Gpg.subkeys_from_key(gpg_key.key)
-
-    gpg_subkeys[gpg_key.primary_keyid.upcase]&.each do |subkey_data|
-      gpg_key.subkeys.build(keyid: subkey_data[:keyid], fingerprint: subkey_data[:fingerprint])
-    end
-
-    # Improve latency by doing all INSERTs in a single call
-    GpgKey.transaction do
-      gpg_key.save!
-    end
-  end
-
-  def update_signatures(gpg_key)
-    return unless gpg_key.subkeys.exists?
-
-    InvalidGpgSignatureUpdateWorker.perform_async(gpg_key.id)
-  end
-end
diff --git a/lib/gitlab/background_migration/delete_diff_files.rb b/lib/gitlab/background_migration/delete_diff_files.rb
deleted file mode 100644
index 664ead1af44..00000000000
--- a/lib/gitlab/background_migration/delete_diff_files.rb
+++ /dev/null
@@ -1,81 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class DeleteDiffFiles
-      class MergeRequestDiff < ActiveRecord::Base
-        self.table_name = 'merge_request_diffs'
-
-        belongs_to :merge_request
-        has_many :merge_request_diff_files
-      end
-
-      class MergeRequestDiffFile < ActiveRecord::Base
-        self.table_name = 'merge_request_diff_files'
-      end
-
-      DEAD_TUPLES_THRESHOLD = 50_000
-      VACUUM_WAIT_TIME = 5.minutes
-
-      def perform(ids)
-        @ids = ids
-
-        # We should reschedule until deadtuples get in a desirable
-        # state (e.g. < 50_000). That may take more than one reschedule.
-        #
-        if should_wait_deadtuple_vacuum?
-          reschedule
-          return
-        end
-
-        prune_diff_files
-      end
-
-      def should_wait_deadtuple_vacuum?
-        return false unless Gitlab::Database.postgresql?
-
-        diff_files_dead_tuples_count >= DEAD_TUPLES_THRESHOLD
-      end
-
-      private
-
-      def reschedule
-        BackgroundMigrationWorker.perform_in(VACUUM_WAIT_TIME, self.class.name.demodulize, [@ids])
-      end
-
-      def diffs_collection
-        MergeRequestDiff.where(id: @ids)
-      end
-
-      def diff_files_dead_tuples_count
-        dead_tuple =
-          execute_statement("SELECT n_dead_tup FROM pg_stat_all_tables "\
-                            "WHERE relname = 'merge_request_diff_files'")[0]
-
-        dead_tuple&.fetch('n_dead_tup', 0).to_i
-      end
-
-      def prune_diff_files
-        removed = 0
-        updated = 0
-
-        MergeRequestDiff.transaction do
-          updated = diffs_collection.update_all(state: 'without_files')
-          removed = MergeRequestDiffFile.where(merge_request_diff_id: @ids).delete_all
-        end
-
-        log_info("Removed #{removed} merge_request_diff_files rows, "\
-                 "updated #{updated} merge_request_diffs rows")
-      end
-
-      def execute_statement(sql)
-        ActiveRecord::Base.connection.execute(sql)
-      end
-
-      def log_info(message)
-        Rails.logger.info("BackgroundMigration::DeleteDiffFiles - #{message}")
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb b/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb
deleted file mode 100644
index 58df74cfa9b..00000000000
--- a/lib/gitlab/background_migration/deserialize_merge_request_diffs_and_commits.rb
+++ /dev/null
@@ -1,149 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Metrics/MethodLength
-# rubocop:disable Metrics/AbcSize
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class DeserializeMergeRequestDiffsAndCommits
-      attr_reader :diff_ids, :commit_rows, :file_rows
-
-      class Error < StandardError
-        def backtrace
-          cause.backtrace
-        end
-      end
-
-      class MergeRequestDiff < ActiveRecord::Base
-        self.table_name = 'merge_request_diffs'
-      end
-
-      BUFFER_ROWS = 1000
-      DIFF_FILE_BUFFER_ROWS = 100
-
-      def perform(start_id, stop_id)
-        merge_request_diffs = MergeRequestDiff
-                               .select(:id, :st_commits, :st_diffs)
-                               .where('st_commits IS NOT NULL OR st_diffs IS NOT NULL')
-                               .where(id: start_id..stop_id)
-
-        reset_buffers!
-
-        merge_request_diffs.each do |merge_request_diff|
-          commits, files = single_diff_rows(merge_request_diff)
-
-          diff_ids << merge_request_diff.id
-          commit_rows.concat(commits)
-          file_rows.concat(files)
-
-          if diff_ids.length > BUFFER_ROWS ||
-              commit_rows.length > BUFFER_ROWS ||
-              file_rows.length > DIFF_FILE_BUFFER_ROWS
-
-            flush_buffers!
-          end
-        end
-
-        flush_buffers!
-      rescue => e
-        Rails.logger.info("#{self.class.name}: failed for IDs #{merge_request_diffs.map(&:id)} with #{e.class.name}")
-
-        raise Error.new(e.inspect)
-      end
-
-      private
-
-      def reset_buffers!
-        @diff_ids = []
-        @commit_rows = []
-        @file_rows = []
-      end
-
-      def flush_buffers!
-        if diff_ids.any?
-          commit_rows.each_slice(BUFFER_ROWS).each do |commit_rows_slice|
-            bulk_insert('merge_request_diff_commits', commit_rows_slice)
-          end
-
-          file_rows.each_slice(DIFF_FILE_BUFFER_ROWS).each do |file_rows_slice|
-            bulk_insert('merge_request_diff_files', file_rows_slice)
-          end
-
-          MergeRequestDiff.where(id: diff_ids).update_all(st_commits: nil, st_diffs: nil)
-        end
-
-        reset_buffers!
-      end
-
-      def bulk_insert(table, rows)
-        Gitlab::Database.bulk_insert(table, rows)
-      rescue ActiveRecord::RecordNotUnique
-        ids = rows.map { |row| row[:merge_request_diff_id] }.uniq.sort
-
-        Rails.logger.info("#{self.class.name}: rows inserted twice for IDs #{ids}")
-      end
-
-      def single_diff_rows(merge_request_diff)
-        sha_attribute = Gitlab::Database::ShaAttribute.new
-        commits = YAML.load(merge_request_diff.st_commits) rescue []
-        commits ||= []
-
-        commit_rows = commits.map.with_index do |commit, index|
-          commit_hash = commit.to_hash.with_indifferent_access.except(:parent_ids)
-          sha = commit_hash.delete(:id)
-
-          commit_hash.merge(
-            merge_request_diff_id: merge_request_diff.id,
-            relative_order: index,
-            sha: sha_attribute.serialize(sha)
-          )
-        end
-
-        diffs = YAML.load(merge_request_diff.st_diffs) rescue []
-        diffs = [] unless valid_raw_diffs?(diffs)
-
-        file_rows = diffs.map.with_index do |diff, index|
-          diff_hash = diff.to_hash.with_indifferent_access.merge(
-            binary: false,
-            merge_request_diff_id: merge_request_diff.id,
-            relative_order: index
-          )
-
-          diff_hash.tap do |hash|
-            diff_text = hash[:diff]
-
-            hash[:too_large] = !!hash[:too_large]
-
-            hash[:a_mode] ||= guess_mode(hash[:new_file], hash[:diff])
-            hash[:b_mode] ||= guess_mode(hash[:deleted_file], hash[:diff])
-
-            # Compatibility with old diffs created with Psych.
-            if diff_text.encoding == Encoding::BINARY && !diff_text.ascii_only?
-              hash[:binary] = true
-              hash[:diff] = [diff_text].pack('m0')
-            end
-          end
-        end
-
-        [commit_rows, file_rows]
-      end
-
-      # This doesn't have to be 100% accurate, because it's only used for
-      # display - it won't change file modes in the repository. Submodules are
-      # created as 600, regular files as 644.
-      def guess_mode(file_missing, diff)
-        return '0' if file_missing
-
-        diff.include?('Subproject commit') ? '160000' : '100644'
-      end
-
-      # Unlike MergeRequestDiff#valid_raw_diff?, don't count Rugged objects as
-      # valid, because we don't render them usefully anyway.
-      def valid_raw_diffs?(diffs)
-        return false unless diffs.respond_to?(:each)
-
-        diffs.all? { |diff| diff.is_a?(Hash) }
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/merge_request_assignees_migration_progress_check.rb b/lib/gitlab/background_migration/merge_request_assignees_migration_progress_check.rb
new file mode 100644
index 00000000000..e948cedaad5
--- /dev/null
+++ b/lib/gitlab/background_migration/merge_request_assignees_migration_progress_check.rb
@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module Gitlab
+  module BackgroundMigration
+    # rubocop: disable Style/Documentation
+    class MergeRequestAssigneesMigrationProgressCheck
+      include Gitlab::Utils::StrongMemoize
+
+      RESCHEDULE_DELAY = 3.hours
+      WORKER = 'PopulateMergeRequestAssigneesTable'.freeze
+      DeadJobsError = Class.new(StandardError)
+
+      def perform
+        raise DeadJobsError, "Only dead background jobs in the queue for #{WORKER}" if !ongoing? && dead_jobs?
+
+        if ongoing?
+          BackgroundMigrationWorker.perform_in(RESCHEDULE_DELAY, self.class.name)
+        else
+          Feature.enable(:multiple_merge_request_assignees)
+        end
+      end
+
+      private
+
+      def dead_jobs?
+        strong_memoize(:dead_jobs) do
+          migration_klass.dead_jobs?(WORKER)
+        end
+      end
+
+      def ongoing?
+        strong_memoize(:ongoing) do
+          migration_klass.exists?(WORKER) || migration_klass.retrying_jobs?(WORKER)
+        end
+      end
+
+      def migration_klass
+        Gitlab::BackgroundMigration
+      end
+    end
+    # rubocop: enable Style/Documentation
+  end
+end
diff --git a/lib/gitlab/background_migration/populate_external_pipeline_source.rb b/lib/gitlab/background_migration/populate_external_pipeline_source.rb
deleted file mode 100644
index 036fe641757..00000000000
--- a/lib/gitlab/background_migration/populate_external_pipeline_source.rb
+++ /dev/null
@@ -1,50 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class PopulateExternalPipelineSource
-      module Migratable
-        class Pipeline < ActiveRecord::Base
-          self.table_name = 'ci_pipelines'
-
-          def self.sources
-            {
-              unknown: nil,
-              push: 1,
-              web: 2,
-              trigger: 3,
-              schedule: 4,
-              api: 5,
-              external: 6
-            }
-          end
-        end
-
-        class CommitStatus < ActiveRecord::Base
-          self.table_name = 'ci_builds'
-          self.inheritance_column = :_type_disabled
-
-          scope :has_pipeline, -> { where('ci_builds.commit_id=ci_pipelines.id') }
-          scope :of_type, -> (type) { where('type=?', type) }
-        end
-      end
-
-      def perform(start_id, stop_id)
-        external_pipelines(start_id, stop_id)
-          .update_all(source: Migratable::Pipeline.sources[:external])
-      end
-
-      private
-
-      def external_pipelines(start_id, stop_id)
-        Migratable::Pipeline.where(id: (start_id..stop_id))
-          .where(
-            'EXISTS (?) AND NOT EXISTS (?)',
-            Migratable::CommitStatus.of_type('GenericCommitStatus').has_pipeline.select(1),
-            Migratable::CommitStatus.of_type('Ci::Build').has_pipeline.select(1)
-          )
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/populate_import_state.rb b/lib/gitlab/background_migration/populate_import_state.rb
deleted file mode 100644
index 695a2a713c5..00000000000
--- a/lib/gitlab/background_migration/populate_import_state.rb
+++ /dev/null
@@ -1,39 +0,0 @@
-# frozen_string_literal: true
-
-module Gitlab
-  module BackgroundMigration
-    # This background migration creates all the records on the
-    # import state table for projects that are considered imports or forks
-    class PopulateImportState
-      def perform(start_id, end_id)
-        move_attributes_data_to_import_state(start_id, end_id)
-      rescue ActiveRecord::RecordNotUnique
-        retry
-      end
-
-      def move_attributes_data_to_import_state(start_id, end_id)
-        Rails.logger.info("#{self.class.name} - Moving import attributes data to project mirror data table: #{start_id} - #{end_id}")
-
-        ActiveRecord::Base.connection.execute <<~SQL
-          INSERT INTO project_mirror_data (project_id, status, jid, last_error)
-          SELECT id, import_status, import_jid, import_error
-          FROM projects
-          WHERE projects.import_status != 'none'
-          AND projects.id BETWEEN #{start_id} AND #{end_id}
-          AND NOT EXISTS (
-            SELECT id
-            FROM project_mirror_data
-            WHERE project_id = projects.id
-          )
-        SQL
-
-        ActiveRecord::Base.connection.execute <<~SQL
-          UPDATE projects
-          SET import_status = 'none'
-          WHERE import_status != 'none'
-          AND id BETWEEN #{start_id} AND #{end_id}
-        SQL
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data.rb b/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data.rb
deleted file mode 100644
index d89ce358bb9..00000000000
--- a/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data.rb
+++ /dev/null
@@ -1,132 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class PopulateMergeRequestMetricsWithEventsData
-      def perform(min_merge_request_id, max_merge_request_id)
-        insert_metrics_for_range(min_merge_request_id, max_merge_request_id)
-        update_metrics_with_events_data(min_merge_request_id, max_merge_request_id)
-      end
-
-      # Inserts merge_request_metrics records for merge_requests without it for
-      # a given merge request batch.
-      def insert_metrics_for_range(min, max)
-        metrics_not_exists_clause =
-          <<-SQL.strip_heredoc
-            NOT EXISTS (SELECT 1 FROM merge_request_metrics
-                        WHERE merge_request_metrics.merge_request_id = merge_requests.id)
-          SQL
-
-        MergeRequest.where(metrics_not_exists_clause).where(id: min..max).each_batch do |batch|
-          select_sql = batch.select(:id, :created_at, :updated_at).to_sql
-
-          execute("INSERT INTO merge_request_metrics (merge_request_id, created_at, updated_at) #{select_sql}")
-        end
-      end
-
-      def update_metrics_with_events_data(min, max)
-        if Gitlab::Database.postgresql?
-          # Uses WITH syntax in order to update merged and closed events with a single UPDATE.
-          # WITH is not supported by MySQL.
-          update_events_for_range(min, max)
-        else
-          update_merged_events_for_range(min, max)
-          update_closed_events_for_range(min, max)
-        end
-      end
-
-      private
-
-      # Updates merge_request_metrics latest_closed_at, latest_closed_by_id and merged_by_id
-      # based on the latest event records on events table for a given merge request batch.
-      def update_events_for_range(min, max)
-        sql = <<-SQL.strip_heredoc
-          WITH events_for_update AS (
-            SELECT DISTINCT ON (target_id, action) target_id, action, author_id, updated_at
-             FROM events
-            WHERE target_id BETWEEN #{min} AND #{max}
-              AND target_type = 'MergeRequest'
-              AND action IN (#{Event::CLOSED},#{Event::MERGED})
-            ORDER BY target_id, action, id DESC
-          )
-          UPDATE merge_request_metrics met
-            SET latest_closed_at = latest_closed.updated_at,
-                latest_closed_by_id = latest_closed.author_id,
-                merged_by_id = latest_merged.author_id
-           FROM (SELECT * FROM events_for_update WHERE action = #{Event::CLOSED}) AS latest_closed
-           FULL OUTER JOIN
-                (SELECT * FROM events_for_update WHERE action = #{Event::MERGED}) AS latest_merged
-          USING (target_id)
-          WHERE target_id = merge_request_id;
-        SQL
-
-        execute(sql)
-      end
-
-      # Updates merge_request_metrics latest_closed_at, latest_closed_by_id based on the latest closed
-      # records on events table for a given merge request batch.
-      def update_closed_events_for_range(min, max)
-        sql =
-          <<-SQL.strip_heredoc
-            UPDATE merge_request_metrics metrics,
-              (#{select_events(min, max, Event::CLOSED)}) closed_events
-            SET metrics.latest_closed_by_id = closed_events.author_id,
-                metrics.latest_closed_at = closed_events.updated_at #{where_matches_closed_events};
-          SQL
-
-        execute(sql)
-      end
-
-      # Updates merge_request_metrics merged_by_id based on the latest merged
-      # records on events table for a given merge request batch.
-      def update_merged_events_for_range(min, max)
-        sql =
-          <<-SQL.strip_heredoc
-            UPDATE merge_request_metrics metrics,
-              (#{select_events(min, max, Event::MERGED)}) merged_events
-            SET metrics.merged_by_id = merged_events.author_id #{where_matches_merged_events};
-          SQL
-
-        execute(sql)
-      end
-
-      def execute(sql)
-        @connection ||= ActiveRecord::Base.connection
-        @connection.execute(sql)
-      end
-
-      def select_events(min, max, action)
-        select_max_event_id = <<-SQL.strip_heredoc
-          SELECT max(id)
-          FROM events
-          WHERE action = #{action}
-          AND target_type = 'MergeRequest'
-          AND target_id BETWEEN #{min} AND #{max}
-          GROUP BY target_id
-        SQL
-
-        <<-SQL.strip_heredoc
-          SELECT author_id, updated_at, target_id
-          FROM events
-          WHERE id IN(#{select_max_event_id})
-        SQL
-      end
-
-      def where_matches_closed_events
-        <<-SQL.strip_heredoc
-          WHERE metrics.merge_request_id = closed_events.target_id
-          AND metrics.latest_closed_at IS NULL
-          AND metrics.latest_closed_by_id IS NULL
-        SQL
-      end
-
-      def where_matches_merged_events
-        <<-SQL.strip_heredoc
-          WHERE metrics.merge_request_id = merged_events.target_id
-          AND metrics.merged_by_id IS NULL
-        SQL
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data_improved.rb b/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data_improved.rb
deleted file mode 100644
index 37592d67dd9..00000000000
--- a/lib/gitlab/background_migration/populate_merge_request_metrics_with_events_data_improved.rb
+++ /dev/null
@@ -1,99 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class PopulateMergeRequestMetricsWithEventsDataImproved
-      CLOSED_EVENT_ACTION = 3
-      MERGED_EVENT_ACTION = 7
-
-      def perform(min_merge_request_id, max_merge_request_id)
-        insert_metrics_for_range(min_merge_request_id, max_merge_request_id)
-        update_metrics_with_events_data(min_merge_request_id, max_merge_request_id)
-      end
-
-      # Inserts merge_request_metrics records for merge_requests without it for
-      # a given merge request batch.
-      def insert_metrics_for_range(min, max)
-        metrics_not_exists_clause =
-          <<-SQL.strip_heredoc
-            NOT EXISTS (SELECT 1 FROM merge_request_metrics
-                        WHERE merge_request_metrics.merge_request_id = merge_requests.id)
-          SQL
-
-        MergeRequest.where(metrics_not_exists_clause).where(id: min..max).each_batch do |batch|
-          select_sql = batch.select(:id, :created_at, :updated_at).to_sql
-
-          execute("INSERT INTO merge_request_metrics (merge_request_id, created_at, updated_at) #{select_sql}")
-        end
-      end
-
-      def update_metrics_with_events_data(min, max)
-        if Gitlab::Database.postgresql?
-          psql_update_metrics_with_events_data(min, max)
-        else
-          mysql_update_metrics_with_events_data(min, max)
-        end
-      end
-
-      def psql_update_metrics_with_events_data(min, max)
-        update_sql = <<-SQL.strip_heredoc
-          UPDATE merge_request_metrics
-          SET (latest_closed_at,
-             latest_closed_by_id) =
-          ( SELECT updated_at,
-                   author_id
-           FROM events
-           WHERE target_id = merge_request_id
-             AND target_type = 'MergeRequest'
-             AND action = #{CLOSED_EVENT_ACTION}
-           ORDER BY id DESC
-           LIMIT 1 ),
-             merged_by_id =
-          ( SELECT author_id
-           FROM events
-           WHERE target_id = merge_request_id
-             AND target_type = 'MergeRequest'
-             AND action = #{MERGED_EVENT_ACTION}
-           ORDER BY id DESC
-           LIMIT 1 )
-          WHERE merge_request_id BETWEEN #{min} AND #{max}
-        SQL
-
-        execute(update_sql)
-      end
-
-      def mysql_update_metrics_with_events_data(min, max)
-        closed_updated_at_subquery = mysql_events_select(:updated_at, CLOSED_EVENT_ACTION)
-        closed_author_id_subquery = mysql_events_select(:author_id, CLOSED_EVENT_ACTION)
-        merged_author_id_subquery = mysql_events_select(:author_id, MERGED_EVENT_ACTION)
-
-        update_sql = <<-SQL.strip_heredoc
-          UPDATE merge_request_metrics
-          SET latest_closed_at = (#{closed_updated_at_subquery}),
-              latest_closed_by_id = (#{closed_author_id_subquery}),
-              merged_by_id = (#{merged_author_id_subquery})
-          WHERE merge_request_id BETWEEN #{min} AND #{max}
-        SQL
-
-        execute(update_sql)
-      end
-
-      def mysql_events_select(column, action)
-        <<-SQL.strip_heredoc
-          SELECT #{column} FROM events
-           WHERE target_id = merge_request_id
-             AND target_type = 'MergeRequest'
-             AND action = #{action}
-           ORDER BY id DESC
-           LIMIT 1
-        SQL
-      end
-
-      def execute(sql)
-        @connection ||= ActiveRecord::Base.connection
-        @connection.execute(sql)
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/redact_links.rb b/lib/gitlab/background_migration/redact_links.rb
deleted file mode 100644
index 92256e59a6c..00000000000
--- a/lib/gitlab/background_migration/redact_links.rb
+++ /dev/null
@@ -1,51 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-require_relative 'redact_links/redactable'
-
-module Gitlab
-  module BackgroundMigration
-    class RedactLinks
-      class Note < ActiveRecord::Base
-        include EachBatch
-        include ::Gitlab::BackgroundMigration::RedactLinks::Redactable
-
-        self.table_name = 'notes'
-        self.inheritance_column = :_type_disabled
-      end
-
-      class Issue < ActiveRecord::Base
-        include EachBatch
-        include ::Gitlab::BackgroundMigration::RedactLinks::Redactable
-
-        self.table_name = 'issues'
-        self.inheritance_column = :_type_disabled
-      end
-
-      class MergeRequest < ActiveRecord::Base
-        include EachBatch
-        include ::Gitlab::BackgroundMigration::RedactLinks::Redactable
-
-        self.table_name = 'merge_requests'
-        self.inheritance_column = :_type_disabled
-      end
-
-      class Snippet < ActiveRecord::Base
-        include EachBatch
-        include ::Gitlab::BackgroundMigration::RedactLinks::Redactable
-
-        self.table_name = 'snippets'
-        self.inheritance_column = :_type_disabled
-      end
-
-      def perform(model_name, field, start_id, stop_id)
-        link_pattern = "%/sent_notifications/" + ("_" * 32) + "/unsubscribe%"
-        model = "Gitlab::BackgroundMigration::RedactLinks::#{model_name}".constantize
-
-        model.where("#{field} like ?", link_pattern).where(id: start_id..stop_id).each do |resource|
-          resource.redact_field!(field)
-        end
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/redact_links/redactable.rb b/lib/gitlab/background_migration/redact_links/redactable.rb
deleted file mode 100644
index baab34221f1..00000000000
--- a/lib/gitlab/background_migration/redact_links/redactable.rb
+++ /dev/null
@@ -1,21 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class RedactLinks
-      module Redactable
-        extend ActiveSupport::Concern
-
-        def redact_field!(field)
-          self[field].gsub!(%r{/sent_notifications/\h{32}/unsubscribe}, '/sent_notifications/REDACTED/unsubscribe')
-
-          if self.changed?
-            self.update_columns(field => self[field],
-                                "#{field}_html" => nil)
-          end
-        end
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/rollback_import_state_data.rb b/lib/gitlab/background_migration/rollback_import_state_data.rb
deleted file mode 100644
index a7c986747d8..00000000000
--- a/lib/gitlab/background_migration/rollback_import_state_data.rb
+++ /dev/null
@@ -1,40 +0,0 @@
-# frozen_string_literal: true
-
-module Gitlab
-  module BackgroundMigration
-    # This background migration migrates all the data of import_state
-    # back to the projects table for projects that are considered imports or forks
-    class RollbackImportStateData
-      def perform(start_id, end_id)
-        move_attributes_data_to_project(start_id, end_id)
-      end
-
-      def move_attributes_data_to_project(start_id, end_id)
-        Rails.logger.info("#{self.class.name} - Moving import attributes data to projects table: #{start_id} - #{end_id}")
-
-        if Gitlab::Database.mysql?
-          ActiveRecord::Base.connection.execute <<~SQL
-            UPDATE projects, project_mirror_data
-            SET
-              projects.import_status = project_mirror_data.status,
-              projects.import_jid = project_mirror_data.jid,
-              projects.import_error = project_mirror_data.last_error
-            WHERE project_mirror_data.project_id = projects.id
-            AND project_mirror_data.id BETWEEN #{start_id} AND #{end_id}
-          SQL
-        else
-          ActiveRecord::Base.connection.execute <<~SQL
-            UPDATE projects
-            SET
-              import_status = project_mirror_data.status,
-              import_jid = project_mirror_data.jid,
-              import_error = project_mirror_data.last_error
-            FROM project_mirror_data
-            WHERE project_mirror_data.project_id = projects.id
-            AND project_mirror_data.id BETWEEN #{start_id} AND #{end_id}
-          SQL
-        end
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/background_migration/schedule_diff_files_deletion.rb b/lib/gitlab/background_migration/schedule_diff_files_deletion.rb
deleted file mode 100644
index 609cf19187c..00000000000
--- a/lib/gitlab/background_migration/schedule_diff_files_deletion.rb
+++ /dev/null
@@ -1,44 +0,0 @@
-# frozen_string_literal: true
-# rubocop:disable Style/Documentation
-
-module Gitlab
-  module BackgroundMigration
-    class ScheduleDiffFilesDeletion
-      class MergeRequestDiff < ActiveRecord::Base
-        self.table_name = 'merge_request_diffs'
-
-        belongs_to :merge_request
-
-        include EachBatch
-      end
-
-      DIFF_BATCH_SIZE = 5_000
-      INTERVAL = 5.minutes
-      MIGRATION = 'DeleteDiffFiles'
-
-      def perform
-        diffs = MergeRequestDiff
-          .from("(#{diffs_collection.to_sql}) merge_request_diffs")
-          .where('merge_request_diffs.id != merge_request_diffs.latest_merge_request_diff_id')
-          .select(:id)
-
-        diffs.each_batch(of: DIFF_BATCH_SIZE) do |relation, index|
-          ids = relation.pluck(:id)
-
-          BackgroundMigrationWorker.perform_in(index * INTERVAL, MIGRATION, [ids])
-        end
-      end
-
-      private
-
-      def diffs_collection
-        MergeRequestDiff
-          .joins(:merge_request)
-          .where("merge_requests.state = 'merged'")
-          .where('merge_requests.latest_merge_request_diff_id IS NOT NULL')
-          .where("merge_request_diffs.state NOT IN ('without_files', 'empty')")
-          .select('merge_requests.latest_merge_request_diff_id, merge_request_diffs.id')
-      end
-    end
-  end
-end
diff --git a/lib/gitlab/import_export/config.rb b/lib/gitlab/import_export/config.rb
new file mode 100644
index 00000000000..f6cd4eb5e0c
--- /dev/null
+++ b/lib/gitlab/import_export/config.rb
@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+module Gitlab
+  module ImportExport
+    class Config
+      # Returns a Hash of the YAML file, including EE specific data if EE is
+      # used.
+      def to_h
+        hash = parse_yaml
+        ee_hash = hash['ee']
+
+        if merge? && ee_hash
+          ee_hash.each do |key, value|
+            if key == 'project_tree'
+              merge_project_tree(value, hash[key])
+            else
+              merge_attributes_list(value, hash[key])
+            end
+          end
+        end
+
+        # We don't want to expose this section after this point, as it is no
+        # longer needed.
+        hash.delete('ee')
+
+        hash
+      end
+
+      # Merges a project relationships tree into the target tree.
+      #
+      # @param [Array<Hash|Symbol>] source_values
+      # @param [Array<Hash|Symbol>] target_values
+      def merge_project_tree(source_values, target_values)
+        source_values.each do |value|
+          if value.is_a?(Hash)
+            # Examples:
+            #
+            # { 'project_tree' => [{ 'labels' => [...] }] }
+            # { 'notes' => [:author, { 'events' => [:push_event_payload] }] }
+            value.each do |key, val|
+              target = target_values
+                .find { |h| h.is_a?(Hash) && h[key] }
+
+              if target
+                merge_project_tree(val, target[key])
+              else
+                target_values << { key => val.dup }
+              end
+            end
+          else
+            # Example: :priorities, :author, etc
+            target_values << value
+          end
+        end
+      end
+
+      # Merges a Hash containing a flat list of attributes, such as the entries
+      # in a `excluded_attributes` section.
+      #
+      # @param [Hash] source_values
+      # @param [Hash] target_values
+      def merge_attributes_list(source_values, target_values)
+        source_values.each do |key, values|
+          target_values[key] ||= []
+          target_values[key].concat(values)
+        end
+      end
+
+      def merge?
+        Gitlab.ee?
+      end
+
+      def parse_yaml
+        YAML.load_file(Gitlab::ImportExport.config_file)
+      end
+    end
+  end
+end
diff --git a/lib/gitlab/import_export/import_export.yml b/lib/gitlab/import_export/import_export.yml
index 71c44af9254..a0fb051e806 100644
--- a/lib/gitlab/import_export/import_export.yml
+++ b/lib/gitlab/import_export/import_export.yml
@@ -1,7 +1,11 @@
 # Model relationships to be included in the project import/export
+#
+# This list _must_ only contain relationships that are available to both CE and
+# EE. EE specific relationships must be defined in the `ee` section further
+# down below.
 project_tree:
   - labels:
-      :priorities
+    - :priorities
   - milestones:
     - events:
       - :push_event_payload
@@ -15,18 +19,18 @@ project_tree:
         - :push_event_payload
     - label_links:
       - label:
-          :priorities
+        - :priorities
     - milestone:
       - events:
         - :push_event_payload
     - resource_label_events:
       - label:
-          :priorities
+        - :priorities
     - :issue_assignees
   - snippets:
     - :award_emoji
     - notes:
-        :author
+      - :author
   - releases:
     - :links
   - project_members:
@@ -46,13 +50,13 @@ project_tree:
     - :timelogs
     - label_links:
       - label:
-          :priorities
+        - :priorities
     - milestone:
       - events:
         - :push_event_payload
     - resource_label_events:
       - label:
-          :priorities
+        - :priorities
   - ci_pipelines:
     - notes:
       - :author
@@ -121,12 +125,22 @@ excluded_attributes:
     - :bfg_object_map
     - :detected_repository_languages
     - :tag_list
+    - :mirror_user_id
+    - :mirror_trigger_builds
+    - :only_mirror_protected_branches
+    - :pull_mirror_available_overridden
+    - :mirror_overwrites_diverged_branches
+    - :packages_enabled
+    - :mirror_last_update_at
+    - :mirror_last_successful_update_at
   namespaces:
     - :runners_token
     - :runners_token_encrypted
   project_import_state:
     - :last_error
     - :jid
+    - :last_update_at
+    - :last_successful_update_at
   prometheus_metrics:
     - :common
     - :identifier
@@ -201,3 +215,12 @@ methods:
     - :action
   project_badges:
     - :type
+
+# EE specific relationships and settings to include. All of this will be merged
+# into the previous structures if EE is used.
+ee:
+  project_tree:
+    - protected_branches:
+      - :unprotect_access_levels
+    - protected_environments:
+      - :deploy_access_levels
diff --git a/lib/gitlab/import_export/reader.rb b/lib/gitlab/import_export/reader.rb
index bc0d18e03fa..8bdf6ca491d 100644
--- a/lib/gitlab/import_export/reader.rb
+++ b/lib/gitlab/import_export/reader.rb
@@ -7,7 +7,7 @@ module Gitlab
 
       def initialize(shared:)
         @shared = shared
-        config_hash = YAML.load_file(Gitlab::ImportExport.config_file).deep_symbolize_keys
+        config_hash = ImportExport::Config.new.to_h.deep_symbolize_keys
         @tree = config_hash[:project_tree]
         @attributes_finder = Gitlab::ImportExport::AttributesFinder.new(included_attributes: config_hash[:included_attributes],
                                                                         excluded_attributes: config_hash[:excluded_attributes],
diff --git a/lib/tasks/gitlab/import_export.rake b/lib/tasks/gitlab/import_export.rake
index 900dbf7be24..5365bd3920f 100644
--- a/lib/tasks/gitlab/import_export.rake
+++ b/lib/tasks/gitlab/import_export.rake
@@ -7,7 +7,7 @@ namespace :gitlab do
 
     desc "GitLab | Display exported DB structure"
     task data: :environment do
-      puts YAML.load_file(Gitlab::ImportExport.config_file)['project_tree'].to_yaml(SortKeys: true)
+      puts Gitlab::ImportExport::Config.new.to_h['project_tree'].to_yaml(SortKeys: true)
     end
 
     desc 'GitLab | Bumps the Import/Export version in fixtures and project templates'