Fix deadlock on MySQL when updating MRsfix-deadlock-updating-mrs

We haven't seen a deadlock on Postgres, but we do see a lock held for a long
time. The problem is that the update to the row in `merge_requests` is part of
the transaction that creates the pipeline, and so it's locked for all that time.

This in turn means that commenting, or updating the MR's commits and diffs, will
take longer, as they may have to wait for the pipeline service to complete.

Making it a separate statement solves this problem as the update to the
`merge_requests` table is fast.

2 files changed, 15 insertions, 5 deletions

diff --git a/app/services/ci/create_pipeline_service.rb b/app/services/ci/create_pipeline_service.rb
index 4f35255fb53..3e8f34fbb1c 100644
--- a/app/services/ci/create_pipeline_service.rb
+++ b/app/services/ci/create_pipeline_service.rb
@@ -47,14 +47,20 @@ module Ci
         return error('No stages / jobs for this pipeline.')
       end
 
-      Ci::Pipeline.transaction do
-        update_merge_requests_head_pipeline if pipeline.save
+      begin
+        Ci::Pipeline.transaction do
+          pipeline.save!
 
-        Ci::CreatePipelineStagesService
-          .new(project, current_user)
-          .execute(pipeline)
+          Ci::CreatePipelineStagesService
+            .new(project, current_user)
+            .execute(pipeline)
+        end
+      rescue ActiveRecord::RecordInvalid => e
+        return error("Failed to persist the pipeline: #{e}")
       end
 
+      update_merge_requests_head_pipeline
+
       cancel_pending_pipelines if project.auto_cancel_pending_pipelines?
 
       pipeline_created_counter.increment(source: source)
diff --git a/changelogs/unreleased/fix-deadlock-updating-mrs.yml b/changelogs/unreleased/fix-deadlock-updating-mrs.yml
new file mode 100644
index 00000000000..67119f46313
--- /dev/null
+++ b/changelogs/unreleased/fix-deadlock-updating-mrs.yml
@@ -0,0 +1,4 @@
+---
+title: Prevent deadlock when pushing to an existing merge request with CI enabled
+merge_request:
+author: