diff options
| author | Sean McGivern <sean@gitlab.com> | 2017-10-27 14:20:55 +0100 |
|---|---|---|
| committer | Sean McGivern <sean@gitlab.com> | 2017-10-27 14:20:55 +0100 |
| commit | ba9b4c4de86aa816e5ddc7a9cde9193c43835223 (patch) | |
| tree | 07523b0355844413bce1c480cd4f1a3b7291c2a8 | |
| parent | e9d352612371424592376b81d0a6c5ca234b9fb5 (diff) | |
| download | gitlab-ce-39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.tar.gz | |
Avoid hitting statement timeout finding MR pipelines39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout
For MRs with many thousands of commits, `SELECT DISTINCT(sha)` will be very
slow.
What we can't do to fix this:
1. Add an index. Postgres won't use it for DISTINCT without a lot of ceremony.
2. Do the `uniq` in Ruby. That can still be very slow with hundreds of
thousands of commits.
3. Use a subquery. We haven't removed the `st_commits` column yet, but we will
soon.
Until 3 is available to us, we can just do 2, but also add a limit clause. There
is no ordering, so this may return different results, but our goal with these
MRs is just to get them to load, so it's not a huge deal.
| -rw-r--r-- | app/models/merge_request.rb | 2 | ||||
| -rw-r--r-- | changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml | 5 |
2 files changed, 6 insertions, 1 deletions
diff --git a/app/models/merge_request.rb b/app/models/merge_request.rb index c3fae16d109..b0a0c753c09 100644 --- a/app/models/merge_request.rb +++ b/app/models/merge_request.rb @@ -874,7 +874,7 @@ class MergeRequest < ActiveRecord::Base # def all_commit_shas if persisted? - column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).pluck('DISTINCT(sha)') + column_shas = MergeRequestDiffCommit.where(merge_request_diff: merge_request_diffs).limit(10_000).pluck('sha') serialised_shas = merge_request_diffs.where.not(st_commits: nil).flat_map(&:commit_shas) (column_shas + serialised_shas).uniq diff --git a/changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml b/changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml new file mode 100644 index 00000000000..47bf30ecb5a --- /dev/null +++ b/changelogs/unreleased/39054-activerecord-statementinvalid-pg-querycanceled-error-canceling-statement-due-to-statement-timeout.yml @@ -0,0 +1,5 @@ +--- +title: Stop merge requests with thousands of commits from timing out +merge_request: 15063 +author: +type: performance |