summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDouwe Maan <douwe@gitlab.com>2016-08-03 19:21:56 +0000
committerDouwe Maan <douwe@gitlab.com>2016-08-03 19:21:56 +0000
commit6a9283600cd4a11b97fe26772e68095d8dc854bd (patch)
treecdaff320c3dbcbd14dab3d627119056155fbbebb
parentf7c360cad6973331b046140acf799427463c0857 (diff)
parent08c1dd348273df67bf14172e9082308e12f94784 (diff)
downloadgitlab-ce-6a9283600cd4a11b97fe26772e68095d8dc854bd.tar.gz
Merge branch 'fix/speed-up-commit-repo-changes' into 'master'
Speed up methods that depend on diff stats only ## What does this MR do? Speeds up and reduces memory usage of `Commit#repo_changes`loop, `Repository#expire_avatar_cache` and `IrkerWorker`. It's a WIP because it need a new version of `gitlab_git` released with [these changes](https://gitlab.com/gitlab-org/gitlab_git/merge_requests/109). ## Are there points in the code the reviewer needs to double check? N/A ## Why was this MR needed? For large files committed, this method ends up loading them in memory which can consume lots of memory. ## What are the relevant issue numbers? https://gitlab.com/gitlab-org/gitlab-ce/issues/19441#note_13425892 ## Screenshots (if relevant) N/A ## Does this MR meet the acceptance criteria? - [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added - [ ] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~ - [ ] ~~API support added~~ - ~~Tests~~ - [ ] ~~Added for this feature/bug~~ - [ ] ~~All builds are passing~~ - [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides) - [x] Branch has no merge conflicts with `master` (if you do - rebase it please) - [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits) See merge request !5568
-rw-r--r--CHANGELOG1
-rw-r--r--Gemfile2
-rw-r--r--Gemfile.lock4
-rw-r--r--app/models/commit.rb2
-rw-r--r--app/models/repository.rb2
-rw-r--r--app/workers/irker_worker.rb2
6 files changed, 7 insertions, 6 deletions
diff --git a/CHANGELOG b/CHANGELOG
index 38e91fc3e98..25911e02ec6 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -57,6 +57,7 @@ v 8.11.0 (unreleased)
- Sensible state specific default sort order for issues and merge requests !5453 (tomb0y)
- Fix RequestProfiler::Middleware error when code is reloaded in development
- Catch what warden might throw when profiling requests to re-throw it
+ - Speed up and reduce memory usage of Commit#repo_changes, Repository#expire_avatar_cache and IrkerWorker
v 8.10.3
- Fix Import/Export issue importing milestones and labels not associated properly. !5426
diff --git a/Gemfile b/Gemfile
index 5f247abd2fc..16f24553ed1 100644
--- a/Gemfile
+++ b/Gemfile
@@ -53,7 +53,7 @@ gem 'browser', '~> 2.2'
# Extracting information from a git repository
# Provide access to Gitlab::Git library
-gem 'gitlab_git', '~> 10.4.2'
+gem 'gitlab_git', '~> 10.4.3'
# LDAP Auth
# GitLab fork with several improvements to original library. For full list of changes
diff --git a/Gemfile.lock b/Gemfile.lock
index 7b4175ea824..866f5014847 100644
--- a/Gemfile.lock
+++ b/Gemfile.lock
@@ -278,7 +278,7 @@ GEM
diff-lcs (~> 1.1)
mime-types (>= 1.16, < 3)
posix-spawn (~> 0.3)
- gitlab_git (10.4.2)
+ gitlab_git (10.4.3)
activesupport (~> 4.0)
charlock_holmes (~> 0.7.3)
github-linguist (~> 4.7.0)
@@ -870,7 +870,7 @@ DEPENDENCIES
github-linguist (~> 4.7.0)
github-markup (~> 1.4)
gitlab-flowdock-git-hook (~> 1.0.1)
- gitlab_git (~> 10.4.2)
+ gitlab_git (~> 10.4.3)
gitlab_meta (= 7.0)
gitlab_omniauth-ldap (~> 1.2.1)
gollum-lib (~> 4.2)
diff --git a/app/models/commit.rb b/app/models/commit.rb
index d58c2fb8106..cc413448ce8 100644
--- a/app/models/commit.rb
+++ b/app/models/commit.rb
@@ -334,7 +334,7 @@ class Commit
def repo_changes
changes = { added: [], modified: [], removed: [] }
- raw_diffs.each do |diff|
+ raw_diffs(deltas_only: true).each do |diff|
if diff.deleted_file
changes[:removed] << diff.old_path
elsif diff.renamed_file || diff.new_file
diff --git a/app/models/repository.rb b/app/models/repository.rb
index 3d95344a68f..c1170c470ea 100644
--- a/app/models/repository.rb
+++ b/app/models/repository.rb
@@ -372,7 +372,7 @@ class Repository
# We don't want to flush the cache if the commit didn't actually make any
# changes to any of the possible avatar files.
if revision && commit = self.commit(revision)
- return unless commit.raw_diffs.
+ return unless commit.raw_diffs(deltas_only: true).
any? { |diff| AVATAR_FILES.include?(diff.new_path) }
end
diff --git a/app/workers/irker_worker.rb b/app/workers/irker_worker.rb
index 07cc7c1cbd7..19f38358eb5 100644
--- a/app/workers/irker_worker.rb
+++ b/app/workers/irker_worker.rb
@@ -141,7 +141,7 @@ class IrkerWorker
end
def files_count(commit)
- diffs = commit.raw_diffs
+ diffs = commit.raw_diffs(deltas_only: true)
files = "#{diffs.real_size} file"
files += 's' if diffs.size > 1