diff options
author | Douwe Maan <douwe@gitlab.com> | 2016-08-03 19:21:56 +0000 |
---|---|---|
committer | Douwe Maan <douwe@gitlab.com> | 2016-08-03 19:21:56 +0000 |
commit | 6a9283600cd4a11b97fe26772e68095d8dc854bd (patch) | |
tree | cdaff320c3dbcbd14dab3d627119056155fbbebb | |
parent | f7c360cad6973331b046140acf799427463c0857 (diff) | |
parent | 08c1dd348273df67bf14172e9082308e12f94784 (diff) | |
download | gitlab-ce-6a9283600cd4a11b97fe26772e68095d8dc854bd.tar.gz |
Merge branch 'fix/speed-up-commit-repo-changes' into 'master'
Speed up methods that depend on diff stats only
## What does this MR do?
Speeds up and reduces memory usage of `Commit#repo_changes`loop, `Repository#expire_avatar_cache` and `IrkerWorker`. It's a WIP because it need a new version of `gitlab_git` released with [these changes](https://gitlab.com/gitlab-org/gitlab_git/merge_requests/109).
## Are there points in the code the reviewer needs to double check?
N/A
## Why was this MR needed?
For large files committed, this method ends up loading them in memory which can consume lots of memory.
## What are the relevant issue numbers?
https://gitlab.com/gitlab-org/gitlab-ce/issues/19441#note_13425892
## Screenshots (if relevant)
N/A
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [ ] ~~API support added~~
- ~~Tests~~
- [ ] ~~Added for this feature/bug~~
- [ ] ~~All builds are passing~~
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !5568
-rw-r--r-- | CHANGELOG | 1 | ||||
-rw-r--r-- | Gemfile | 2 | ||||
-rw-r--r-- | Gemfile.lock | 4 | ||||
-rw-r--r-- | app/models/commit.rb | 2 | ||||
-rw-r--r-- | app/models/repository.rb | 2 | ||||
-rw-r--r-- | app/workers/irker_worker.rb | 2 |
6 files changed, 7 insertions, 6 deletions
diff --git a/CHANGELOG b/CHANGELOG index 38e91fc3e98..25911e02ec6 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -57,6 +57,7 @@ v 8.11.0 (unreleased) - Sensible state specific default sort order for issues and merge requests !5453 (tomb0y) - Fix RequestProfiler::Middleware error when code is reloaded in development - Catch what warden might throw when profiling requests to re-throw it + - Speed up and reduce memory usage of Commit#repo_changes, Repository#expire_avatar_cache and IrkerWorker v 8.10.3 - Fix Import/Export issue importing milestones and labels not associated properly. !5426 @@ -53,7 +53,7 @@ gem 'browser', '~> 2.2' # Extracting information from a git repository # Provide access to Gitlab::Git library -gem 'gitlab_git', '~> 10.4.2' +gem 'gitlab_git', '~> 10.4.3' # LDAP Auth # GitLab fork with several improvements to original library. For full list of changes diff --git a/Gemfile.lock b/Gemfile.lock index 7b4175ea824..866f5014847 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -278,7 +278,7 @@ GEM diff-lcs (~> 1.1) mime-types (>= 1.16, < 3) posix-spawn (~> 0.3) - gitlab_git (10.4.2) + gitlab_git (10.4.3) activesupport (~> 4.0) charlock_holmes (~> 0.7.3) github-linguist (~> 4.7.0) @@ -870,7 +870,7 @@ DEPENDENCIES github-linguist (~> 4.7.0) github-markup (~> 1.4) gitlab-flowdock-git-hook (~> 1.0.1) - gitlab_git (~> 10.4.2) + gitlab_git (~> 10.4.3) gitlab_meta (= 7.0) gitlab_omniauth-ldap (~> 1.2.1) gollum-lib (~> 4.2) diff --git a/app/models/commit.rb b/app/models/commit.rb index d58c2fb8106..cc413448ce8 100644 --- a/app/models/commit.rb +++ b/app/models/commit.rb @@ -334,7 +334,7 @@ class Commit def repo_changes changes = { added: [], modified: [], removed: [] } - raw_diffs.each do |diff| + raw_diffs(deltas_only: true).each do |diff| if diff.deleted_file changes[:removed] << diff.old_path elsif diff.renamed_file || diff.new_file diff --git a/app/models/repository.rb b/app/models/repository.rb index 3d95344a68f..c1170c470ea 100644 --- a/app/models/repository.rb +++ b/app/models/repository.rb @@ -372,7 +372,7 @@ class Repository # We don't want to flush the cache if the commit didn't actually make any # changes to any of the possible avatar files. if revision && commit = self.commit(revision) - return unless commit.raw_diffs. + return unless commit.raw_diffs(deltas_only: true). any? { |diff| AVATAR_FILES.include?(diff.new_path) } end diff --git a/app/workers/irker_worker.rb b/app/workers/irker_worker.rb index 07cc7c1cbd7..19f38358eb5 100644 --- a/app/workers/irker_worker.rb +++ b/app/workers/irker_worker.rb @@ -141,7 +141,7 @@ class IrkerWorker end def files_count(commit) - diffs = commit.raw_diffs + diffs = commit.raw_diffs(deltas_only: true) files = "#{diffs.real_size} file" files += 's' if diffs.size > 1 |