diff options
| author | Rémy Coutable <remy@rymai.me> | 2016-08-01 13:18:39 +0000 |
|---|---|---|
| committer | Rémy Coutable <remy@rymai.me> | 2016-08-01 15:28:53 +0200 |
| commit | a4dbb3efdc60682e42cf085f28c993bccbb47310 (patch) | |
| tree | 24e3c687b9c38c8109b58b6347d265ff7e3e7f12 | |
| parent | 3fbe2a5ea5e6974deded8853ba5be92cea0e92a6 (diff) | |
| download | gitlab-ce-a4dbb3efdc60682e42cf085f28c993bccbb47310.tar.gz | |
Merge branch 'fix/importing-io-timing-issue' into 'master'
Fix timing problems running imports on production
Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151
I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).
See merge request !5523
Signed-off-by: Rémy Coutable <remy@rymai.me>
| -rw-r--r-- | CHANGELOG | 1 | ||||
| -rw-r--r-- | lib/gitlab/import_export/file_importer.rb | 18 |
2 files changed, 18 insertions, 1 deletions
diff --git a/CHANGELOG b/CHANGELOG index 43a3fb94d21..8285acc8c9d 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -2,6 +2,7 @@ Please view this file on the master branch, on stable branches it's out of date. v 8.10.3 (unreleased) - Fix Import/Export issue importing milestones and labels not associated properly. !5426 + - Fix timing problems running imports on production. !5523 v 8.10.2 - User can now search branches by name. !5144 diff --git a/lib/gitlab/import_export/file_importer.rb b/lib/gitlab/import_export/file_importer.rb index 82d1e1805c5..eca6e5b6d51 100644 --- a/lib/gitlab/import_export/file_importer.rb +++ b/lib/gitlab/import_export/file_importer.rb @@ -3,6 +3,8 @@ module Gitlab class FileImporter include Gitlab::ImportExport::CommandLineUtil + MAX_RETRIES = 8 + def self.import(*args) new(*args).import end @@ -14,7 +16,10 @@ module Gitlab def import FileUtils.mkdir_p(@shared.export_path) - decompress_archive + + wait_for_archived_file do + decompress_archive + end rescue => e @shared.error(e) false @@ -22,6 +27,17 @@ module Gitlab private + # Exponentially sleep until I/O finishes copying the file + def wait_for_archived_file + MAX_RETRIES.times do |retry_number| + break if File.exist?(@archive_file) + + sleep(2**retry_number) + end + + yield + end + def decompress_archive result = untar_zxf(archive: @archive_file, dir: @shared.export_path) |