Memoize GitLab logger to reduce open file descriptorssh-memoize-logger

We see that in gitlab-org/gitlab-ee#3664 that if we log a lot of
data in Sidekiq workers, the number of open file descriptors
reaches over 1000. To avoid this, we can memoize the logger per
thread via RequestStore.

Closes gitlab-org/gitlab-ee#3664

2 files changed, 11 insertions, 2 deletions

diff --git a/changelogs/unreleased/sh-memoize-logger.yml b/changelogs/unreleased/sh-memoize-logger.yml
new file mode 100644
index 00000000000..1b6567ce72f
--- /dev/null
+++ b/changelogs/unreleased/sh-memoize-logger.yml
@@ -0,0 +1,5 @@
+---
+title: Memoize GitLab logger to reduce open file descriptors
+merge_request:
+author:
+type: fixed
diff --git a/lib/gitlab/logger.rb b/lib/gitlab/logger.rb
index 6bffd410ed0..dd88da73e91 100644
--- a/lib/gitlab/logger.rb
+++ b/lib/gitlab/logger.rb
@@ -13,7 +13,7 @@ module Gitlab
     end
 
     def self.read_latest
-      path = Rails.root.join("log", file_name)
+      path = self.full_log_path
 
       return [] unless File.readable?(path)
 
@@ -22,7 +22,11 @@ module Gitlab
     end
 
     def self.build
-      new(Rails.root.join("log", file_name))
+      RequestStore[self.full_log_path] ||= new(self.full_log_path)
+    end
+
+    def self.full_log_path
+      Rails.root.join("log", file_name)
     end
   end
 end