Move Sidekiq exporter logs to log/sidekiq_exporter.logsh-move-sidekiq-exporter-logs

The Sidekiq exporter logs were mixing with the normal Sidekiq logs. In order
to support structured logging in Sidekiq, we either need to split this data
out or convert the exporter to produce structured logs. Since Sidekiq job
processing is fundamentally different information from Web server traffic,
it seems cleaner to move the metrics traffic into a separate file, where they
can be parsed by a different filter if needed.

Relates to #20060

4 files changed, 25 insertions, 2 deletions

diff --git a/changelogs/unreleased/sh-move-sidekiq-exporter-logs.yml b/changelogs/unreleased/sh-move-sidekiq-exporter-logs.yml
new file mode 100644
index 00000000000..1990f4f6124
--- /dev/null
+++ b/changelogs/unreleased/sh-move-sidekiq-exporter-logs.yml
@@ -0,0 +1,5 @@
+---
+title: Move Sidekiq exporter logs to log/sidekiq_exporter.log
+merge_request:
+author:
+type: other
diff --git a/doc/administration/logs.md b/doc/administration/logs.md
index 00a2f3d01b8..cd107a5b39c 100644
--- a/doc/administration/logs.md
+++ b/doc/administration/logs.md
@@ -206,4 +206,12 @@ is populated whenever `gitlab-ctl reconfigure` is run manually or as part of an
 Reconfigure logs files are named according to the UNIX timestamp of when the reconfigure
 was initiated, such as `1509705644.log`
 
+## `sidekiq_exporter.log`
+
+If Prometheus metrics and the Sidekiq Exporter are both enabled, Sidekiq will
+start a Web server and listen to the defined port (default: 3807). Access logs
+will be generated in `/var/log/gitlab/gitlab-rails/sidekiq_exporter.log` for
+Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for
+installations from source.
+
 [repocheck]: repository_checks.md
diff --git a/lib/gitlab/metrics/sidekiq_metrics_exporter.rb b/lib/gitlab/metrics/sidekiq_metrics_exporter.rb
index db8bdde74b2..47b4af5d649 100644
--- a/lib/gitlab/metrics/sidekiq_metrics_exporter.rb
+++ b/lib/gitlab/metrics/sidekiq_metrics_exporter.rb
@@ -4,6 +4,8 @@ require 'prometheus/client/rack/exporter'
 module Gitlab
   module Metrics
     class SidekiqMetricsExporter < Daemon
+      LOG_FILENAME = File.join(Rails.root, 'log', 'sidekiq_exporter.log')
+
       def enabled?
         Gitlab::Metrics.metrics_folder_present? && settings.enabled
       end
@@ -17,7 +19,13 @@ module Gitlab
       attr_reader :server
 
       def start_working
-        @server = ::WEBrick::HTTPServer.new(Port: settings.port, BindAddress: settings.address)
+        logger = WEBrick::Log.new(LOG_FILENAME)
+        access_log = [
+          [logger, WEBrick::AccessLog::COMBINED_LOG_FORMAT]
+        ]
+
+        @server = ::WEBrick::HTTPServer.new(Port: settings.port, BindAddress: settings.address,
+                                            Logger: logger, AccessLog: access_log)
         server.mount "/", Rack::Handler::WEBrick, rack_app
         server.start
       end
diff --git a/spec/lib/gitlab/metrics/sidekiq_metrics_exporter_spec.rb b/spec/lib/gitlab/metrics/sidekiq_metrics_exporter_spec.rb
index 6721e02fb85..61eb059a731 100644
--- a/spec/lib/gitlab/metrics/sidekiq_metrics_exporter_spec.rb
+++ b/spec/lib/gitlab/metrics/sidekiq_metrics_exporter_spec.rb
@@ -38,7 +38,9 @@ describe Gitlab::Metrics::SidekiqMetricsExporter do
 
             expect(::WEBrick::HTTPServer).to have_received(:new).with(
               Port: port,
-              BindAddress: address
+              BindAddress: address,
+              Logger: anything,
+              AccessLog: anything
             )
           end
         end