Auto merge of #4607 - will-in-wi:clean_up_workers, r=segiddins

Clean up worker threads once done with them

Right now, the thread pools created by CompactIndex are not cleaned up once they are done. I assume that over time, they would be garbage collected, but in the meantime there could be 200+ threads running. Many shared hosts have fork bomb protection set up which kills Bundler.

This patch will clean up the threads as soon as they are done, keeping the total number of active threads at any one time to a minimum.

Fixes https://github.com/bundler/bundler/issues/4367

2 files changed, 25 insertions, 1 deletions

diff --git a/lib/bundler/fetcher/compact_index.rb b/lib/bundler/fetcher/compact_index.rb
index c893955dfa..4c2c0f49da 100644
--- a/lib/bundler/fetcher/compact_index.rb
+++ b/lib/bundler/fetcher/compact_index.rb
@@ -89,7 +89,7 @@ module Bundler
               worker_name = "Compact Index (#{display_uri.host})"
               worker = Bundler::Worker.new(25, worker_name, func)
               inputs.each {|input| worker.enq(input) }
-              inputs.map { worker.deq }
+              inputs.map { worker.deq }.tap { worker.stop }
             end
           end
         end
diff --git a/spec/bundler/fetcher/compact_index_spec.rb b/spec/bundler/fetcher/compact_index_spec.rb
new file mode 100644
index 0000000000..e111d8a3b6
--- /dev/null
+++ b/spec/bundler/fetcher/compact_index_spec.rb
@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+require "spec_helper"
+
+describe Bundler::Fetcher::CompactIndex do
+  let(:downloader)  { double(:downloader) }
+  let(:remote)      { double(:remote, :cache_slug => "lsjdf") }
+  let(:display_uri) { URI("http://sampleuri.com") }
+  let(:compact_index) { described_class.new(downloader, remote, display_uri) }
+
+  describe "#specs_for_names" do
+    it "has only one thread open at the end of the run" do
+      compact_index.specs_for_names(["lskdjf"])
+
+      thread_count = Thread.list.select {|thread| thread.status == "run" }.count
+      expect(thread_count).to eq 1
+    end
+
+    it "calls worker#stop during the run" do
+      expect_any_instance_of(Bundler::Worker).to receive(:stop).at_least(:once)
+
+      compact_index.specs_for_names(["lskdjf"])
+    end
+  end
+end