Auto merge of #5806 - bundler:seg-dont-hit-remote-twice, r=indirect

Avoid hitting RubyGems remotes when all missing gems are in --without groups

### What was the end-user problem that led to this PR?

The problem was we would attempt to fetch dependency info from a RubyGems remote even when it was only needed for gems whose groups were excluded.

### Was was your diagnosis of the problem?

My diagnosis was that the test we had for this case was insufficient, since it checked only that `stderr` was empty, and since Bundler 1 prints errors to `stdout`... it went undetected for a year.

### What is your fix for the problem, implemented in this PR?

My fix is to only report that deps are missing when they are either requested or are path/git/gemspec-sourced.

### Why did you choose this fix out of the possible options?

I chose this fix because it keeps the behavior of always ensuring all git repos are checked out.

3 files changed, 21 insertions, 7 deletions

diff --git a/lib/bundler/definition.rb b/lib/bundler/definition.rb
index 7157c46b98..6d16acd495 100644
--- a/lib/bundler/definition.rb
+++ b/lib/bundler/definition.rb
@@ -167,8 +167,7 @@ module Bundler
                              "to a different version of #{locked_gem} that hasn't been removed in order to install."
         end
         unless specs["bundler"].any?
-          local = Bundler.settings[:frozen] ? rubygems_index : index
-          bundler = local.search(Gem::Dependency.new("bundler", VERSION)).last
+          bundler = rubygems_index.search(Gem::Dependency.new("bundler", VERSION)).last
           specs["bundler"] = bundler if bundler
         end
 
@@ -194,9 +193,9 @@ module Bundler
       missing
     end
 
-    def missing_dependencies?
+    def missing_specs?
       missing = []
-      resolve.materialize(current_dependencies, missing)
+      resolve.materialize(requested_dependencies, missing)
       return false if missing.empty?
       Bundler.ui.debug "The definition is missing #{missing.map(&:full_name)}"
       true
@@ -591,6 +590,9 @@ module Bundler
 
       # order here matters, since Index#== is checking source.specs.include?(locked_index)
       locked_index != source.specs
+    rescue PathError, GitError => e
+      Bundler.ui.debug "Assuming that #{source} has not changed since fetching its specs errored (#{e})"
+      false
     end
 
     # Get all locals and override their matching sources.
@@ -768,7 +770,19 @@ module Bundler
 
         # Path sources have special logic
         if s.source.instance_of?(Source::Path) || s.source.instance_of?(Source::Gemspec)
-          other = s.source.specs[s].first
+          other_sources_specs = begin
+            s.source.specs
+          rescue PathError, GitError
+            # if we won't need the source (according to the lockfile),
+            # don't error if the path/git source isn't available
+            next if @locked_specs.
+                    for(requested_dependencies, [], false, true, false).
+                    none? {|locked_spec| locked_spec.source == s.source }
+
+            raise
+          end
+
+          other = other_sources_specs[s].first
 
           # If the spec is no longer in the path source, unlock it. This
           # commonly happens if the version changed in the gemspec
diff --git a/lib/bundler/installer.rb b/lib/bundler/installer.rb
index c47012973b..2085f65db6 100644
--- a/lib/bundler/installer.rb
+++ b/lib/bundler/installer.rb
@@ -248,7 +248,7 @@ module Bundler
 
     def resolve_if_needed(options)
       if !options["update"] && !options[:inline] && !options["force"] && Bundler.default_lockfile.file?
-        return if @definition.nothing_changed? && !@definition.missing_dependencies?
+        return if @definition.nothing_changed? && !@definition.missing_specs?
       end
 
       options["local"] ? @definition.resolve_with_cache! : @definition.resolve_remotely!
diff --git a/lib/bundler/spec_set.rb b/lib/bundler/spec_set.rb
index 9642633578..e68da87704 100644
--- a/lib/bundler/spec_set.rb
+++ b/lib/bundler/spec_set.rb
@@ -76,7 +76,7 @@ module Bundler
     end
 
     def materialize(deps, missing_specs = nil)
-      materialized = self.for(deps, [], false, true, missing_specs).to_a
+      materialized = self.for(deps, [], false, true, !missing_specs).to_a
       deps = materialized.map(&:name).uniq
       materialized.map! do |s|
         next s unless s.is_a?(LazySpecification)