Add '--unsafe_absolute_includes' option so that more can be compiled in

pump-mode.

An occurrence of say

  #include "/usr/include/acl.h"

will normally force the include processor to abort (even when this include is
platform-specific and #ifdef'ed away), because there are no guarantees that the
file /usr/include/acl.h exists on the host.  With this option, these includes
will be ignored.

A message like:

   WARNING include server: absolute filepath '/usr/include/acl.h' was IGNORED (correctness of build may be affected)

will be printed.

The normal message raised when this option is off, the default, has been
modified.  It now mentions the option.

An include server manual page is to follow, which explains the consequences in
detail of using this option.

Testing: this makes the samba-2.2.7 benchmark build without hiccups using distcc
pump.  Without the change many compilations fail because the include server
bails out and because another unrelated bug in the include server makes it later
dish up with include closures that are too small.

Revievers: csilvers@google.com, fergus@google.com


git-svn-id: http://distcc.googlecode.com/svn/trunk@337 01de4be4-8c4a-0410-9132-4925637da917

1 files changed, 20 insertions, 3 deletions

diff --git a/include_server/cache_basics.py b/include_server/cache_basics.py
index 7be7507..44ea352 100755
--- a/include_server/cache_basics.py
+++ b/include_server/cache_basics.py
@@ -218,6 +218,7 @@ Debug = basics.Debug
 DEBUG_TRACE = basics.DEBUG_TRACE
 DEBUG_TRACE1 = basics.DEBUG_TRACE1
 DEBUG_TRACE2 = basics.DEBUG_TRACE2
+DEBUG_WARNING = basics.DEBUG_WARNING
 NotCoveredError = basics.NotCoveredError
 
 
@@ -496,6 +497,10 @@ class RelpathMapToIndex(MapToIndex):
   This is useful for "cheap" normalization: this invariant ensures that
   os.path.join(some-directorymap-string, some-relpathmap-string) can
   be implemented using +.
+
+  We actually do allow storing absolute paths if option
+  --unsafe_absolute_includes is in use.  But, then, we're careful in Resolve
+  (below) to bail out.
   """
 
   def Index(self, relpath):
@@ -504,9 +509,16 @@ class RelpathMapToIndex(MapToIndex):
       directory: a string not starting with /.
     """
     if os.path.isabs(relpath):
-      raise NotCoveredError("Filepath must be relative but isn't: '%s'." %
-                            relpath,
-                            send_email=False)
+      if basics.opt_unsafe_absolute_includes:
+        Debug(DEBUG_WARNING,
+              "absolute filepath '%s' was IGNORED"
+              " (correctness of build may be affected)", relpath)
+      else:
+          raise NotCoveredError("Filepath must be relative but isn't: '%s'."
+                                " Consider setting INCLUDE_SERVER_ARGS='--"
+                                "unsafe_absolute_includes'."
+                                % relpath,
+                                send_email=False)
     # Now, remove leading "./" so as not to start an infinite regression when
     # say foo.c contains:
     #
@@ -693,6 +705,11 @@ class BuildStatCache(object):
        os.getcwd() + '/' == self.directory_map.string[currdir_idx]
     """
     includepath = self.includepath_map.string[includepath_idx]
+    if includepath.startswith('/'):
+      # We really don't want to start exploring absolute includepaths; what's
+      # the sl_idx to return for example? And what about the use of '+'
+      # (as an optimization) below instead of os.path.join.
+      return (None, None)
     dir_map_string = self.directory_map.string   # memoize the fn pointer
     build_stat = self.build_stat
     real_stat = self.real_stat