diff options
| author | fergus.henderson <fergus.henderson@01de4be4-8c4a-0410-9132-4925637da917> | 2008-06-10 01:42:04 +0000 |
|---|---|---|
| committer | fergus.henderson <fergus.henderson@01de4be4-8c4a-0410-9132-4925637da917> | 2008-06-10 01:42:04 +0000 |
| commit | 6f4794969db53c35b34bbd6118794762bc0c99d1 (patch) | |
| tree | 5c9965869af64a4b9f9d1c461fedd1052f3ed6bc | |
| parent | d10341e797df5a51cbb20a1455f2adebb1977518 (diff) | |
| download | distcc-6f4794969db53c35b34bbd6118794762bc0c99d1.tar.gz | |
Updated ChangeLog file with entries from the Google revision logs.
- Started with the output of
g4 describelog devtools/distcc/distcc_pump/...
g4 describelog experimental/distcc_pump/...
g4 describelog tools/distcc/...
- Reformatted entries to a more Changelog-like style.
- Deleted references to Google-specific file names, changelist numbers,
file revision numbers, and bug numbers. In some cases, this was done
by trimming the change description to remove explanations of how the
change was tested. In many cases, "in CL <number>" was changed to
"in a previous change".
- Deleted changes that only moved files from one repository to another.
- Deleted changes that only affected files that never made it to the
open-source svn repository, e.g. the OWNERS file, or that were made
to a branch after development had moved to a different repository.
Reviewers: Craig Silverstein
git-svn-id: http://distcc.googlecode.com/svn/trunk@500 01de4be4-8c4a-0410-9132-4925637da917
| -rw-r--r-- | ChangeLog | 3982 |
1 files changed, 3974 insertions, 8 deletions
@@ -1,21 +1,3987 @@ - ## These changes were made internally to Google, and not integrated ## into the main distcc distribution until 2008. The dates here are ## the dates they were applied to the internal-Google branch, and ## interleave with the changes to the main branch, below. +2008/03/03 11:35:34 Fergus Henderson <opensource@google.com> + + Add extra "-I" option that seems to be needed when building distcc-pump + with Blaze: we need _two_ ".."'s to get from $BUILDDIR/include_server/build + to $BUILDDIR/distcc/src/config.h. + + Without this change, building distcc fails with a compilation + error about "config.h" not found when run in a fresh workspace. + + R=klarlund + DELTA=2 (2 added, 0 deleted, 0 changed) + + * include_server/setup.py: edit + + +2007/12/14 19:17:16 Fergus Henderson <opensource@google.com> + + Fix a bug which showed up if the include server was run from + a different directory than the distcc clients: in that case, + it was at some times wrongly interpreting relative paths in the + --stat_reset_trigger option as relative to the distcc client's + cwd, rather than the include server's cwd. + + TODO(fergus): add a regression test. + + R=klarlund + CC=distcc-pump + APPROVED=klarlund + DELTA=20 (18 added, 2 deleted, 0 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node_test.py: edit + + +2007/12/14 10:26:29 Nils Klarlund <opensource@google.com> + + Fix location of stamp file that record autoconf execution. + + The location was incorrect: the stamp file that pertains to autoconf + must necessarily be in the source code directory because autoconf'ing is + the process of generating the configure script. Only the result of + configuring is to be carried out in the object directory. + + R=fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * Makefile.in: edit + + +2007/12/13 21:52:59 Nils Klarlund <opensource@google.com> + + Fix typos in Makefile.in. + + R=fergus + DELTA=2 (0 added, 0 deleted, 2 changed) + + * include_server/Makefile.in: edit + + +2007/12/07 16:19:16 Nils Klarlund <opensource@google.com> + + This file is generated by run_all_autoconf.sh and should not be checked + in. (Because that leads to Perforce complaints about 'clobbering'.) + + R=fergus + DELTA=262 (0 added, 262 deleted, 0 changed) + + * distcc/src/config.h.in: delete + + +2007/12/07 12:57:55 Nils Klarlund <opensource@google.com> + + Fix bug caused by non-idempotency of 'ln' and ensuing falsely positive + test result. + + So, it turns out that 'ln -sf' is not idempotent. If C/D and E are + directories, then + + ln -sf $(readlink -f C/D) E/D + + creates a link E/D -> realpath(C/D) and subsequent + + ln -sf $(readlink -f C/D) E/D + + creates a link C/D/D -> realpath(C/D), a self-reference of sorts. This + is nasty because the second link created is in C/D, which was supposed + to be just a target directory, not something to be modified! + + This bug enabled a faulty test to pass: the directory structure created + by the non-potency bug is not cleaned on make clean. And the existence of + the inserted extra directory made resolution of a certain include path + possible when it should not have. + + R=fergus + CC=manos + DELTA=11 (9 added, 0 deleted, 2 changed) + + * include_server/Makefile.in: edit + * include_server/include_analyzer_test.py: edit + + +2007/12/03 18:37:17 Nils Klarlund <opensource@google.com> + + + This version is to incorporate: + +' Fix bug that leads to concurre' Change 2007/12/03 by Nils Klarlund <opensource@google.com> + + R=fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/12/03 16:36:06 Nils Klarlund <opensource@google.com> + + + Fix bug that leads to concurrent build interference: one distcc-pump + build will severely affect another. The logic for naming the root + directory was flawed: it used the pid of the parent, not the child, + which is where the include server resides. + + Also fix less severe bug that allowed the cleaning up to end + prematurely. + + The code is refactored at the same time -- for better readability and to + address the parent/child pid confusion. + + The fix also includes the introduction of an additional communication + mechanism, a semaphor: the child is now starting the include server and + it must tell the parent that it has done so before the parent is allowed + to exit. + + Testing: + + - 'make check' + - Made several concurrent runs and verified that interference no longer + takes place. + + R=fergus + DELTA=65 (45 added, 1 deleted, 19 changed) + + * include_server/include_server.py: edit + + +2007/11/29 12:30:45 Nils Klarlund <opensource@google.com> + + + Add more meta-dependencies (those that involve autoconf) to the + Makefile.in files and add sanity check to run_all_autoconf.sh. + + These dependencies should help prevent builds that do not reflect the + configuration changes such as to version.sh or configure.ac. + + Dependencies are added, so this should be a conservative CL: making + this build system less fragile. + + Tests: patched this CL into a clean client and rebuilt software: + + ./run_all_autoconf.sh + ./configure --prefix=$(pwd)/installation + make + make install + + R=fergus + DELTA=73 (62 added, 0 deleted, 11 changed) + + * Makefile.in: edit + * distcc/Makefile.in: edit + * include_server/Makefile.in: edit + * run_all_autoconf.sh: edit + + +2007/11/06 15:58:38 Nils Klarlund <opensource@google.com> + + Prepare 'pump19'. It contains: + +'Add basics_test rule to include' Change 2007/11/06 12:43:59 by Nils Klarlund <opensource@google.com> +' Fix typo and missing condition' Change 2007/11/06 11:51:01 by Nils Klarlund <opensource@google.com> +'Fix an infinite loop in the inc' Change 2007/11/05 14:11:47 by Nils Klarlund <opensource@google.com> +' Bump up length of interval tha' Change 2007/11/05 07:25:13 by Nils Klarlund <opensource@google.com> +' Make the _CleanOutOthers routi' Change 2007/11/02 12:36:27 by Nils Klarlund <opensource@google.com> +' Increase include server timeou' Change 2007/11/01 14:46:44 by Nils Klarlund <opensource@google.com> +'Fix buglet: when the contents o' Change 2007/11/01 14:28:22 by Nils Klarlund <opensource@google.com> + + R=fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/11/06 15:43:59 Nils Klarlund <opensource@google.com> + + Add basics_test rule to include_server/Makefile.in. This enables + 'make check' to run all tests. + + Test: + + ./run_all_autoconf.sh; ./configure --prefix=$(pwd)/installation + make basics_test + make -C include_server basics_test + make check + + R=fergus + DELTA=4 (4 added, 0 deleted, 0 changed) + + * include_server/Makefile.in: edit + + +2007/11/06 14:51:01 Nils Klarlund <opensource@google.com> + + + Fix typo and missing conditional of a previous change, which "broke" the build. + + Tests: + - make -C include_server check + + R=fergus + DELTA=3 (1 added, 0 deleted, 2 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/11/05 17:11:47 Nils Klarlund <opensource@google.com> + + Fix an infinite loop in the include server. FindNode would call itself + ad infinitum prompted by unusual link structures and includes. + + This change is non-trivial. In particular, it relates to a + previous bug fix which is rolled-back as part of the present + change -- and then the problem reported in the bug report is + solved instead by properly mending the code that was removed in + that change. The fix consists of making the memoization removed + in that change cognizant of the file directory or rather the + absolute path of the file directory, see extensive new comments + in class IncludeAnalyzerMemoizingNode. + + Small stuff: we change 'filepath' to 'fp' in all identifiers in + FindNode. That's because the identifiers were otherwise getting too long + in the new code. And, we used that abbreviation already. + + Testing: a test_DotdotInInclude test case specifically exercices the + situation in the bug. [Plus additional Google-specific correctness + and performance tests.] + + Additionally, we're now doing the test_IncludeAnalyzer in two ways: + directly and through a symlink farm. This helps us test regression + against subtle bugs that may occur with symbolic links. + + R=fergus + DELTA=327 (194 added, 12 deleted, 121 changed) + + * include_server/cache_basics.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/test_data/dfoo/include_dotdot_foo: add + * include_server/test_data/symlink_farm/abc_post.c: add + * include_server/test_data/symlink_farm/abc_pre.c: add + * include_server/test_data/symlink_farm/computed_includes.c: add + * include_server/test_data/symlink_farm/dbar: add + * include_server/test_data/symlink_farm/dfoo: add + * include_server/test_data/symlink_farm/foo: add + * include_server/test_data/symlink_farm/p1.h: add + * include_server/test_data/symlink_farm/parse.c: add + * include_server/test_data/symlink_farm/sub_farm/link_to_dotdot_dotdot_dfoo_include_dotdot_foo: add + * include_server/test_data/symlink_farm/test_directory_probing.c: add + * include_server/test_data/symlink_farm/test_include_next: add + + +2007/11/05 10:25:13 Nils Klarlund <opensource@google.com> + + + Bump up length of interval that determines when SIGALRM is + raised. Testing showed that with srcfs -- which increases user time -- + I/O between distcc and include server (in the C extension part) would be + tripped up by SIGALRM when it is raised every 2s. This results in a + Python exception such as IOError, which is wrong and even harmful, + because it terminates the include server. The interval in the currently + released version is 3s, so 4s is better. + + This is to be followed up by CL that makes the algorithmic code + "protected" by SIGALRM free of I/O. + + R=fergus + DELTA=5 (4 added, 0 deleted, 1 changed) + + * include_server/basics.py: edit + + +2007/11/02 15:36:27 Nils Klarlund <opensource@google.com> + + + Make the _CleanOutOthers routine be called. Before this CL, this routine + was not called at all, because a variable describing the tmp directory + has not been initialized. We split the InitializeTmpDirectories function + into two parts to distinguish between creation of the temporary + directory and the creation of client root under it. + + Also, fixed bug in the glob expression. + + Also, don't write out warning when cleaning -- that's just bound to + confuse users. Demote message to TRACE level. + + Also, fix basics_test.py and make sure it's called as part of + 'make check'. + + R=fergus + DELTA=97 (52 added, 17 deleted, 28 changed) + + * include_server/Makefile.in: edit + * include_server/basics.py: edit + * include_server/basics_test.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node_test.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + * include_server/parse_file_test.py: edit + + +2007/11/01 17:46:44 Nils Klarlund <opensource@google.com> + + + Increase include server timeout to 3.8s. We are still getting emails + about the include server running out of time for gws files and many + others. + + R=fergus + DELTA=2 (0 added, 0 deleted, 2 changed) + + * include_server/basics.py: edit + + +2007/11/01 17:28:22 Nils Klarlund <opensource@google.com> + + Fix buglet: when the contents of the file descriptor would contain + percent signs the call to DEBUG would fail. + + R=fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * include_server/include_server.py: edit + + +2007/10/23 17:56:23 Nils Klarlund <opensource@google.com> + + Prepare new release, pump18, changing a timing constant in include server. + + R=fergus + CC=distcc-pump + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/10/23 17:53:40 Nils Klarlund <opensource@google.com> + + Increase time quota introduced in a previous change for handling a request. We + released pump17 10/22. One day later, it turns out that there are a + cases where the include server spends more than 2s processsing a + compilation unit. Here we increase the 2s to 3s. + + R=fergus + DELTA=9 (4 added, 0 deleted, 5 changed) + + * include_server/basics.py: edit + + +2007/10/22 13:43:09 Nils Klarlund <opensource@google.com> + + + Limit the time that the includer analyzer is allowed to spend servicing + a request. We use the rudimentary SIGALRM mechanism, because generally + Python does not allow one thread to raise an exception in another. + + Experiments show that SIGALRM interferes with the use of Popen. When + determining the default compiler, the include server uses Popen to + invoke gcc. If the SIGALRM is raised, then it be caught in the 'select', + which then aborts. + + Therefore, we stop the timer while the include server determines default + compilers. + + The timeout exception is raised according to utime passed, not real + time. That prevents the include server from going bananas, restarting + again and again, because of NFS delays. + + The risk with this CL is that something else breaks in unexpected ways, + perhaps a file operation, with the signaling. However, we've put the + first signal to occur 2s after the start of the request. That should be + more than ample time. In practice, we have not seen the include server + using more than 2s per request (but this is based on limited + measurements). + + We have also tested this change with a 1s timer. That showed the SIGALRM + problem with Popen. After, we fixed that this CL seems to work fine. + + R=manos + CC=distcc-pump + DELTA=110 (89 added, 1 deleted, 20 changed) + + * include_server/basics.py: edit + * include_server/compiler_defaults.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/10/22 12:46:42 Nils Klarlund <opensource@google.com> + + Make version 'pump17'. + + R=fergus + CC=distcc-pump + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/10/22 12:39:29 Nils Klarlund <opensource@google.com> + + Fix name of stamp file. Remove initial '.'. + + R=manos + CC=fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * run_all_autoconf.sh: edit + + +2007/10/22 11:06:40 Fergus Henderson <opensource@google.com> + + Partial rollback of a previous change. + + In particular, comment out the code to support the Objective-C file + extensions and the Objective-C test case. Currently Objective-C + support breaks things if you don't have an Objective-C compiler + installed, even if distcc is used only to compile C/C++ code; we try + to compute the default include paths for all languages at startup, + and barf it if fails. + + Leave in place the support for "#import". + It turns out that "#import" can be used in GNU C/C++ code, + as a GNU extension, and some Google code makes use of this. + + *** Original change description *** + + Add support for Objective-C to distcc-pump. + + In particular, recognize the Objective-C file extensions (".m", ".mi") + and the "#import" declaration. (#import is like #include, but never + includes the same file twice. So for the distcc-pump include server, + processing of #import is exactly the same as processing of #include.) + + Tested with "make maintainer-check" (all tests pass) and + "make pump-check" (no regressions). Also added a test of + Objective-C compilation. + + R=klarlund + CC=distcc-pump + APPROVED=klarlund + DELTA=21 (11 added, 0 deleted, 10 changed) + + * distcc/test/testdistcc.py: edit + * include_server/basics.py: edit + + +2007/10/19 16:52:44 Nils Klarlund <opensource@google.com> + + Loosen assumption about the syntax -D options. A user had written + something the include server couldn't understand. It provoked a rare + internal error. The compilation did not succeed locally either. The + include server must remain unfazed when confronted with weird syntax. + + Also, corrected comment for ParseCommandArgs. + + R=fergus + CC=distcc-pump + + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/parse_command.py: edit + * include_server/parse_command_test.py: edit + + +2007/10/19 16:28:57 Nils Klarlund <opensource@google.com> + + When storing warning messages in temporary file, we are using 'print + >>', which appends a newline. Then, when presenting the errors to the + user, we'd also use 'print', which appends another newline. With this + fix, the newline is not append the first time around (we append + ','). The other use of the contents of the temporary file, in the mail + sending routines, should not be affected by one less newline. + + R=manos + CC=fergus + DELTA=3 (0 added, 0 deleted, 3 changed) + + * include_server/include_server.py: edit + + +2007/10/19 10:57:39 Nils Klarlund <opensource@google.com> + + Fix so that 'make' fails when version.sh is out of date. We remade the + dependency checking in the makefile so that the need to autoreconf is + expressed as a depedency on something certainly produced by autoreconf + when run through our script, namely a time stamp file. + + R=fergus + DELTA=27 (4 added, 12 deleted, 11 changed) + + * Makefile.in: edit + * run_all_autoconf.sh: edit + + +2007/10/19 10:15:14 Nils Klarlund <opensource@google.com> + + Don't send email when a file that exists cannot be read. Also, slight + correction to error message. + + R=fergus + DELTA=5 (4 added, 0 deleted, 1 changed) + + * include_server/parse_file.py: edit + + +2007/10/18 15:56:20 Fergus Henderson <opensource@google.com> + + Fix a missing word in a comment. + + R=klarlund + DELTA=5 (0 added, 2 deleted, 3 changed) + + * distcc/src/compile.c: edit + + +2007/10/18 11:41:46 Fergus Henderson <opensource@google.com> + + Add support for Objective-C to distcc-pump. + + In particular, recognize the Objective-C file extensions (".m", ".mi") + and the "#import" declaration. (#import is like #include, but never + includes the same file twice. So for the distcc-pump include server, + processing of #import is exactly the same as processing of #include.) + + Tested with "make maintainer-check" (all tests pass) and + "make pump-check" (no regressions). Also added a test of + Objective-C compilation. + + R=klarlund + CC=distcc-pump + APPROVED=klarlund + DELTA=136 (100 added, 8 deleted, 28 changed) + + * distcc/test/testdistcc.py: edit + * include_server/basics.py: edit + * include_server/parse_command.py: edit + * include_server/parse_file.py: edit + + +2007/10/12 14:10:36 Nils Klarlund <opensource@google.com> + + This corrects description of the previous change. + + Prepare a new release, pump16. Deploy in NYC only. To be + installed under 2007.10.12 in buildstatic. This release includes: + +'Rather than just replacing the ' Change 2007/10/11 by Fergus Henderson <opensource@google.com> +'Prevent distcc-pump from revert' Change 2007/10/11 by Nils Klarlund <opensource@google.com> + + R=fergus + CC=manos + DELTA=2 (2 added, 0 deleted, 0 changed) + + * version.sh: edit + + +2007/10/12 11:30:45 Nils Klarlund <opensource@google.com> + + Prepare a new release, pump16, to include the change "Prevent + distcc-pump from reverting to local compilations". Deploy in + NYC only. To be installed under 2007.10.11 in buildstatic. + + R=fergus + CC=manos + DELTA=2 (0 added, 0 deleted, 2 changed) + + * version.sh: edit + + +2007/10/11 22:06:33 Fergus Henderson <opensource@google.com> + + Rather than just replacing the server's current working directory + with the clients' current working directory, we need to replace + the server's root directory with the client's root directory. + + This is needed because the problems with gdb arise not only + from the "current working directory" field (DW_AT_comp_dir), + but also with the file names (DW_AT_file_name). These file names + can be absolute paths to files that are outside of the current + working directory. + + This change is basically just a one-line fix in serve.c, but + for clarity I've also changed some variable names and comments + in fix_debug_info.c. + + R=klarlund + CC=distcc-pump + DELTA=36 (13 added, 0 deleted, 23 changed) + + * distcc/src/fix_debug_info.c: edit + * distcc/src/serve.c: edit + + +2007/10/11 16:20:33 Nils Klarlund <opensource@google.com> + + Prevent distcc-pump from reverting to local compilations hundreds + or thousands of times. Such a situation should never occur but if it + occurs, we want to fall back to distcc behavior. For example, this is + desirable if the include server for reason picks up files that don't + exist any more or symbolic links that changed surreptiously during the + build. + + A typical output is shown below (where we have tampered with the include + server to produce too few files). Note that ERRORs still occur in the + output (from the remote server). + + distcc[7791] ERROR: compile third_party/pcre/pcre-7.3/pcre_globals.c on 192.168.1.137,lzo,cpp failed + distcc[7791] (dcc_build_somewhere) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_globals.c' failed, retrying locally + distcc[7791] Warning: failed to distribute third_party/pcre/pcre-7.3/pcre_globals.c to 192.168.1.137,lzo,cpp, running locally instead + distcc[7708] (dcc_please_send_email_note_discrepancy_except_fresh_dependency) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_chartables.c' failed, retried locally and got a different result. + distcc[7708] (dcc_note_discrepancy) Warning: now using plain distcc, possibly due to inconsistent file system changes during build + distcc[7708] Warning: Will send an email to distcc-pump-errors@google.com + distcc[7780] ERROR: compile third_party/pcre/pcre-7.3/pcre_get.c on 192.168.1.135,lzo,cpp failed + distcc[7780] (dcc_build_somewhere) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_get.c' failed, retrying locally + distcc[7780] Warning: failed to distribute third_party/pcre/pcre-7.3/pcre_get.c to 192.168.1.135,lzo,cpp, running locally instead + distcc[7869] ERROR: compile third_party/pcre/pcre-7.3/pcre_ord2utf8.c on 192.168.1.124,lzo,cpp failed + distcc[7869] (dcc_build_somewhere) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_ord2utf8.c' failed, retrying locally + distcc[7869] Warning: failed to distribute third_party/pcre/pcre-7.3/pcre_ord2utf8.c to 192.168.1.124,lzo,cpp, running locally instead + __________Compiling obj/gcc-4.2.1-glibc-2.2.2-piii-linux-dbg/bin/third_party/pcre/pcre-7.3/pcre_ucp_searchfuncs.c.o + distcc[7780] (dcc_please_send_email_note_discrepancy_except_fresh_dependency) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_get.c' failed, retried locally and got a different result. + distcc[7780] Warning: Will send an email to distcc-pump-errors@google.com + distcc[7791] (dcc_please_send_email_note_discrepancy_except_fresh_dependency) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_globals.c' failed, retried locally and got a different result. + distcc[7791] Warning: Will send an email to distcc-pump-errors@google.com + __________Compiling obj/gcc-4.2.1-glibc-2.2.2-piii-linux-dbg/bin/third_party/pcre/pcre-7.3/pcre_valid_utf8.c.o + distcc[7723] (dcc_please_send_email_note_discrepancy_except_fresh_dependency) Warning: remote compilation of 'third_party/pcre/pcre-7.3/pcre_compile.c' failed, retried locally and got a different result. + distcc[7723] Warning: Will send an email to distcc-pump-errors@google.com + + R=fergus,manos + DELTA=211 (174 added, 8 deleted, 29 changed) + + * distcc/src/compile.c: edit + * distcc/src/compile.h: edit + * distcc/src/h_compile.c: edit + * distcc/test/testdistcc.py: edit + + +2007/10/09 12:40:31 Nils Klarlund <opensource@google.com> + + + The obvious radical approach to clearing all caches. With this CL, we + run all the initialization code of the major include analyzer object. + This changes passes all usual tests and some third_party tests. + All to be run before release. + + R=fergus,manos + DELTA=52 (17 added, 32 deleted, 3 changed) + + * include_server/cache_basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/mirror_path.py: edit + + +2007/10/09 12:38:53 Nils Klarlund <opensource@google.com> + + Introduce a time analysis of source files in order to improve S/N + ratio of automatically generated error emails. Currently, we send + an email in situations where: + + -- the remote compilation fails because a file contains something bad, + -- the file is changed to not contain bad stuff, and consequently the + -- the local compilation succeeds. + + This makes it sometimes impossible for us to determine accurately from + the email whether some error really occurred or whether the user changed + source files during the build. + + We can carry out the analysis only if there is a .d file. We check each + dependency descriped there. If anyone changed after the build started, + then we really don't want to hear about distcc-pump errors, because + dependencies shouldn't change. The generated files are exceptions. To + disregard these, the distcc user may specify a glob pattern in + environment variable DISTCC_EXCLUDE_FRESH_FILES defined when + invoking distcc. + + R=fergus + APPROVED=fergus,sunq + DELTA=378 (362 added, 4 deleted, 12 changed) + + * distcc/Makefile.in: edit + * distcc/configure.ac: edit + * distcc/src/compile.c: edit + * distcc/src/compile.h: edit + * distcc/src/emaillog.c: edit + * distcc/src/h_compile.c: add + * distcc/test/testdistcc.py: edit + * pump.in: edit + + +2007/10/09 12:08:14 Nils Klarlund <opensource@google.com> + + Release pump15 to buildstatic under name 2007.10.09. To include: + + Fix a bug with the check to see whether /usr/lib/rpm/... + The obvious radical approach to... + Ensure that the build process reruns... + Introduce a time analysis of source files... + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/10/08 22:56:48 Fergus Henderson <opensource@google.com> + + Fix a bug with the check to see whether /usr/lib/rpm/find-requires + has been patched to special-case the linux-gate.so dependency. + The sense of the check was inverted. On my system, find-requires + had been properly patched already, but build-distcc.sh was + incorrectly telling me that I needed to patch it. + + R=klarlund + CC=distcc-pump + DELTA=1 (0 added, 0 deleted, 1 changed) + + * build-distcc.sh: edit + + +2007/10/08 22:49:19 Fergus Henderson <opensource@google.com> + + Ensure that the build process reruns autoconf and autoheader. + This is needed in case configure.ac has been changed since + the last build. + + R=klarlund + CC=distcc-pump + DELTA=1 (1 added, 0 deleted, 0 changed) + + * distcc.spec: edit + + +2007/10/08 10:35:13 Nils Klarlund <opensource@google.com> + + Start a new snaphost of the file system when caches are cleared as part + of the stat trigger mechanism. + + The stat reset is supposed to mean: ``don't trust anything about + previous analyses of includes.'' So files, directories, and symbolic + links, as known to the include server, are all invalid. + + In particular, all information gathered in the client root directory is + stale. (This is where we compress source file and mirror the directory + and link structure of the part of the file system explored for include + analysis.) + + With this CL, we make the client root concept generational, as indicated + by a new suffix, a generation number, added to the automatically + generated root name. A trigger event bumps the generation number by + one. + + R=manos,fergus + DELTA=86 (60 added, 4 deleted, 22 changed) + + * include_server/basics.py: edit + * include_server/compress_files.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/include_server.py: edit + * include_server/mirror_path.py: edit + + +2007/10/04 18:18:18 Manos Renieris <opensource@google.com> + + Update the init script. Major difference is the understanding of comments. + + R=klarlund + CC=ahyun,distcc-pump + DELTA=248 (25 added, 62 deleted, 161 changed) + + * distcc/packaging/RedHat/init.d/distcc: edit + + +2007/10/04 15:05:43 Fergus Henderson <opensource@google.com> + + Tidy up the output from c_extensions_test. + Also change the test so that it will die if some unreachable code is + reached, rather than quietly ignoring the error condition, + and avoid using the message "internal error 3" for two different errors. + + R=manos + CC=distcc-pump + APPROVED=klarlund + DELTA=8 (5 added, 0 deleted, 3 changed) + + * include_server/Makefile.in: edit + * include_server/c_extensions_test.py: edit + + +2007/10/04 13:25:56 Nils Klarlund <opensource@google.com> + + Warning message clean-up. + + 1) Change all information printed by Debug to go to stderr. + + 2) When an exception is raised it should generally be reported as a new + warning (as defined by Debug called with DEBUG_WARNING, so that this + output is controllable), not just printed to stderr. In particular, + this change makes situations not covered by the include processor (such + as absolute filepaths in #include's) be reported as warnings. + + 3) We removed an unnecessary try/except block for totally unexpected + siutations. Those are to be handled by the general except clause of the + 'handler'. + + Fatal issues and timing information (not normally used) is still + reported to stderr through print statements. + + R=fergus,manos + DELTA=18 (0 added, 4 deleted, 14 changed) + + * include_server/basics.py: edit + * include_server/include_server.py: edit + + +2007/10/04 13:06:01 Fergus Henderson <opensource@google.com> + + Include h_fix_debug_info in the list of check_programs, + so that it gets removed by "make clean". + + R=klarlund + DELTA=1 (1 added, 0 deleted, 0 changed) + + * distcc/Makefile.in: edit + + +2007/10/04 11:37:39 Nils Klarlund <opensource@google.com> + + If Ctrl-c is issued while the include server is running in 'handle', + then an unfortunate double error message would sometimes occur, along + with a stack trace. Fixed here by propagating exception to outer handler + in 'Main'. [While we're at it, we change 'main' to 'Main'. There's no + reason to lowercase it.] + + Also, we fix the situations where the exception raised in 'handle' is an + internal error or the one corresponding to SIGTERM. Both situations are + uncommon. And, in both cases, we raise SIGTERM for Main to catch, so + that include server can be terminated without stack traces to be printed + out. + + R=fergus,manos + DELTA=6 (0 added, 0 deleted, 6 changed) + + * include_server/include_server.py: edit + + +2007/10/04 10:13:58 Fergus Henderson <opensource@google.com> + + Set VPATH. This is needed to make things work if you run configure + in a different directory than the source directory. + + R=klarlund + CC=distcc-pump + DELTA=1 (1 added, 0 deleted, 0 changed) + + * Makefile.in: edit + + +2007/10/04 01:10:42 Fergus Henderson <opensource@google.com> + + Add the header file for a source file I added in a previous change. + I had accidentally omitted this file from that CL. + + R=iant + CC=distcc-pump + APPROVED=iant + DELTA=10 (10 added, 0 deleted, 0 changed) + + * distcc/src/fix_debug_info.h: add + + +2007/10/03 20:41:00 Nils Klarlund <opensource@google.com> + + Version 14 will contain: + + - a server update (for .o rewriting), to be installed 10/4 in NYC + - fix for include server failure to update symbolic links when clearing + caches + - minor changes to messages in exception circumstances + - possibly, change that analyzes .d files before sending emails + about discrepancies between server and client + + R=fergus,manos + + * version.sh: edit + + +2007/10/03 15:03:24 Nils Klarlund <opensource@google.com> + + Add clarifying comment. + + R=fergus,manos + APPROVED=manos + DELTA=1 (1 added, 0 deleted, 0 changed) + + * include_server/test_data/dfoo/foo2.h: edit + + +2007/10/03 12:04:55 Fergus Henderson <opensource@google.com> + + Change distcc so that it generates the correct value for + the "current directory" field in the debug info. + + I do this by mmapping the ".o" file into memory, parsing the + ELF headers to find the ".debug_info" section, and then doing a + blind string search and replace in that section. This should be + quite safe: although there is a remote possibility of false hits, + they will only affect the debug_info section. + + R=iant,klarlund + CC=distcc-pump + APPROVED=klarlund + DELTA=628 (594 added, 5 deleted, 29 changed) + + * distcc/Makefile.in: edit + * distcc/configure.ac: edit + * distcc/src/config.h.in: edit + * distcc/src/fix_debug_info.c: add + * distcc/src/serve.c: edit + * distcc/test/testdistcc.py: edit + + +2007/09/28 17:52:51 Nils Klarlund <opensource@google.com> + + Fix .d target error. This error was unfortunately introduced with + a previous change, which corrected other errors and attempted to put this + logic into a unit test straightjacket. + + R=fergus,manos + DELTA=2 (0 added, 1 deleted, 1 changed) + + * distcc/src/dotd.c: edit + * distcc/test/testdistcc.py: edit + + +2007/09/28 17:13:53 Nils Klarlund <opensource@google.com> + + Prepare release of pump13 to overcome bug in yesterday's release. + + R=fergus,manos + + * version.sh: edit + + +2007/09/27 14:18:09 Nils Klarlund <opensource@google.com> + + Prepare new release. + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/09/26 22:31:54 Nils Klarlund <opensource@google.com> + + Allow globs in trigger path expressions. + + R=fergus,manos + DELTA=101 (85 added, 0 deleted, 16 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/include_server.py: edit + * pump.in: edit + + +2007/09/26 22:01:05 Nils Klarlund <opensource@google.com> + + Fix parsing of -D options to allow for function-like macros. + + R=fergus,manos + DELTA=88 (55 added, 28 deleted, 5 changed) + + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/parse_file.py: edit + * include_server/test_data/func_macro.c: add + + +2007/09/26 21:51:45 Nils Klarlund <opensource@google.com> + + The interpretation of the # operator of CPP was incorrect because + whitespace after # was not ignored. For example, + + # a + + must expand to + + "a" + + not + + "" a + + We fix this problem here. + + R=fergus,manos + DELTA=3 (2 added, 0 deleted, 1 changed) + + * include_server/macro_eval.py: edit + * include_server/macro_eval_test.py: edit + + +2007/09/21 22:57:17 Fergus Henderson <opensource@google.com> + + Fix two compiler warnings, one of them serious: + - The type of the "len" argument in a call to + PyArg_ParseType("#s", &string, &len) + must be "int", not "size_t". This could cause + serious problems for a 64-bit architecture. + - Include "distcc.h" before the other header file, + so that we get the definition of 'enum dcc_protover' + before it is used. + + R=klarlund + CC=distcc-pump + DELTA=4 (1 added, 2 deleted, 1 changed) + + * include_server/c_extensions/distcc_pump_c_extensions_module.c: edit + + +2007/09/21 22:56:33 Fergus Henderson <opensource@google.com> + + When compiling the distcc code and the extension module, enable C + compiler warnings ("-Wall -Wextra"), and enforce them ("-Werror"). + Also make sure that we compile with "-D_GNU_SOURCE", since distcc + uses that, and we need that to avoid some compiler warnings. + + R=klarlund + CC=distcc-pump + DELTA=3 (0 added, 0 deleted, 3 changed) + + * include_server/setup.py: edit + + +2007/09/21 22:56:08 Fergus Henderson <opensource@google.com> + + Pass the "-v" (verbose) option to "autoreconf". + + R=klarlund + DELTA=1 (0 added, 0 deleted, 1 changed) + + * run_all_autoconf.sh: edit + + +2007/09/21 13:41:36 Nils Klarlund <opensource@google.com> + + Fix bugs in dcc_get_dotd_info that give rise to erroneous .d file names + or erroneous target names in .d files. + + - If derived from the file name given by -o, then the suffix must be + stripped before .d is appended. + + - If the file name ends in '.', then only "d" is appended. + + - If an -o name is given and the name ends in '.', then this name, not + the input filename, is still to be used for the name of the .d file. + + - If a target is provided in DEPENDENCIES_OUTPUT, then this target must + be returned -- otherwise the .d file computed on the server will be + incorrect if the target happens not to be the object file. + + Extra: a test for the length of the string in variable extension was + unnecessary. + + Tests: We made a unit test that exercices the function in a number of + ways. Heuristics about the names of .d files, expressed as globs, are + used to pinpoint the actual naming conventions used by gcc. + + R=manos,fergus + DELTA=255 (217 added, 20 deleted, 18 changed) + + * distcc/Makefile.in: edit + * distcc/src/dotd.c: edit + * distcc/src/h_dotd.c: add + * distcc/test/testdistcc.py: edit + + +2007/09/20 11:36:58 Nils Klarlund <opensource@google.com> + + Fix sending of emails by include server. Avoid sending emails when + translation unit could not be found or if command line is + malformed. Also, fix buglet that made the previous translation unit name + appear in the case of malformed command line. + + R=fergus,manos + DELTA=6 (2 added, 0 deleted, 4 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_server_test.py: edit + * include_server/parse_command.py: edit + + +2007/09/20 11:33:48 Nils Klarlund <opensource@google.com> + + Prepare new release to include "Reset caches more aggressively". + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/09/20 07:54:03 Nils Klarlund <opensource@google.com> + + Reset caches more aggressively. According to a bug report, + there are uncommon build situations where distcc-pump fails. We address + this by: + + -- Making sure to reset the cache use by the link-gathering mechanism of + mirror_path in the include server. This is a real bug although + probably not the culprit. + + -- Also monitoring 'includes/third_party/stl/third_party/stl', which is + the symbolic link changed (with path-doubling) by the set symlink + mechanism of the third_party/stl/BUILD file. + + R=fergus,manos + DELTA=6 (5 added, 0 deleted, 1 changed) + + * include_server/include_analyzer.py: edit + * include_server/mirror_path.py: edit + * pump.in: edit + + +2007/09/18 13:03:49 Nils Klarlund <opensource@google.com> + + Prepare 'pump10' release. This release is to correct minor issues such + as interaction with ccache and the handling of command line -D options. + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/09/18 12:59:45 Nils Klarlund <opensource@google.com> + + Correct two mistakes in the handling of filepaths: + + -- an elementary one in basics.py: the rstrip() function of string is + functional + + -- a more tricky one: in an include of the form "#include ./foo.h", we + must normalize the file name to 'foo.h', otherwise an infinite regression + would occur if foo.h includes ./foo.h. + + R=fergus,manos + DELTA=13 (10 added, 1 deleted, 2 changed) + + * include_server/basics.py: edit + * include_server/cache_basics.py: edit + + +2007/09/17 14:51:56 Nils Klarlund <opensource@google.com> + + + Fix client to change protocol version from 3 (pump) to 2 (plain) in + situations involving already preprocessed code, such as arising from + ccache use, or involving certain insidious environment variables. + + Test: + make -C distcc valgrind-check + + R=fergus,manos + DELTA=41 (40 added, 1 deleted, 0 changed) + + * distcc/src/compile.c: edit + + +2007/09/17 13:56:56 Nils Klarlund <opensource@google.com> + + When announcing pump installation, show the real location -- unmasked + by symbolic links. + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * pump.in: edit + + +2007/09/17 13:26:48 Fergus Henderson <opensource@google.com> + + Fix a bug in "pump --startup": don't print out a setting for + DISTCC_HOSTS if we didn't set DISTCC_HOSTS. + + (Note: this is not an urgent fix, because I've made Blaze work + around the bug. But worth fixing nevertheless.) + + R=klarlund + CC=distcc-pump + DELTA=3 (2 added, 0 deleted, 1 changed) + + * pump.in: edit + + +2007/09/17 12:33:21 Nils Klarlund <opensource@google.com> + + Make 'pump' executable. + + R=fergus,manos + DELTA=2 (1 added, 0 deleted, 1 changed) + + * configure.ac: edit + + +2007/09/17 09:48:08 Nils Klarlund <opensource@google.com> + + Propagate symbols defined through -D options to include analyzer. This + makes computed includes operable wtih values provided on the command + line. + + Tests: + + 1. New tests exercise that when reissuing an analysis for the same file + and with same search directories, but with a symbol definition affecting + a previously computed include, the include graph is recalculated, not + reused, and the new value of the symbol is used for resolving the + computed include. + + 2. The new code was also verified to solve the problem reported in + a bug report (I used a workstation in ZRH to build the + actual code that cause the issue). + + 3. I did 'make maintainer-check'. + + R=fergus,manos + DELTA=66 (56 added, 0 deleted, 10 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_analyzer_memoizing_node_test.py: edit + * include_server/include_analyzer_test.py: edit + * include_server/parse_command.py: edit + * include_server/parse_command_test.py: edit + + +2007/09/15 21:47:49 Nils Klarlund <opensource@google.com> + + Update a comment as per Fergus's review comment for a previous change. + + R=fergus,manos + DELTA=3 (1 added, 0 deleted, 2 changed) + + * include_server/compiler_defaults.py: edit + + +2007/09/13 23:40:04 Nils Klarlund <opensource@google.com> + + Introduces mechanism for monitoring the real paths that are resolved + during include analysis. Invoke as --realpath_warning_re=RE. For more, + see include_server.py, Usage() in this CL. This allows a user to find + out which source files are actually picked out by the include analysis + during the build. Such information can useful if it is suspected that + include directories are wrong. + + TESTING: this CL passes make check and make maintainer-check. It was + tested earlier on Google software as part of the + discussion of this feature now found on the distcc-pump wiki. + + From the wiki: + + INCLUDE_SERVER_ARGS='--realpath_warning_re=RE' make-dbg .... + + makes the include server write a warning to stderr whenever a filename + is resolved to a realpath (a canonicalized absolute pathname - see + 'man realpath') that is matched by RE, which is a regular expression + in Python syntax. For example, to find out which C/C++ files from + /home/build/buildonly are used during a build of gws:gws, do: + + INCLUDE_SERVER_ARGS='--realpath_warning_re=/auto/buildonly' make-dbg -distcc_pump -g0 gws:gws + + Then, you will see messages like: + + WARNING include server: For translation unit + 'foo/bar.cc' + while processing 'foo/bar.cc' + lookup of file 'baz.h' resolved to + '../whatever/baz.h' + whose realpath is + '/auto/whatever/blah/blah/baz.h'. + + (Here we have formatted the message from it's one-liner original.) + Note that the include server currently does not identify the header + file containing the #include for which the realpath matched, but only + the translation unit. Also, because of caching in the include server, + a message for a particular resolution appears usually only once. Thus + this analysis cannot be used to find out which files a particular + compilation unit uses. + + R=fergus,manos + DELTA=42 (40 added, 0 deleted, 2 changed) + + * include_server/basics.py: edit + * include_server/cache_basics.py: edit + * include_server/include_server.py: edit + + +2007/09/13 23:15:37 Nils Klarlund <opensource@google.com> + + Introduce flag for NotCoveredError exceptions that specify whether + email is to be sent or not. + + We use this parameter to suppress warnings that related to + + #include "/foo/bar.h" + + and similar uses. + + Also, clean up of some error messages (add '.'). We fix a few typos and + unneeded variables (e.g.: opt_build_stat_verify), while we're looking at + these files. + + Additionally, improve NotCovered messages by adding the file name (in + most cases). + + Finally, introduce option --no-email to negate the default of --email in + the pump script if so desired through INCLUDE_SERVER_ARGS. + + TESTING: + + This change passes 'make check' and 'make maintainer-check'. We also ran + system tests and verified that occurences of #include "/foo/bar.h" now + does not result in mail being generated. These tests are not part of + this changelist, but contained in another change. + + R=manos,fergus + DELTA=186 (93 added, 49 deleted, 44 changed) + + * include_server/basics.py: edit + * include_server/cache_basics.py: edit + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + * include_server/parse_file.py: edit + + +2007/09/13 18:18:25 Nils Klarlund <opensource@google.com> + + Reorganize and extend tests: + + - move existing tests in include_server.py to new include_server_test.py + + - test the main 'handler' in include_server, including how exceptions, + such as internal error, affect automated email generation + + R=fergus,manos + DELTA=613 (374 added, 198 deleted, 41 changed) + + * include_server/Makefile.in: edit + * include_server/include_analyzer_test.py: add + * include_server/include_server_test.py: edit + * include_server/test_data/contains_abs_include.c: add + + +2007/09/13 14:54:43 Nils Klarlund <opensource@google.com> + + Prepare for release; bump to version 'pump9'. + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/09/13 14:41:34 Nils Klarlund <opensource@google.com> + + Remove Google crosstool specification (which made the build fail + anyway). + + R=manos,fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * build-distcc.sh: edit + + +2007/09/13 14:38:24 Nils Klarlund <opensource@google.com> + + Correct flawed configuration scripts and introduce configuration sanity + checks. + + autoconfig assumes that if an .ac script has not changed then the + resulting configure scripts do not need to change. + + This assumption is untrue because we use a shell escape. (This + escape has been rewritten according to Fergus's suggestion.) + + Moreover, m4 caches processed files. This royally messes up things + beyond and in addition to just the aforementioned problem. + + We fix this by using autoreconf (suggested by Fergus). It seems to + address these issues. + + We have verified that: + + ./run_all_autoconf.sh; ./configure; make clean; make all + + works and embeds a changed version number (in version.sh) in the right + places. + + With further CLs "Remove Google crosstool specification" and + "Introduce needed cast.", it is possible to use + ./build-distcc.sh to make RPM and .deb packages. + + R=fergus,manos + DELTA=76 (36 added, 20 deleted, 20 changed) + + * Makefile.in: edit + * configure.ac: edit + * distcc/configure.ac: edit + * include_server/configure.ac: edit + * run_all_autoconf.sh: edit + * version.sh: edit + + +2007/09/13 14:21:11 Nils Klarlund <opensource@google.com> + + Introduce needed cast. This problem was detected with gcc + version 4.0.3 (Ubuntu 4.0.3-1ubuntu5). + + R=fergus,manos + DELTA=3 (2 added, 0 deleted, 1 changed) + + * distcc/src/stats.c: edit + + +2007/09/13 14:18:44 Nils Klarlund <opensource@google.com> + + Not deleting .d files breaks the build sometimes. + + R=fergus,manos + DELTA=4 (1 added, 0 deleted, 3 changed) + + * distcc/Makefile.in: edit + + +2007/09/12 21:08:29 Fergus Henderson <opensource@google.com> + + Use the standard autoconf substitution process to create pump + from pump.in, rather than a hand-crafted Makefile rule. + + R=klarlund + CC=distcc-pump + DELTA=16 (4 added, 6 deleted, 6 changed) + + * Makefile.in: edit + * configure.ac: edit + * pump.in: edit + + +2007/09/12 20:49:57 Fergus Henderson <opensource@google.com> + + Fix a Posix conformance bug: we were passing the same fd set to + select() for both the writeable fd set and the error fd set, but + according to Posix (specifically The Open Group Base Specifications Issue 6, + available at <http://www.opengroup.org/onlinepubs/009695399/>), + the arguments to select() are declared with the "restrict" attribute, + which means they must not be aliased. + + Also, issue a trace message in the case when select() returns + due to an error condition on the fd rather than because the fd + has become writable. + + R=manos + CC=distcc-pump + DELTA=23 (18 added, 0 deleted, 5 changed) + + * distcc/src/io.c: edit + + +2007/09/12 18:57:03 Fergus Henderson <opensource@google.com> + + Fix manual memory management bugs, and simplify the memory management + a bit. + + dcc_set_output() was inconsistent about whether it copied its + input argument or aliased it, about whether it freed the previous + value in that slot. I changed the function so that it always + copies the input argument, and always frees the previous value + in the argv array slot that it is about to overwrite. + I likewise changed dcc_set_input to copy its argument. + + dcc_set_output() was also inconsistent about whether or not it + logged the change to the argument; I fixed that too, by always + logging it. + + In serve.c I simplified the code and comments a little - there + is no longer any aliasing going on, so the deallocation logic + is simpler. + + R=manos + CC=distcc-pump + DELTA=35 (9 added, 12 deleted, 14 changed) + + * distcc/src/arg.c: edit + * distcc/src/serve.c: edit + + +2007/09/12 18:51:05 Fergus Henderson <opensource@google.com> + + Add support for running a single test case at a time, + rather than running all of them. This is useful for + debugging. + + R=manos + CC=distcc-pump + APPROVED=klarlund + DELTA=46 (46 added, 0 deleted, 0 changed) + + * distcc/Makefile.in: edit + * distcc/test/onetest.py: add + + +2007/09/12 18:48:53 Fergus Henderson <opensource@google.com> + + Fix a timing dependence in the distcc test suite that caused the + 'NoDetachDaemon_Case' test case to sometimes fail when run under + valgrind. Specifically, when running the server in --no-detach mode, + wait until the server starts accepting connections before running + the test, rather than just waiting 0.5 seconds, which under valgrind + is not always long enough. + + R=klarlund + CC=distcc-pump + DELTA=6 (3 added, 0 deleted, 3 changed) + + * distcc/test/testdistcc.py: edit + + +2007/09/12 18:47:34 Fergus Henderson <opensource@google.com> + + Add "--startup" and "--shutdown" options to the pump script, + for use by Blaze. Blaze can't use the ordinary mode of the + pump script, because the Blaze server's lifetime is greater than + the lifetime of the include server. + + R=klarlund + CC=distcc-pump + DELTA=181 (119 added, 12 deleted, 50 changed) + + * pump.in: edit + + +2007/09/11 13:59:25 Nils Klarlund <opensource@google.com> + + Annul enviroment when probing for built-in include path. Fixes subtle + problem reported in a bug report. + + R=fergus,manos + DELTA=6 (3 added, 0 deleted, 3 changed) + + * include_server/compiler_defaults.py: edit + + +2007/09/11 12:37:56 Nils Klarlund <opensource@google.com> + + Fix leftover comment from earlier fix: delete it. + + R=manos + DELTA=3 (0 added, 3 deleted, 0 changed) + + * include_server/include_analyzer_memoizing_node.py: edit + + +2007/09/06 16:02:39 Nils Klarlund <opensource@google.com> + + Improve mechanishm for overcoming unsoundness in build system. Change + the semantics of stat_reset_triggers so that the changing of a symbolic + link during the build triggers a reset of all include server caches. + + R=fergus,manos + DELTA=57 (28 added, 1 deleted, 28 changed) + + * include_server/basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + + +2007/09/05 15:50:58 Fergus Henderson <opensource@google.com> + + Make distcc warning-free: + - Remove "-Wconversion" from CFLAGS; this warning was useful for + writing code that had to be compatible with pre-C89 C compilers + that didn't support prototypes, but that isn't needed anymore, + and it causes too many spurious warnings for perfectly legitimate + code. + - Add "-Wno-unused" to cflags for popt/*.c; those files have + a bunch of unused parameters. + - Eliminate all the remaining compiler warnings. + - Add "-Werror" to CFLAGS so we won't get any regressions! + + TESTED = make maintainer-check valgrind-check + + R=manos + CC=distcc-pump + APPROVED=manos + DELTA=56 (27 added, 6 deleted, 23 changed) + + * distcc/Makefile.in: edit + * distcc/configure.ac: edit + * distcc/popt/popt.c: edit + * distcc/popt/popt.h: edit + * distcc/popt/popthelp.c: edit + * distcc/src/bulk.c: edit + * distcc/src/emaillog.c: edit + * distcc/src/exec.c: edit + * distcc/src/hosts.c: edit + * distcc/src/include_server_if.c: edit + * distcc/src/lsdistcc.c: edit + * distcc/src/prefork.c: edit + * distcc/src/srvnet.c: edit + * distcc/src/stats.c: edit + + +2007/08/30 14:01:37 Manos Renieris <opensource@google.com> + + When receiving a string, print it out. + + R=klarlund,fergus + DELTA=2 (2 added, 0 deleted, 0 changed) + + * distcc/src/rpc.c: edit + + +2007/08/30 13:58:45 Nils Klarlund <opensource@google.com> + + Make tracing for client print out strings that are sent, not just their lengths. + + R=manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * distcc/src/rpc.c: edit + + +2007/08/29 21:44:15 Nils Klarlund <opensource@google.com> + + Fix bug in include server that gives rise to the include server being + unable to find the translation unit. Also, delete some unneeded + variables and make the remaining ones lowercased. + + R=manos,fergus + DELTA=86 (73 added, 0 deleted, 13 changed) + + * include_server/basics.py: edit + * include_server/basics_test.py: add + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/08/29 21:35:45 Nils Klarlund <opensource@google.com> + + Prepare new release. + + R=manos,fergus + DELTA=3 (2 added, 0 deleted, 1 changed) + + * version.sh: edit + + +2007/08/28 17:56:22 Nils Klarlund <opensource@google.com> + + Fix typos incidentally introduced in a previous change -- and fix a function + that scrubs file names in a test; it didn't work with links. + + R=manos + DELTA=10 (1 added, 0 deleted, 9 changed) + + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + + +2007/08/28 17:50:46 Nils Klarlund <opensource@google.com> + + Add test files, missing from a previous change. + + R=manos,fergus + APPROVED=manos + DELTA=1 (1 added, 0 deleted, 0 changed) + + * include_server/test_data/dfoo/stat_triggers.h: add + * include_server/test_data/stat_triggers.c: add + * include_server/test_data/stat_triggers.h: add + + +2007/08/28 17:47:06 Manos Renieris <opensource@google.com> + + Observe the HAVE_SENDFILE config variable. + + R=klarlund + DELTA=7 (6 added, 0 deleted, 1 changed) + + * distcc/src/bulk.c: edit + + +2007/08/28 16:23:43 Nils Klarlund <opensource@google.com> + + Make a central place for defining versions. Remove various generated + files from source control. + + R=manos,fergus + DELTA=13450 (41 added, 13401 deleted, 8 changed) + + * INSTALL: edit + * build-distcc.sh: edit + * configure: delete + * configure.ac: edit + * distcc/configure: delete + * distcc/configure.ac: edit + * include_server/configure: delete + * include_server/configure.ac: edit + * run_all_autoconf.sh: add + * version.sh: add + + +2007/08/28 12:41:59 Nils Klarlund <opensource@google.com> + + + New option --stat_reset_triggers. The --stat_reset_triggers + argument is a list of path names. If any such path changes from + non-existing to existing during the build, then all stat-related caches + are cleared. + + A test in include_server_test.py checks that the include analysis of a + file is done from scratch after a trigger path went from non-existing to + existing. + + R=manos,fergus + DELTA=229 (174 added, 44 deleted, 11 changed) + + * include_server/basics.py: edit + * include_server/cache_basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + * pump.in: edit + + +2007/08/27 14:05:26 Manos Renieris <opensource@google.com> + + Part of a change from Nils, with a test added. This once fixes the + #include"foobar" + case. + + R=klarlund,fergus + APPROVED=klarlund + DELTA=5 (4 added, 0 deleted, 1 changed) + + * include_server/parse_file.py: edit + * include_server/parse_file_test.py: edit + + +2007/08/27 13:56:47 Manos Renieris <opensource@google.com> + + Part of a change from Nils. This one adds support for the -include flag. + I've moved the test data and added a test for parse_command.py. + + R=klarlund,fergus + APPROVED=klarlund + DELTA=1545 (1498 added, 9 deleted, 38 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node_test.py: edit + * include_server/parse_command_test.py: edit + * include_server/test_data/distcc/README: add + * include_server/test_data/distcc/src/bulk.h: add + * include_server/test_data/distcc/src/compile.h: add + * include_server/test_data/distcc/src/config.h: add + * include_server/test_data/distcc/src/distcc.c: add + * include_server/test_data/distcc/src/distcc.h: add + * include_server/test_data/distcc/src/emaillog.h: add + * include_server/test_data/distcc/src/exitcode.h: add + * include_server/test_data/distcc/src/hosts.h: add + * include_server/test_data/distcc/src/implicit.h: add + * include_server/test_data/distcc/src/include_me.h: add + * include_server/test_data/distcc/src/state.h: add + * include_server/test_data/distcc/src/trace.h: add + * include_server/test_data/distcc/src/util.h: add + + +2007/08/27 13:05:08 Manos Renieris <opensource@google.com> + + Report the client ip if we refuse a connection. + + R=dkegel,klarlund,fergus + CC=ahyun + DELTA=14 (7 added, 1 deleted, 6 changed) + + * distcc/src/srvnet.c: edit + + +2007/08/20 13:21:57 Manos Renieris <opensource@google.com> + + Add include_analyzer_memoizing_node_test to the list of tests. + + R=klarlund,fergus + DELTA=2 (1 added, 0 deleted, 1 changed) + + * include_server/Makefile.in: edit + + +2007/08/20 12:00:47 Manos Renieris <opensource@google.com> + + Minimize the output during tests; it obscures the test results. + + R=klarlund,fergus + APPROVED=klarlund + DELTA=22 (1 added, 1 deleted, 20 changed) + + * include_server/c_extensions_test.py: edit + * include_server/include_server_test.py: edit + * include_server/macro_eval_test.py: edit + + +2007/08/10 11:21:02 Nils Klarlund <opensource@google.com> + + Fix a bug: memoization in the central FindNode algorithm was too + optimistic. + + R=fergus,manos + DELTA=20 (0 added, 20 deleted, 0 changed) + + * include_server/include_analyzer_memoizing_node.py: edit + + +2007/08/08 17:42:57 Nils Klarlund <opensource@google.com> + + Make include server not write a stack trace when the exception is + NotCovered. Such a situation is not bad. + + Also, remove a couple of unnecessary imports. + + R=manos,fergus + DELTA=19 (5 added, 4 deleted, 10 changed) + + * include_server/include_server.py: edit + + +2007/08/08 15:22:54 Nils Klarlund <opensource@google.com> + + More missing test files. This time for computed includes. + + R=fergus,manos + APPROVED=manos + DELTA=21 (21 added, 0 deleted, 0 changed) + + * include_server/test_data/test_computed_includes/helper.c: add + * include_server/test_data/test_computed_includes/incl.h: add + * include_server/test_data/test_computed_includes/inclA.h: add + * include_server/test_data/test_computed_includes/src.c: add + * include_server/test_data/test_computed_includes/srcA.c: add + + +2007/08/08 00:58:27 Nils Klarlund <opensource@google.com> + + Fix the problem that stat's in the main 'Resolve' routine of + cache_basics were based on os.path.exists, not on os.path.isfile. Write + new test to verify that the behavior is now correct. Fix little problem + in compress_files.py + + R=fergus,manos + APPROVED=fergus,manos + DELTA=88 (66 added, 0 deleted, 22 changed) + + * include_server/c_extensions/distcc_pump_c_extensions_module.c: edit + * include_server/c_extensions_test.py: edit + * include_server/cache_basics.py: edit + * include_server/compress_files.py: edit + * include_server/include_server_test.py: edit + * include_server/test_data/dfoo/i_am_perhaps_a_directory.h: add + * include_server/test_data/i_am_perhaps_a_directory.h/empty_file: add + * include_server/test_data/test_directory_probing.c: add + + +2007/08/07 23:06:24 Nils Klarlund <opensource@google.com> + + 'make test' was failing --- this CL fixes that. + + R=fergus,manos + DELTA=1 (1 added, 0 deleted, 0 changed) + + * include_server/setup.py: edit + + +2007/08/07 23:06:00 Nils Klarlund <opensource@google.com> + + Make release subversion = pump6. + + R=fergus,manos + DELTA=31 (0 added, 0 deleted, 31 changed) + + * build-distcc.sh: edit + * configure: edit + * configure.ac: edit + * distcc/configure: edit + * distcc/configure.ac: edit + * include_server/configure: edit + * include_server/configure.ac: edit + + +2007/08/07 16:26:27 Manos Renieris <opensource@google.com> + + a) Include the server side error messages in the maintainer email + b) Don't output them on the screen. + + R=fergus,klarlund + DELTA=116 (107 added, 1 deleted, 8 changed) + + * distcc/src/bulk.c: edit + * distcc/src/bulk.h: edit + * distcc/src/clirpc.c: edit + * distcc/src/compile.c: edit + * distcc/src/compile.h: edit + * distcc/src/distcc.h: edit + * distcc/src/emaillog.c: edit + * distcc/src/emaillog.h: edit + * distcc/src/remote.c: edit + + +2007/08/07 16:20:57 Manos Renieris <opensource@google.com> + + For testing purposes, add a way to stub out the include server. + + R=fergus,klarlund + DELTA=73 (72 added, 1 deleted, 0 changed) + + * distcc/src/include_server_if.c: edit + * distcc/src/util.c: edit + * distcc/src/util.h: edit + + +2007/08/06 23:07:25 Nils Klarlund <opensource@google.com> + + Reintroduced 'introspection' so that an installed package can + be moved from say buildstaticrw to buildstatic with all references + to executables moving along. Added overide mechanism as well. + + Also, introduced a necessary check when running off source directory so + see whether there's more than on .so file. + + Finally, changed messages so that we're printing a message that the + include server has started instead of one indicating that is starting. + + R=fergus,manos + DELTA=74 (52 added, 9 deleted, 13 changed) + + * Makefile.in: edit + * pump.in: edit + + +2007/08/06 21:56:31 Nils Klarlund <opensource@google.com> + + Various minor changes in response to comment by Fergus for a previous + change. + + R=fergus + DELTA=12 (3 added, 0 deleted, 9 changed) + + * include_server/include_server.py: edit + + +2007/08/06 14:29:44 Nils Klarlund <opensource@google.com> + + Check that /dev/shm is actually writable and executable. Also, correct + two very, very minor mistakes. + + R=fergus,manos + DELTA=14 (4 added, 1 deleted, 9 changed) + + * include_server/basics.py: edit + * include_server/compress_files.py: edit + * include_server/include_server.py: edit + + +2007/08/02 16:15:06 Nils Klarlund <opensource@google.com> + + Add email notifications from include server. Delete email notifications + from distcc client when caused by include server providing an error + indication. + + Features: + - new command line flags for enabling emails for exception conditions + - a limit on the number of emails sent due to NotCovered exception + - stack trace is included + + Testing: + - tested new commnand line arguments + - tested for each of the three exception handlers where email is + sent that email is indeed being sent + - 'make test' for include_server + + R=fergus,manos + DELTA=106 (86 added, 2 deleted, 18 changed) + + * distcc/src/compile.c: edit + * include_server/basics.py: edit + * include_server/include_server.py: edit + * pump.in: edit + + +2007/08/02 14:40:27 Nils Klarlund <opensource@google.com> + + Add option -t or --time for total time spent in include server to be + printed on stderr. Example: + + Include server timing. Elapsed: 18.2s User: 0.1s System: 0.0s Total: 0.1s + + R=fergus,manos + DELTA=47 (32 added, 0 deleted, 15 changed) + + * include_server/basics.py: edit + * include_server/include_server.py: edit + + +2007/08/01 17:33:50 Nils Klarlund <opensource@google.com> + + + Make the handling of computed includes sound. Upto now, the include + server failed to reprocess a file F with the property: + - F transitively includes a file G + - G contains a computed include and the support of its expression + contains a symbol X + - X is defined or redefined after G is processed the first time + + We hope to compile OpenOffice with this fix. + + Little things: + + - Fixed include_analyzer_memoizing_node_test.py to allow regression test + for distcc itself to pass. + + - Removed dead code EvalExprDirs in include_analyzer.py. + + + Correctness testing: + + - In include_server: 'make test' works. + + - Function test_AdvancedComputedIncludes is added in + 'include_server/include_server_test.py'. It contains the basic + scenario we are correcting for. + + - We ran multiple tests 'make-dbg -r -j80 third_party/lzo'; every test + succeeded. These did not succeed with existing version. + + - We built gws:gws and localsearch/mustang + + Performance testing: + + For make-dbg gws:gws -g0, the best time before the change: + + make-dbg + + real 7m44.230s + user 7m0.494s + sys 2m25.385s + + Include server + Elapsed: 463.9s User: 72.5s System: 37.3s Total: 109.9s + + + The best time after: + + make-dbg + real 7m44.772s + user 6m46.249s + sys 2m2.944s + Include server + Elapsed: 464.6s User: 87.4s System: 36.9s User + System: 124.3s + + Thus, it seems that we've added 15s or about 8% to the include server + time. (These times are on a hyperthreaded machine and therefore hard to + interpret.) + + Remarks: the central algorithm still is in the need of refactoring. That + 8% running time was added because of the current change adds to my + impression that the refactoring must be done with care. + + R=fergus,manos + DELTA=316 (237 added, 25 deleted, 54 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_analyzer_memoizing_node_test.py: edit + * include_server/include_server_test.py: edit + * include_server/macro_eval.py: edit + * include_server/macro_eval_test.py: edit + * include_server/parse_file.py: edit + + +2007/07/30 17:10:04 Manos Renieris <opensource@google.com> + + Bump up the release number. + + R=klarlund,fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * build-distcc.sh: edit + + +2007/07/27 20:23:30 Manos Renieris <opensource@google.com> + + Push declaration to the top of block so that it's more portable. + + TBR=klarlund,fergus + DELTA=4 (2 added, 2 deleted, 0 changed) + + * distcc/src/compile.c: edit + * distcc/src/serve.c: edit + + +2007/07/27 19:40:59 Manos Renieris <opensource@google.com> + + Respect the second part of the DEPENDENCIES_OUTPUT env variable. + + R=fergus,klarlund + DELTA=79 (65 added, 5 deleted, 9 changed) + + * distcc/src/compile.c: edit + * distcc/src/distcc.h: edit + * distcc/src/dotd.c: edit + * distcc/src/dotd.h: edit + * distcc/src/serve.c: edit + + +2007/07/27 09:09:53 Nils Klarlund <opensource@google.com> + + This file went MIA. This CL will make 'make -C include_server test' pass. + + R=fergus,manos + DELTA=14 (14 added, 0 deleted, 0 changed) + + * include_server/test_data/more_macros.c: add + + +2007/07/26 21:25:25 Manos Renieris <opensource@google.com> + + Print the protocol description even if opt_numeric is true. + + R=fergus,klarlund + DELTA=4 (1 added, 0 deleted, 3 changed) + + * distcc/src/lsdistcc.c: edit + + +2007/07/26 15:39:00 Manos Renieris <opensource@google.com> + + Explicitly rewrite the .d file dependency on the server; + -MT does not work quite as advertised + (see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12448): + gcc adds the argument of -MT to the list of object files in + the dependencies, rather than replacing the default object file name + with the argument of -MT; dotd does not tolerate two object + files in the .d file. + + R=fergus,klarlund + DELTA=71 (59 added, 5 deleted, 7 changed) + + * distcc/src/compile.c: edit + * distcc/src/dotd.c: edit + * distcc/src/dotd.h: edit + * distcc/src/serve.c: edit + + +2007/07/26 15:07:34 Fergus Henderson <opensource@google.com> + + Add -Wuninitialized to CFLAGS. + + TESTED= Ran "autoconf && configure && make clean && make" and + verified that no new warnings were generated. + + R=manos + CC=distcc-pump + DELTA=2 (2 added, 0 deleted, 0 changed) + + * distcc/configure: edit + * distcc/configure.ac: edit + + +2007/07/25 11:28:33 Manos Renieris <opensource@google.com> + + Don't output the statistics for the doti file if there is no doti file. + + R=fergus,klarlund + DELTA=1 (0 added, 0 deleted, 1 changed) + + * distcc/src/remote.c: edit + + +2007/07/25 11:26:03 Manos Renieris <opensource@google.com> + + Fix a (serious) bug. sets_dotd was not initialized, and we were + not rewritting the dependency in the .d file. + + R=fergus,klarlund + DELTA=2 (1 added, 0 deleted, 1 changed) + + * distcc/src/compile.c: edit + * distcc/src/dotd.c: edit + + +2007/07/24 17:33:23 Manos Renieris <opensource@google.com> + + Always close the file descriptors after communicating with the + server, regardless of how soon we bail out. + + R=fergus,klarlund + DELTA=19 (9 added, 9 deleted, 1 changed) + + * distcc/src/remote.c: edit + + +2007/07/18 19:49:54 Manos Renieris <opensource@google.com> + + Don't try to send the preprocessed file to the server if the local + preprocessing failed. + + Otherwise, either the file sent to the server is wrong, or it does not + exist at all. In the first case, the result of the compilation is + wrong. In the second, trying to send an existing file is considered a + communication or remote error, and we end up thinking that the + remote host is down or in some other way unresponsive. + + Under the current fallback mode, this ends up printing the erro + messages from local preprocessing twice, once while its trying to + preprocess the file to send it to the server, and once while trying to + compile locally. I think that's ok. + + R=fergus,klarlund + DELTA=3 (3 added, 0 deleted, 0 changed) + + * distcc/src/remote.c: edit + + +2007/07/18 16:21:35 Manos Renieris <opensource@google.com> + + Added some notes on the implementation of protocol version 3. + + R=fergus,klarlund + DELTA=61 (61 added, 0 deleted, 0 changed) + + * distcc/doc/protocol-3-impl.txt: add + + +2007/07/13 14:22:13 Manos Renieris <opensource@google.com> + + Tiny change to use the phony feature directly for the pump target. + + R=klarlund,fergus + DELTA=5 (1 added, 3 deleted, 1 changed) + + * Makefile.in: edit + + +2007/07/12 15:17:54 Nils Klarlund <opensource@google.com> + + Addressing Fergus's comments for a previous change + more changes. + + R=fergus,manos + DELTA=45 (27 added, 6 deleted, 12 changed) + + * README: edit + +2007/07/10 17:18:31 Manos Renieris <opensource@google.com> + + Added a description for protocol version 3. + + R=klarlund,fergus + DELTA=82 (82 added, 0 deleted, 0 changed) + + * distcc/doc/protocol-3.txt: add + + +2007/07/09 16:01:36 Nils Klarlund <opensource@google.com> + + New subversion number for packages. + + R=fergus,manos + DELTA=1 (0 added, 0 deleted, 1 changed) + + * build-distcc.sh: edit + + +2007/07/09 13:50:11 Nils Klarlund <opensource@google.com> + + When using --verbose, the variable denoting the option level is not + touched. So, --verbose is not equivalent to setting the option level to + 'debug' as the manual says. This is fixed. + + R=fergus,manos + DELTA=1 (1 added, 0 deleted, 0 changed) + + * distcc/src/dopt.c: edit + + +2007/07/09 13:41:11 Nils Klarlund <opensource@google.com> + + Simplify memory allocation strategy in dcc_run_job. + + Tests: 'make valgrind-check'. + + R=fergus,manos + DELTA=22 (9 added, 3 deleted, 10 changed) + + * distcc/src/serve.c: edit + + +2007/07/05 14:38:09 Nils Klarlund <opensource@google.com> + + Fix server and include server to allow absolute translation unit. I + refactored a bit to cut down the length of dcc_run_job (which is still + too long). + + TESTING: I have verified that regexp-options.cc now compiles on pump + server. I have compiled several hundred other files. Also, I verified + that 'make valgrind-check' passes. + + Note: a problem occurred in an lsdistcc test. The output from lsdistcc + was not in the expected order. I changed the test to sort the output + before the comparison. + + R=fergus,manos + DELTA=241 (145 added, 76 deleted, 20 changed) + + * distcc/src/serve.c: edit + * distcc/test/testdistcc.py: edit + * include_server/include_analyzer.py: edit + + +2007/06/29 15:19:42 Nils Klarlund <opensource@google.com> + + Fix email address in comments and code by introducing an abstraction. + + R=fergus,manos + DELTA=7 (1 added, 0 deleted, 6 changed) + + * distcc/src/compile.c: edit + * distcc/src/emaillog.c: edit + * distcc/src/emaillog.h: edit + + +2007/06/29 15:15:39 Nils Klarlund <opensource@google.com> + + Fix problem that a stack trace was produced for Ctrl-C. In particular, + we discovered that printing out args[0] of the instance object is not + wise, because an index error may ensue. + + R=fergus,manos + DELTA=20 (9 added, 5 deleted, 6 changed) + + * include_server/include_server.py: edit + + +2007/06/28 18:30:54 Nils Klarlund <opensource@google.com> + + Remove offensive code. I'm now handling configuring of pump in an + auxiliary script, also called 'pump'. + + R=fergus,manos + DELTA=11 (0 added, 8 deleted, 3 changed) + + * Makefile.in: edit + * pump.in: edit + + +2007/06/28 15:51:09 Manos Renieris <opensource@google.com> + + Change the address we send errors to. + + R=fergus,klarlund + DELTA=1 (0 added, 0 deleted, 1 changed) + + * distcc/src/emaillog.c: edit + + +2007/06/26 22:03:23 Manos Renieris <opensource@google.com> + + Added -Pprotocol option, which will print out only the servers that + support (at least) the given protocol. + lsdistcc wll now print the protocol type out with the host + i.e. host.domain.com,lzo. + + R=fergus,klarlund + CC=distcc-pump + DELTA=134 (116 added, 9 deleted, 9 changed) + + * distcc/Makefile.in: edit + * distcc/src/lsdistcc.c: edit + + +2007/06/21 17:45:39 Manos Renieris <opensource@google.com> + + Tell the user why they are seeing error messages twice. + + R=fergus,klarlund + DELTA=3 (2 added, 0 deleted, 1 changed) + + * distcc/src/compile.c: edit + + +2007/06/21 16:40:25 Manos Renieris <opensource@google.com> + + Forgot emaillog.h. Oops! + + R=fergus,klarlund + DELTA=8 (8 added, 0 deleted, 0 changed) + + * distcc/src/emaillog.h: add + + +2007/06/21 13:04:17 Manos Renieris <opensource@google.com> + + Add email logging. + + R=klarlund,fergus + DELTA=170 (137 added, 15 deleted, 18 changed) + + * distcc/Makefile.in: edit + * distcc/src/compile.c: edit + * distcc/src/distcc.c: edit + * distcc/src/emaillog.c: add + * distcc/src/trace.c: edit + * distcc/src/traceenv.c: edit + + +2007/06/20 16:10:59 Nils Klarlund <opensource@google.com> + + Fix spelling error in Makefile.in, bump-up subversion number. + + R=fergus,manos + DELTA=2 (0 added, 0 deleted, 2 changed) + + * build-distcc.sh: edit + * include_server/Makefile.in: edit + + +2007/06/20 15:36:43 Nils Klarlund <opensource@google.com> + + Further enviroment variables for controlling behavior of include server + when invoked through pump. + + Also, we fixed the problem that for the common situation that configure + is run in the top level directory of the source tree it was impossible + to run 'pump'. + + The solution is to regard the script alteration by the Makefile rule for + 'pump' something that regards mainly the installed version. The source + tree version of pump finds its Python files in the source directory and + its .so file under the 'build/lib.xxx/include_server' subdirectory of th + scr directory. + + Finally, we took out the hardcoding for values of the potential distcc + servers. When the make rule for 'pump' is run, the environment variable + DISTCC_POTENTIAL_HOSTS with its current value is copied into 'pump' -- + if it is set. + + So, now I prepare distribution by doing: + + DISTCC_POTENTIAL_HOSTS="test-distcc1.ame test-distcc2.ame test-distcc3.ame test-distcc4.ame test-distcc5.ame pump1.ame pump2.ame pump3.ame pump4.ame pump5.ame pump6.ame pump7.ame pump8.ame pump9.ame distcc1" make pump + + This also of course makes the 'pump' script work in the source + directory, where it'll now pick up the servers I specified for the ad + hoc and regression tests I run. + + With this machinery, I was again able to easily test the other pending + changelists for the include server. + + + * Makefile.in: edit + * pump.in: edit + + +2007/06/20 01:38:51 Fergus Henderson <opensource@google.com> + + When aborting, make sure we exit with a non-zero exit status. + + TBR=klarlund + + * include_server/include_server.py: edit + + +2007/06/19 23:44:15 Nils Klarlund <opensource@google.com> + + Fix the problem that source was nuked by 'make clean'. + + R=manos,fergus + DELTA=6 (0 added, 1 deleted, 5 changed) + + * include_server/Makefile.in: edit + + +2007/06/19 23:41:55 Nils Klarlund <opensource@google.com> + + Reapplying after rollback. + Make --verify mode work with the include server and improve error + reporting. An internal error would now look like: + + WARNING: Preprocessing locally. Include server internal error: + 'exceptions.TypeError: expected string or buffer' for translation unit + 'maps/frontend/geocode_api_renderer.cc' + + We also fix a bug regarding TERM signal that would not properly terminate an + include server. + + R=fergus,manos + DELTA=58 (38 added, 2 deleted, 18 changed) + + * include_server/cache_basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/06/19 23:39:46 Nils Klarlund <opensource@google.com> + + + We fix several bugs in the include server. + + - The macro evaluator has been completely rewritten. In particular, it + can now be used to parse files in maps/frontend that involve computed + includes like: + + #include MAPS_TEMPLATE_VARNAMES(map_kml_results) + + - We also now parse macro calls on arguments that contain macro calls. + + - We implement the 'disabled' semantics of macro invocation (for + infinite recursion prevention). + + - We now apply # and ## only as part of expanding function-like macros. + + - We added several new tests, in part taken from the CPP Internals + document. + + - A test for ResolveExpr with pedagodical value has been added (and + another file for this test to work). + + - The macro evaluator has been thouroughly commented. + + - The parser has been updated so that it no longer thinks that '//' + within a filepath is a C++ comment. + + - The parser's handling of macro calls (and definitions) was + improved. Note that we now only use the parser's understanding of + macros for function-like defines, not for macro invocations, which are + treated in a new routine in macro_eval. + + R=fergus,manos + DELTA=531 (373 added, 68 deleted, 90 changed) + + * include_server/macro_eval.py: edit + * include_server/macro_eval_test.py: edit + * include_server/parse_file.py: edit + * include_server/parse_file_test.py: edit + * include_server/test_data/maps/foo.tpl.varnames.h: add + * include_server/test_data/parse.c: edit + + +2007/06/19 13:27:49 Manos Renieris <opensource@google.com> + + Update the rpm prefix to be /usr, because that's where + the init.d scripts expect it. + + R=klarlund,fergus + DELTA=4 (0 added, 0 deleted, 4 changed) + + * distcc.spec: edit + + +2007/06/18 23:10:26 Fergus Henderson <opensource@google.com> + + Give better error messages in cases where we can't determine the + compiler's default search path(s). Also don't try to terminate the + include server in those cases (it wasn't actually terminating anyway, + since the sys.exit() exception was being caught by the SocketServer + class, I think). + + R=karlund + CC=distcc-pump + DELTA=17 (9 added, 0 deleted, 8 changed) + + * include_server/compiler_defaults.py: edit + + +2007/06/15 15:07:45 Fergus Henderson <opensource@google.com> + + Another change towards getting "make check" working again. + + TBR=manos + + * include_server/Makefile.in: edit + + +2007/06/15 14:58:27 Fergus Henderson <opensource@google.com> + + Make the "pump" target depend on the "Makefile" target, + so that it gets rebuilt if you configure with a different + prefix. + This is in response to manos' verbal post-submit review + comments on a previous change. + + TBR=manos + + * Makefile.in: edit + + +2007/06/15 14:52:08 Nils Klarlund <opensource@google.com> + + This CL was submitted too early. + + Automated g4 rollback of a previous change: + + Make --verify mode work with the include server and improve error + reporting. An internal error would now look like: + + WARNING: Preprocessing locally. Include server internal error: + 'exceptions.TypeError: expected string or buffer' for translation unit + 'maps/frontend/geocode_api_renderer.cc' + + We fix a bug regarding TERM signal that would not properly terminate an + include server. + + + * include_server/cache_basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/06/15 14:35:39 Fergus Henderson <opensource@google.com> + + Fix bugs with the generation of the "pump" script from "pump.in": + - $ was not properly escaped and was being interpreted + by Make rather than the shell + - the pump.in script that is being copied could be read-only + and we did not (a) remove the old read-only copy nor (b) make + the new copy writable. + - pump should be a real target, not a phony target + - we need a dependency "all: pump" + + P.S. After syncing this change, you need to run "autoconf" (or + sync Manos' change which updated the configure script), + "configure", and then "make". + + TBR=manos + CC=distcc-pump + DELTA=5 (1 added, 0 deleted, 4 changed) + + * Makefile.in: edit + + +2007/06/15 14:33:52 Fergus Henderson <opensource@google.com> + + Temporary work-around for problem where "make clean" nukes the source + tree. + + TBR=manos,klarlund + + * include_server/Makefile.in: edit + + +2007/06/15 14:26:45 Fergus Henderson <opensource@google.com> + + Add support for running tests in lzo mode or in pump mode. + + R=manos + CC=distcc-pump + DELTA=38 (27 added, 2 deleted, 9 changed) + + * distcc/Makefile.in: edit + * distcc/test/testdistcc.py: edit + + +2007/06/15 13:34:48 Manos Renieris <opensource@google.com> + + Update the configure script. + Remove pump, which is now generated by make. + + R=klarlund,fergus + DELTA=272 (52 added, 219 deleted, 1 changed) + + * configure: edit + * pump: delete + + +2007/06/15 07:13:52 Nils Klarlund <opensource@google.com> + + Fix file permission to +x. + + R=manos + + * build-distcc.sh: edit + + +2007/06/14 22:51:35 Nils Klarlund <opensource@google.com> + + Make --verify mode work with the include server and improve error + reporting. An internal error would now look like: + + WARNING: Preprocessing locally. Include server internal error: + 'exceptions.TypeError: expected string or buffer' for translation unit + 'maps/frontend/geocode_api_renderer.cc' + + We fix a bug regarding TERM signal that would not properly terminate an + include server. + + R=fergus,manos + + * include_server/cache_basics.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + + +2007/06/14 17:21:22 Manos Renieris <opensource@google.com> + + Script and spec file for building rpm. + + R=fergus,klarlund + DELTA=490 (490 added, 0 deleted, 0 changed) + + * build-distcc.sh: add + * distcc.spec: add + + +2007/06/14 16:10:47 Manos Renieris <opensource@google.com> + + Add DESTDIR so that rpm building can find the pump script. + + R=fergus,klarlund + DELTA=6 (3 added, 0 deleted, 3 changed) + + * Makefile.in: edit + + +2007/06/14 14:06:05 Manos Renieris <opensource@google.com> + + Revamp the configure/make/make install process: + - the python building and installation is done by setup.py. + - include_server is now a python package. + - include_server/c_extensions is not buildable by itself anymore. + - include_server tests are build into their own directory. + - the pump script is "finished" at make time, before install. + - run.py is not needed anymore + + R=fergus,klarlund + DELTA=633 (456 added, 147 deleted, 30 changed) + + * Makefile.in: edit + * configure.ac: edit + * distcc/configure: edit + * distcc/configure.ac: edit + * include_server/Makefile.in: edit + * include_server/__init__.py: add + * include_server/c_extensions/Makefile.in: delete + * include_server/c_extensions/setup.py: delete + * include_server/c_extensions_test.py: edit + * include_server/configure: edit + * include_server/configure.ac: edit + * include_server/setup.py: add + * pump.in: add + + +2007/06/11 16:30:54 Nils Klarlund <opensource@google.com> + + Improving start-up time. It turns out that the include server + is not the bottleneck -- it's about 200ms to start it. But + lsdistcc is. So with change I'm just trying to save that + amount of time by starting the include server while doing lsdistcc. + + R=fergus,manos + DELTA=106 (70 added, 2 deleted, 34 changed) + + * pump: edit + + +2007/06/11 16:22:37 Nils Klarlund <opensource@google.com> + + + Fix "ERROR: failed to connect to UNIX-DOMAIN + /dev/shm/include-server: Resource temporarily + unavailable". The former assignment to + server.request_queue_size was ineffectual because this + parameter is used by the constructor for a server. The + parameter is passed to socket.listen. + + R=manos,fergus + DELTA=22 (17 added, 2 deleted, 3 changed) + + * include_server/include_server.py: edit + + +2007/06/08 18:15:16 Fergus Henderson <opensource@google.com> + + Some minor fixes to avoid warnings from gcc. + + Also a small bug fix: when cleanup_tempfiles is invoked from + a signal handler, we should not call free(), because it is + unsafe to call free() from an asynchronous signal handler + that could be executed in the middle of another call to malloc() + or free(). Also fix the use of volatile in cleanup_tempfiles. + + R=manos + CC=distcc-pump + DELTA=87 (42 added, 16 deleted, 29 changed) + + * distcc/src/cleanup.c: edit + * distcc/src/distcc.c: edit + * distcc/src/distcc.h: edit + * distcc/src/dsignal.c: edit + * distcc/src/trace.c: edit + * distcc/src/util.c: edit + + +2007/06/08 17:52:40 Fergus Henderson <opensource@google.com> + + Allow ".i" and ".ii" file names as inputs to the include server. + Also some improvements to the messages produced by the include server. + + R=klarlund + CC=distcc-pump + DELTA=26 (11 added, 7 deleted, 8 changed) + + * include_server/basics.py: edit + * include_server/include_server.py: edit + * include_server/parse_command.py: edit + * include_server/parse_file.py: edit + * include_server/run.py: edit + + +2007/06/08 16:40:16 Nils Klarlund <opensource@google.com> + + Improve warning messages regarding include server failures. + + R=fergus,manos + DELTA=3 (2 added, 0 deleted, 1 changed) + + * distcc/src/compile.c: edit + * distcc/src/include_server_if.c: edit + + +2007/06/07 21:41:25 Manos Renieris <opensource@google.com> + + Put all declarations before executable code, for compatibility with older + C compilers. + + R=fergus,klarlund + DELTA=15 (6 added, 7 deleted, 2 changed) + + * distcc/src/clirpc.c: edit + * distcc/src/include_server_if.c: edit + * distcc/src/srvrpc.c: edit + * include_server/c_extensions/distcc_pump_c_extensions_module.c: edit + + +2007/06/07 15:47:57 Nils Klarlund <opensource@google.com> + + Check that execution of gcc to find defaults succeeds and warns + appropriately if not. + + R=fergus,manos + DELTA=17 (11 added, 5 deleted, 1 changed) + + * include_server/compiler_defaults.py: edit + + +2007/06/07 15:41:55 Nils Klarlund <opensource@google.com> + + Fixed bugs from the autoconf conversions. Now, the include + server extension builds. + + R=fergus,manos + DELTA=8 (0 added, 3 deleted, 5 changed) + + * distcc/Makefile.in: edit + * include_server/c_extensions/setup.py: edit + + +2007/06/05 14:50:22 Nils Klarlund <opensource@google.com> + + Fixes: so that we can find the include server, also in + installation. Note that we erase previous value of PYTHONPATH + for performance reasons. + + R=fergus + DELTA=137 (91 added, 23 deleted, 23 changed) + + * include_server/run.py: edit + * pump: edit + + +2007/06/04 17:46:36 Nils Klarlund <opensource@google.com> + + This is to repair a previous change, which I submitted by + mistake. + This CL together with the previous CL fix bugs: + + - check that include_server_pid exists before killing it + + - don't strip \n before stuffing hosts into temporary variable + + Also fixes a couple of minor syntax issues in comments. + + R=manos,fergus + DELTA=4 (1 added, 0 deleted, 3 changed) + + * pump: edit + + +2007/06/04 16:04:51 Fergus Henderson <opensource@google.com> + + Fix a bug where we were attempting to use memory that had already been + freed. (I detected this by running the tests under valgrind.) + + R=manos + CC=distcc-pump + DELTA=19 (13 added, 0 deleted, 6 changed) + + * distcc/src/serve.c: edit + + +2007/06/04 14:53:03 Nils Klarlund <opensource@google.com> + + Fixed bug: 'opts' were appended to only last host. Fixed very + minor syntax issues in comments. + + + * pump: edit + + +2007/06/04 14:32:33 Fergus Henderson <opensource@google.com> + + 1. Use srcdir rather than top_srcdir to locate the mkinstalldirs script; + now that distcc_pump has its own autoconf-generated configure script, + top_srcdir is now "distcc_pump" rather than "distcc". + + 2. Add USE_VALGRIND variable, and use it in the maintainer-check rule, + so that you can do 'make USE_VALGRIND=--valgrind" maintainer-check' + to run the tests under valgrind. + + R=manos + CC=distcc-pump + DELTA=7 (5 added, 0 deleted, 2 changed) + + * distcc/Makefile.in: edit + + +2007/06/04 13:43:00 Manos Renieris <opensource@google.com> + + Make lsdistcc 80 columns wide. + + R=klarlund,fergus + DELTA=199 (113 added, 5 deleted, 81 changed) + + * distcc/src/lsdistcc.c: edit + + +2007/06/04 11:40:08 Fergus Henderson <opensource@google.com> + + Add support for running tests under valgrind. + + R=manos + CC=distcc-pump + DELTA=108 (59 added, 3 deleted, 46 changed) + + * distcc/test/testdistcc.py: edit + + +2007/06/04 11:31:24 Fergus Henderson <opensource@google.com> + + Add "install" targets to the makefiles. + + Use autoconf-generated configure scripts, in order to support + the standard configure options such as '--prefix', etc. + + R=klarlund + CC=distcc-pump + DELTA=4959 (4816 added, 110 deleted, 33 changed) + + * Makefile: delete + * Makefile.in: add + * configure: edit + * configure.ac: add + * include_server/Makefile: delete + * include_server/Makefile.in: add + * include_server/c_extensions/Makefile: delete + * include_server/c_extensions/Makefile.in: add + * include_server/configure: add + * include_server/configure.ac: add + + +2007/06/04 11:20:11 Fergus Henderson <opensource@google.com> + + Rewrite the pump script using functions. + Use lowercase names for shell variables that are local to this process. + Document the DISTCC_PUMP_LOCATION environment variable in the usage + message. + Ensure error messages go to stderr rather than stdout. + Fix some problems with the cleanup: the order of cleanup was wrong. + It was doing "rmdir $SOCKET_DIR" before "rm $SOCKET", which meant + the rmdir could fail since the directory isn't empty yet. + Check for "lsdistcc" in the same directory as "pump", if it isn't + found in the "distcc" subdirectory. + + R=klarlund + CC=manos,distcc-pump + DELTA=197 (119 added, 51 deleted, 27 changed) + + * pump: edit + + +2007/06/04 00:27:53 Fergus Henderson <opensource@google.com> + + Use gcc's automatic dependency generation. + + Without this change, the Makefile was missing dependencies on + all the header files, so editing say src/distcc.h and then + running "make" would not cause a recompile. + + R=manos + CC=distcc-pump + DELTA=5 (3 added, 0 deleted, 2 changed) + + * distcc/Makefile.in: edit + * distcc/configure: edit + * distcc/configure.ac: edit + + +2007/06/01 19:24:28 Nils Klarlund <opensource@google.com> + + Handshake mechanism for starting include server. + Introduce os.fork() in include server, so that the parent + returns right after starting the child. The child becomes the + real include server. Change options so as to allow the parent + to write the child pid to a file. That will allow an invoking + script to send SIGTERM to the include server later. + + Fix missing deletion of temp file in include_server. + + Also, fixed bug: the initialization of the temp directory was + done twice, from two different modules. + + R=fergus,manos + DELTA=168 (85 added, 8 deleted, 75 changed) + + * include_server/basics.py: edit + * include_server/include_server.py: edit + * pump: edit + + +2007/06/01 19:06:50 Nils Klarlund <opensource@google.com> + + Change file permission to +x. + + + * distcc/contrib/distccd-on-servers: edit + + +2007/06/01 19:04:28 Nils Klarlund <opensource@google.com> + + + New tests for include analyzer; corrections to DirNameCache. + + DirnameCache was hardcoded to return absolute directories -- + that was wrong. + + Also, define helper function RetrieveDirectoriesExceptSys in + cache_basic for test purposes. + + Make _CalculateIncludeClosureExceptSystem call Fillcache so as + to avoid prerequisite condition. + + Fix missing deletion of temp file in include_server. + + Fix another couple of minor bugs. + + R=fergus,manos + DELTA=331 (266 added, 21 deleted, 44 changed) + + * include_server/Makefile: edit + * include_server/cache_basics.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_analyzer_memoizing_node_test.py: add + * include_server/include_server_test.py: edit + * include_server/parse_command_test.py: edit + * include_server/run.py: edit + + +2007/06/01 15:48:51 Nils Klarlund <opensource@google.com> + + Fixes bug so that 'status' works as intended. Also fixes bugs + that meant that the return command was given a non-integer argument. + + R=fergus,manos + DELTA=17 (3 added, 0 deleted, 14 changed) + + * distcc/contrib/distccd-on-servers: edit + + +2007/06/01 13:22:00 Manos Renieris <opensource@google.com> + + Plug memory leaks in the server. + + R=klarlund,fergus + CC=distcc-pump + DELTA=111 (54 added, 26 deleted, 31 changed) + + * distcc/src/arg.c: edit + * distcc/src/serve.c: edit + * distcc/src/srvrpc.c: edit + + +2007/05/31 20:26:35 Nils Klarlund <opensource@google.com> + + Initial submission: script for handling distccd on servers. + + R=fergus,manos + DELTA=142 (142 added, 0 deleted, 0 changed) + + * distcc/contrib/distccd-on-servers: add + + +2007/05/31 19:14:53 Fergus Henderson <opensource@google.com> + + Change the distcc_pump Makefile and configure script to support + building with the build directory being different than the + source directory, e.g. + + mkdir obj + cd obj + ../configure + make + + R=klarlund + CC=distcc-pump + DELTA=144 (91 added, 14 deleted, 39 changed) + + * INSTALL: edit + * Makefile: edit + * configure: edit + * distcc/Makefile.in: edit + * include_server/Makefile: edit + * include_server/c_extensions/Makefile: edit + * include_server/c_extensions/setup.py: edit + * include_server/run.py: edit + + +2007/05/31 15:32:41 Manos Renieris <opensource@google.com> + + Retry connecting if we get EAGAIN. + + R=klarlund,fergus + DELTA=5 (4 added, 0 deleted, 1 changed) + + * distcc/src/clinet.c: edit + + +2007/05/30 20:13:37 Nils Klarlund <opensource@google.com> + + Continuing "Continuing cleanup of the code...", + we fix various tiny buglets and wipe unnecessary variables + from mirror-path. + + TESTED=(cd include_closure; make test) + + R=fergus + CC=csilvers + DELTA=24 (0 added, 17 deleted, 7 changed) + + * include_server/compiler_defaults.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/macro_eval.py: edit + * include_server/mirror_path.py: edit + * include_server/mirror_path_test.py: edit + + +2007/05/30 16:59:02 Craig Silverstein <opensource@google.com> + + Continuing cleanup of the code, mostly to get varnames to + be consistent. But also some functionality changes: + + 1) Separated out the realpath_map into two data structures, + one that canonicalizes the paths, and the other that keeps + the systemdir-prefix information. Both are now part of the + mondo-cache object. There's still a bit more work to be + done to get the systemdir-prefix cache working intuitively. + + 2) Added RelpathMapToIndex, which requires that its keys not + start with a /. Use this new class for includepaths, so we + get better error checking; we'll now correctly fail when we + see code like #include </usr/local/string.h>, which we + can't handle properly. + + 3) [smaller] We now fall back on the python equivalents if we + don't have the C-speedups for realpath and stat(). + + 4) [smaller] Modified the convenience functions to be per + data-type rather than per varialbe: we have SearchdirIndex + and CurrdirIndex rather than just DirectoryIndex. (Since + this function is never used, it doesn't really matter, + but...) + + These were more syntactic cleanup: + + 1) Use currdir everywhere to mean the current directory, to + match cache_basics.py. + + 2) I started some comment changes to be more consistent. For + instance, every time we had a realpath_map argument, I try + to say what a realpath_map is (in one line!) rather than + just say its type -- which is, admittedly, better than + nothing. Only small work done here; still lots more. + + 3) Some places are still using fp instead of includepath. + I've fixed up most, but for some I'm not sure that the fp + is an includepath exactly, so I'm waiting to clean those + up. + + 4) Several places would pass a list in to a routine by + reference, so the procedure would fill the passed-in list + rather than returning a new list. This can be more + efficient, but is harder to follow, especially in python + where pass-by-value is expected. Change to a more natural + control-flow in situations where efficiency isn't critical. + + I also corrected the following deviations from the style + guide: + + 1) (Almost) no "naked" exceptions. Before there was lots of + "try: os-operation except: die". Now we actually check + that it's an IOError or an OSError. We also now print out + the 'why' status as part of the error message. + + 2) Got rid of unused imports. + + 3) Made sure there was at least two spaces between code and + end-of-line comments. Still more places to fix this one... + + 4) Nixed trailing whitespace (at the end of each line). + + R=klarlund + DELTA=719 (297 added, 193 deleted, 229 changed) + + * include_server/basics.py: edit + * include_server/cache_basics.py: edit + * include_server/compiler_defaults.py: edit + * include_server/compress_files.py: edit + * include_server/include_analyzer.py: edit + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_server.py: edit + * include_server/include_server_test.py: edit + * include_server/macro_eval.py: edit + * include_server/macro_eval_test.py: edit + * include_server/mirror_path.py: edit + * include_server/mirror_path_test.py: edit + * include_server/parse_command_test.py: edit + * include_server/parse_file.py: edit + * include_server/parse_file_test.py: edit + + +2007/05/25 18:35:29 Nils Klarlund <opensource@google.com> + + + The sed -i command I ran to perform renaming according to + cache_basics.py tricked me; these files were not added to + my CL that took care of several little changes. I have + verified that + + make -C include_server test + + now works. + + R=fergus,csilvers + DELTA=8 (0 added, 0 deleted, 8 changed) + + * include_server/include_analyzer_memoizing_node.py: edit + * include_server/include_server_test.py: edit + * include_server/mirror_path_test.py: edit + + +2007/05/25 17:53:45 Nils Klarlund <opensource@google.com> + + Installation and use improvements. + + Now the 'pump' wrapper is generic; no-hardcoded hosts, but a + an environment variable must be set. Also, it uses lsdistcc to + discover which potential hosts are actually ready. + + The INSTALL instructions are still rudimentary, but improved. + + The 'configure' script simply calls 'configure' in distcc subdir. + + R=fergus,csilvers + DELTA=82 (47 added, 33 deleted, 2 changed) + + * INSTALL: edit + * configure: add + * pump: edit + + +2007/05/25 16:05:37 Nils Klarlund <opensource@google.com> + + Add test targets. + + R=fergus + DELTA=26 (26 added, 0 deleted, 0 changed) + + * include_server/Makefile: edit + + +2007/05/25 15:49:30 Nils Klarlund <opensource@google.com> + + It turns out that the gcc system path discovery command + creates a temporary file. Make tempdir to overcome this problem. + + R=fergus,manos + DELTA=14 (4 added, 0 deleted, 10 changed) + + * include_server/compiler_defaults.py: edit + + +2007/05/25 15:48:56 Nils Klarlund <opensource@google.com> + + Make safer tempdir mechanism. + + R=fergus + DELTA=4 (1 added, 0 deleted, 3 changed) + + * include_server/basics.py: edit + + +2007/05/25 15:47:27 Nils Klarlund <opensource@google.com> + + Lots of little changes for final integration. One theme: names + changes. + + R=fergus,manos + DELTA=145 (51 added, 26 deleted, 68 changed) + + * include_server/include_analyzer.py: edit + * include_server/include_server.py: edit + * include_server/macro_eval_test.py: edit + * include_server/mirror_path.py: edit + * include_server/parse_command_test.py: edit + * include_server/parse_file.py: edit + * include_server/parse_file_test.py: edit + * include_server/statistics.py: edit + * include_server/test_data/gws-main.o.cmd: edit + + +2007/05/25 15:17:51 Nils Klarlund <opensource@google.com> + + This CL should have accompanied "This CL is in response to csilvers's questions.." + + + * include_server/parse_command.py: edit + + +2007/05/25 15:15:43 Nils Klarlund <opensource@google.com> + + Initial submission: "Compress files in an include closure." + + + * include_server/compress_files.py: add + + +2007/05/25 13:47:00 Nils Klarlund <opensource@google.com> + + A unittest for module mirror_path, featuring dependency + injection. This version is working. It replaces something that didn't. + + R=fergus,manos + DELTA=121 (87 added, 3 deleted, 31 changed) + + * include_server/mirror_path_test.py: edit + + +2007/05/25 11:38:56 Nils Klarlund <opensource@google.com> + + Initial submit of module to compute approximation of + include dependencies for C/C++. Also, some accompanying + changes to basics.py + + + * include_server/basics.py: edit + * include_server/include_server.py: add + * include_server/include_server_test.py: add + + +2007/05/25 11:29:51 Nils Klarlund <opensource@google.com> + + Wrapper script for distcc-pump test users. + + R=fergus,manos + DELTA=80 (80 added, 0 deleted, 0 changed) + + * pump: add + + +2007/05/24 21:49:42 Fergus Henderson <opensource@google.com> + + Add -r<PORT> option to lsdistcc to specify which port to connect + to. ("-p", "-o", and "-t" were already taken.) + Fix a bug where "lsdistcc distcc%d" was only returning the first host. + Add some unit tests for lsdistcc. + Also tidy up the usage message a little. + + R=manos + CC=distcc-pump,dkegel,aaw + DELTA=57 (47 added, 0 deleted, 10 changed) + + * distcc/src/lsdistcc.c: edit + * distcc/test/testdistcc.py: edit + + +2007/05/24 18:12:30 Fergus Henderson <opensource@google.com> + + Fix a bug that broke "cd distcc; configure; make": + move definition of the "echo_common_obj" target to the end of the file, + so that it doesn't become the default target (Make treats the first + rule in the file as the default target). + + R=klarlund + CC=distcc-pump + DELTA=10 (5 added, 5 deleted, 0 changed) + + * distcc/Makefile.in: edit + + +2007/05/24 14:17:43 Nils Klarlund <opensource@google.com> + + Fix "Usage" logic and printing error. Improve error reporting + when path to execute is not found. + + R=fergus,manos + DELTA=12 (7 added, 0 deleted, 5 changed) + + * include_server/run.py: edit + + +2007/05/24 09:14:17 Nils Klarlund <opensource@google.com> + + Tests for distcc_pump_c_extensions. Writes out doc strings and + calls some distcc rpc functions. Also, times the speed-up of + using libc versions of os.path.realpath and os.path.exists. + + R=fergus,manos + DELTA=115 (115 added, 0 deleted, 0 changed) + + * include_server/c_extensions_test.py: add + + +2007/05/24 09:11:50 Nils Klarlund <opensource@google.com> + + A file used for testing parse_command.py. Should not be part + of open source release. + + + * include_server/test_data/gws-main.o.cmd: add + + +2007/05/24 09:03:57 Nils Klarlund <opensource@google.com> + + Wrapper for running include server and tests. Initial submission. + + R=fergus,manos + DELTA=49 (49 added, 0 deleted, 0 changed) + + * include_server/run.py: add + + +2007/05/23 23:48:11 Nils Klarlund <opensource@google.com> + + directory and link structure." + + R=fergus,manos + DELTA=159 (159 added, 0 deleted, 0 changed) + + * include_server/mirror_path.py: add + * include_server/mirror_path_test.py: add + + +2007/05/23 23:12:36 Nils Klarlund <opensource@google.com> + + Initial submission: CPP macro evaluation. This admittedly needs a little + more work as explained. But there are some tests. + + R=manos,fergus + DELTA=338 (338 added, 0 deleted, 0 changed) + + * include_server/macro_eval.py: add + * include_server/macro_eval_test.py: add + + +2007/05/23 22:31:52 Nils Klarlund <opensource@google.com> + + + Initial submission. + + include_analyzer: "The skeleton, including caches, of an + include analyzer." + + include_analyzer_memoizing_node: "An graph-based algorithm + for memoizing include closure calculations. + + R=fergus,csilvers + DELTA=545 (545 added, 0 deleted, 0 changed) + + * include_server/include_analyzer.py: add + * include_server/include_analyzer_memoizing_node.py: add + + +2007/05/23 19:01:57 Craig Silverstein <opensource@google.com> + + Augment comments and normalize variable names. In addition, + clean up the code a bit, mostly by removing redundant code. + That said, with the variable-name changing, almost every line + in this file is affected by this CL. + + The file now is at a point where it makes sense to me!, though + I am perhaps too close to the algorithm these days to be a + good 'outside observer' judge. I think it's more consistent + in its naming, in any case. + + That said, I left in many TODOs where I still need some + clarification of functionality, or have cleanups to do + later (I tried not to change the functionality too much). + + Some outside-visible function/variable names may have changed + as a result of this CL, so other changes may be needed to get + the system working again after this. + + R=klarlund + DELTA=411 (108 added, 35 deleted, 268 changed) + + * include_server/cache_basics.py: edit + + +2007/05/22 15:25:09 Nils Klarlund <opensource@google.com> + + + This bunch of files is used for various tests of the include + server. I submit them as a bundle for further perusal with the + various *_test.py code. + + R=fergus,manos + DELTA=52 (52 added, 0 deleted, 0 changed) + + * include_server/test_data/abc_post.c: add + * include_server/test_data/abc_pre.c: add + * include_server/test_data/computed_includes.c: add + * include_server/test_data/dbar/dbar1/bar.h: add + * include_server/test_data/dbar/foo.h: add + * include_server/test_data/dfoo/foo.h: add + * include_server/test_data/dfoo/foo2.h: add + * include_server/test_data/dfoo/include_next_foo.h: add + * include_server/test_data/include_include_next_foo.h: add + * include_server/test_data/p1.h: add + * include_server/test_data/parse.c: add + * include_server/test_data/test_include_next/bar/x.h: add + * include_server/test_data/test_include_next/bar/y.h: add + * include_server/test_data/test_include_next/baz/start_x.c: add + * include_server/test_data/test_include_next/baz/start_y.c: add + * include_server/test_data/test_include_next/baz/x.h: add + * include_server/test_data/test_include_next/baz/y.h: add + * include_server/test_data/test_include_next/biz/x.h: add + * include_server/test_data/test_include_next/foo/x.h: add + * include_server/test_data/test_include_next/foo/y.h: add + + +2007/05/21 18:37:57 Manos Renieris <opensource@google.com> + + Change the test regexp to match the version text. + + R=fergus,klarlund + DELTA=4 (2 added, 0 deleted, 2 changed) + + * distcc/test/testdistcc.py: edit + + +2007/05/21 18:29:41 Nils Klarlund <opensource@google.com> + + Initial submission. "How to find system path defaults for gcc." + + R=fergus,manos + APPROVED=fergus + DELTA=136 (136 added, 0 deleted, 0 changed) + + * include_server/compiler_defaults.py: add + + +2007/05/21 17:01:16 Nils Klarlund <opensource@google.com> + + Initial submissin. Statistics gathering for the distcc-pump include server. + + R=fergus,manos + DELTA=110 (110 added, 0 deleted, 0 changed) + + * include_server/statistics.py: add + + +2007/05/21 12:09:10 Nils Klarlund <opensource@google.com> + + + This CL is in response to csilvers's questions about the use + of indices and caches in the include server. I've tried to + make the terminology coherent and to make the distinction + between indexed and string representation clearer. Also, I + moved the actual use of the classes into function SetUpCaches + that constructs the intended hierarchy for memoization. + + Now: also includes parse_command. + + R=fergus,csilvers + CC=manos + DELTA=410 (184 added, 40 deleted, 186 changed) + + * include_server/cache_basics.py: edit + + +2007/05/21 11:59:41 Manos Renieris <opensource@google.com> + + Initialize needs_dotd. + + R=klarlund,fergus + DELTA=1 (0 added, 0 deleted, 1 changed) + + * distcc/src/compile.c: edit + + +2007/05/21 11:59:13 Manos Renieris <opensource@google.com> + + Fix the cleanup function. + + R=klarlund,fergus + DELTA=3 (0 added, 0 deleted, 3 changed) + + * distcc/src/cleanup.c: edit + + +2007/05/18 17:22:48 Manos Renieris <opensource@google.com> + + Added the new files and one missing file to the Makefile. + + R=klarlund,fergus + DELTA=8 (3 added, 0 deleted, 5 changed) + + * distcc/Makefile.in: edit + + +2007/05/18 11:37:55 Manos Renieris <opensource@google.com> + + Undo the local_cpu lock if we are using the include server. + + R=fergus,klarlund + DELTA=6 (5 added, 0 deleted, 1 changed) + + * distcc/src/compile.c: edit + + +2007/05/17 18:21:26 Nils Klarlund <opensource@google.com> + + Initial check-in of directives-only C/C++ parser. + + R=manos,fergus + DELTA=395 (395 added, 0 deleted, 0 changed) + + * include_server/parse_file.py: add + * include_server/parse_file_test.py: add + + +2007/05/17 17:27:24 Nils Klarlund <opensource@google.com> + + This change accompanies a previous change. It allows the extension + module to extract the needed .o files from the distcc Makefile. + + R=fergus,manos + DELTA=5 (5 added, 0 deleted, 0 changed) + + * distcc/Makefile.in: edit + + +2007/05/17 16:15:36 Nils Klarlund <opensource@google.com> + + This CL enables us to compile distcc and to install the Python + extension module that provides access to some distcc rpc + functions as well as lzo. It no longer uses the Python + interface that is part of the lzo distribution. + + The CL also include some initial attempts a Makefile's. + + R=fergus,manos + DELTA=467 (467 added, 0 deleted, 0 changed) + + * INSTALL: add + * Makefile: add + * include_server/Makefile: add + * include_server/c_extensions/Makefile: add + * include_server/c_extensions/distcc_pump_c_extensions_module.c: add + * include_server/c_extensions/setup.py: add + + +2007/05/17 11:09:39 Nils Klarlund <opensource@google.com> + + + Include server for distcc-pump. The basic stuff. Initial submit. + The basic caching mechanisms for distcc-pump. Initial submit. + + R=manos,fergus + DELTA=596 (596 added, 0 deleted, 0 changed) + + * include_server/basics.py: add + * include_server/cache_basics.py: add + + +2007/05/16 17:03:03 Fergus Henderson <opensource@google.com> + + Update the output from "distcc --version". + 1. src/help.c: mention that it supports version 3 of the protocol, + i.e. distcc_pump + 2. configure.ac: change version to "distcc 2.18.3-13gg-pump1" + rather than "distcc 2.18.3", i.e. include both + a. "-13gg" to show it has all the existing Google patches, and + b. "-pump1" for the distcc_pump extensions that support + preprocessing on the server. + 3. configure: regenerate based on the new configure.ac. + This resulted in some slight differences due to different + autoconf versions, but they appear innocuous. + + TESTED=Ran make and manually verified the output of "distcc --version" + and "distccd --version". + Tried running "make maintainer-check"; that failed, but I think + the failures are unrelated to my change (probably they're due + to problems with the merge of distcc-pump with the Google distcc + patches). + + R=manos + CC=distcc-pump + DELTA=135 (24 added, 86 deleted, 25 changed) + + * distcc/configure: edit + * distcc/configure.ac: edit + * distcc/src/help.c: edit + + +2007/05/16 15:04:29 Manos Renieris <opensource@google.com> + + Change the name of the temp directories to reflect that + they are server-side. + + R=fergus,klarlund + DELTA=1 (0 added, 0 deleted, 1 changed) + + * distcc/src/tempfile.c: edit + + +2007/05/14 17:16:41 Craig Silverstein <opensource@google.com> + + Rewrite the argument-parser to be more efficient: this code + now runs about 50% faster than it used to (tested by + time_ParseCommandArguments.py). + + I can't totally vouch for the fact the output is the same now + as before, since there's not a complete test for this code yet, + but it seems to be on the one example I tried. + + Also, a few TODOs remain to deal with some obscure cases I + didn't understand (how does "-iquote=foo" differ from "-iquote + foo"?). But the basics should be right on. + + R=klarlund,fergus + DELTA=295 (153 added, 76 deleted, 66 changed) + + * include_server/parse_command.py: edit + * include_server/parse_command_test.py: edit + + +2007/05/14 17:05:15 Manos Renieris <opensource@google.com> + + Improve the robustness of cleanup. + + R=klarlund,fergus + DELTA=33 (18 added, 1 deleted, 14 changed) + + * distcc/src/cleanup.c: edit + * distcc/src/srvrpc.c: edit + * distcc/src/tempfile.c: edit + + +2007/05/14 11:37:26 Manos Renieris <opensource@google.com> + + Mark the top tmp directory for deletion. + + R=fergus,klarlund + DELTA=3 (3 added, 0 deleted, 0 changed) + + * distcc/src/tempfile.c: edit + + +2007/05/14 11:22:04 Manos Renieris <opensource@google.com> + + Close the file descriptors after being done talking to + the include server. + + R=klarlund,fergus + DELTA=14 (11 added, 1 deleted, 2 changed) + + * distcc/src/include_server_if.c: edit + + +2007/05/10 10:04:32 Nils Klarlund <opensource@google.com> + + The routines for interpreting a gcc command, along with tests. + Two issues still to be resolved: + - This code is too slow. + - For computed includes, we need to parse -D etc. + + R=fergus,manos + DELTA=518 (518 added, 0 deleted, 0 changed) + + * include_server/parse_command.py: add + * include_server/parse_command_test.py: add + + +2007/05/09 23:13:18 Fergus Henderson <opensource@google.com> + + Update Makefile.in to fix some bugs introduced in CL a previous change (which just + applied the Google and LSB-specific patches). Some object files had been + moved into $(common_obj), but were still listed in h_*_obj, resulting + in duplicate symbol errors when running "make maintainer-check". + + R=manos + CC=dkegel,klarlund + DELTA=3 (0 added, 1 deleted, 2 changed) + + * distcc/Makefile.in: edit + + +2007/05/09 19:12:09 Manos Renieris <opensource@google.com> + + A bit more const-correctness. + + R=fergus + DELTA=6 (2 added, 0 deleted, 4 changed) + + * distcc/src/serve.c: edit + + +2007/05/09 18:41:10 Manos Renieris <opensource@google.com> + + Distcc-pump first implementation. + + Various little changes, making functions from static into + global, all around the code; a bug fix in argutil.c. + + Important changes are: + - On the protocol, which is extended as "version 3", that + supports compression and also cpp on the server. + - distcc.h: Added the enum for where to do the cpp. + - hosts.c: Map a set of features (currently a subset of + where to do cpp and whether to do compression) onto + a protocol number. + - rpc.c: The client sometimes sends either a file, or + a link to the server. The server needs to be able to + accept either, so dcc_r_sometoken_int allows some + alternation to the protocol. + - Talking to the include server: + - compile.c: dcc_build_somewhere tries to talk to the + include server, and pushes the information about + the dotd file to dcc_compile_remote. This is where + the fallback-to-the-original-protocol code lives. + - include_server_if.c: Talk to the include server. + - The communication from client to server. The client needs to + send multiple files, and perhaps links, to the server; the + client also needs to sent its current working directory, + - remote.c: Entry points for sending multiple files to + the server. + - rpc.c: The server needs to handle not only files, but also + links (directories are handled implicitly). + - clirpc.c: Added a function to send the current working + directory. Added a function, dcc_x_many_files. It sends + multiple files to the server. In addition, it changes + the filenames to chop off the additions of the include + server. + - serve.c: Added a function to receive the current working + directory. + - On the server side, the server receives multiple files, + and the working directory, builds a directory structure (which + it keeps of, to clean it up at the end) + - srvrpc.c: + Added a function, dcc_r_many_files, to receive multiple + files. In addition, it changes the filenames to put all + the files under a "fake root" directory. + - serve.c: lots of work to set up "fake root" directories, + chdir to the right place, produce a .d file, and + absolutize include directives. + - tempfile.c: Creation of whole temp directories, + not just files. + - cleanup.c: Since we are receiving multiple files, + we need to keep a dynamic list of what temporary + files we create. It can now handle directories. + - ".d" files: the client needs to find out where the dotd file + should go; the server needs to produce it, remove all + references to the local filesystem (gcc can't do + that from the outside) and send it back to the client. + - dotd.c: Most of the code that handles the dotd: figuring + out if we need it, and where it should go; also, + cleaning it up after it gets created on the server. + - clirpc.c Changed dcc_retrieve_results to retrieve + the dotd file. + + R=klarlund,fergus + DELTA=1208 (1058 added, 36 deleted, 114 changed) + + * distcc/Makefile.in: edit + * distcc/README.google: add + * distcc/src/arg.c: edit + * distcc/src/argutil.c: edit + * distcc/src/bulk.c: edit + * distcc/src/bulk.h: edit + * distcc/src/cleanup.c: edit + * distcc/src/clinet.c: edit + * distcc/src/clinet.h: edit + * distcc/src/clirpc.c: edit + * distcc/src/compile.c: edit + * distcc/src/compile.h: edit + * distcc/src/compress.c: edit + * distcc/src/distcc.h: edit + * distcc/src/dotd.c: add + * distcc/src/dotd.h: add + * distcc/src/filename.c: edit + * distcc/src/hosts.c: edit + * distcc/src/hosts.h: edit + * distcc/src/include_server_if.c: add + * distcc/src/include_server_if.h: add + * distcc/src/lock.c: edit + * distcc/src/remote.c: edit + * distcc/src/rpc.c: edit + * distcc/src/rpc.h: edit + * distcc/src/serve.c: edit + * distcc/src/srvrpc.c: edit + * distcc/src/tempfile.c: edit + + +2007/05/08 17:53:54 Manos Renieris <opensource@google.com> + + A few "\n\" were missing. + + R=fergus,klarlund + DELTA=4 (0 added, 0 deleted, 4 changed) + + * distcc/src/lsdistcc.c: edit + + * Thu Jun 14 2007 Manos Renieris <opensource@google.com> - Added all the distcc-pump related parts to distcc.spec. - Changed the way the package is built. - Remove source package generation. - Man pages are now unzipped. - -* Mon May 29 2007 Dongmin Zhang <opensource@google.com> -- Add -r<PORT> option to lsdistcc to specify which port to connect to. - ("-p", "-o", and "-t" were already taken.) -- Fix a bug where "lsdistcc distcc%d" was only returning the first host. -- Add some unit tests for lsdistcc. -- Also tidy up the usage message a little. * Mon Feb 8 2007 Dongmin Zhang <opensource@google.com> - Added hosts list option to lsdistcc, such that lsdistcc can check only the |