summaryrefslogtreecommitdiff
path: root/test/Parallel
diff options
context:
space:
mode:
authorSteven Knight <knight@baldmt.com>2008-04-30 05:39:46 +0000
committerSteven Knight <knight@baldmt.com>2008-04-30 05:39:46 +0000
commitd5061a1effee5927efc7a6acd55443fa00218a78 (patch)
treef3178f3f085f178e22d840046acc862e23422870 /test/Parallel
parent1a10d4bc650741b2e279321ad32948ecba127c1c (diff)
downloadscons-d5061a1effee5927efc7a6acd55443fa00218a78.tar.gz
Merged revisions 2898-2901,2903-2927 via svnmerge from
http://scons.tigris.org/svn/scons/branches/core ........ r2903 | stevenknight | 2008-04-22 08:23:52 -0700 (Tue, 22 Apr 2008) | 3 lines Issue 2019: don't blow up when a "dictionary" source_scanner doesn't have a more-specific scanner entry for a specified source file's suffix. ........ r2904 | stevenknight | 2008-04-23 11:13:19 -0700 (Wed, 23 Apr 2008) | 3 lines Fix regression in use of $CXXFLAGS by the default $SHCXXFLAGS setting. (Greg Noel) ........ r2905 | stevenknight | 2008-04-23 11:23:12 -0700 (Wed, 23 Apr 2008) | 3 lines Update the test/CXX/SHCXXFLAGS.py with some more modern idioms copied from test/CXX/CXXFLAGS.py. ........ r2921 | belley | 2008-04-28 08:52:13 -0700 (Mon, 28 Apr 2008) | 98 lines Candidate list must be flushed when stopping the taskmaster. ============================================================ The taskmaster's candidate list must be flushed when the taskmaster is stopped, either because of a build error or a keyboard interrupt. This involves failing each candidates along with their waiting parents. This is necessary because the waiting parents might be in the pending_children set() that is used by the taskmaster to determine if cycles exist in the dependency graph. Failure to do so leads to confusing error messages such as: scons: *** Found dependency cycle(s): Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/DynamicProperty.h (<SCons.Node.FS.File instance at 0x95d230c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/NotSupported.h (<SCons.Node.FS.File instance at 0x960634c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/InvalidRequest.h (<SCons.Node.FS.File instance at 0x95d2bac>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/RequestCanceled.h (<SCons.Node.FS.File instance at 0x960648c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/InvalidType.h (<SCons.Node.FS.File instance at 0x95d2cec>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Core/InvalidProperty.h (<SCons.Node.FS.File instance at 0x9606d2c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/SecurityError.h (<SCons.Node.FS.File instance at 0x96065cc>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Core/PropertyCollector.h (<SCons.Node.FS.File instance at 0x9606e6c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/Fault/InvalidArgument.h (<SCons.Node.FS.File instance at 0x95d2a6c>) Internal Error: no cycle found for node /bld/vmkernel-main/scons/build/vmodl/obj/generic/Vmodl/Vmomi/DynamicData.h (<SCons.Node.FS.File instance at 0x95d21cc>) Changes made: Job.py: ------ o Implemented an InterruptState object that allows the sharing of the interrupt state between the Jobs, the Serial/Parallel and the Worker thread object. This allows for somewhat faster interruption when Ctrl-C is pressed because the Worker thread will not start any new task, i.e. the ones siting in the request queue. Taskmaster.py: -------------- o Moved the code contained the function Task.fail_continue() to the function Taskmaster.will_not_build(). This function perform the clean-up for nodes that will never be built. This includes failing these nodes along with their waiting parents and removing them from the list of pending children. o Improved the taskmaster tracing so that the pending children ref counts of nodes is also printed. o When stopping the taskmaster, no_next_candidate() now calls will_not_build() repetitively to make sure that all candidate nodes that will never be build are properly cleaned-up. test/Parallel/multiple-parents.py --------------------------------- Improved the test to also exercise the following functionality: o Nodes with multiple targets (not just aliases as it was before). o SideEffect() o Interrupting the build. Submitted against changeset 2920. All tests pass with both Python 2.5.1 and 1.5.2 Benoit ........ r2922 | stevenknight | 2008-04-28 11:50:10 -0700 (Mon, 28 Apr 2008) | 26 lines Issue 2024: http://scons.tigris.org/issues/show_bug.cgi?id=2024 Some public packages were importing specific SCons.Options.*Option submodules directly, instead of using the SCons.Options package itself (which actually exposed all the SCons.Options.*Option variable names directly, but what they hey). Our scripts didn't test for direct importing of the submodules, so this broke when I implemented backwards comptability with a SCons/Options.py module, instead of a full SCons/Options/*.py package. This restores the SCons/Options/*.py package files with stub modules that provide backwards compatibility (by referring to the new SCons.Variables functions and classes) and allow for importing all of The test/Deprecated/Options/*Option.py scripts have had specific submodule imports added. To make sure this corresponding "feature" continues to work in the future, I'll be following this with another checkin that adds submodule imports for the SCons.Variables.*Variable modules. I ran this with the SCons/Options.py file in place, and also with a compiled SCons/Options.pyc file in place, to try to make sure it works regardless. We've seen issues on Debian where their packaging doesn't clean up old *.pyc files when a Python *.py module is removed. It looks like this should work regardless of whether the Options.pyc file is there or not. ........ r2923 | stevenknight | 2008-04-28 11:52:52 -0700 (Mon, 28 Apr 2008) | 3 lines Update the test/Variables/*Variable.py scripts to verify that importing SCons.Variables.*Variable submodules continues to work in the future. ........ r2924 | belley | 2008-04-28 14:06:54 -0700 (Mon, 28 Apr 2008) | 6 lines Fixed a test that my morning change broke. The taskmaster tracing now includes the pending children ref count. Benoit ........ r2925 | stevenknight | 2008-04-28 18:00:25 -0700 (Mon, 28 Apr 2008) | 4 lines Issue 2028: Make sure already-existing Entry Nodes that we find are converted to Dir Nodes when searching a *PATH list for implicit dependencies. ........ r2926 | stevenknight | 2008-04-29 08:11:40 -0700 (Tue, 29 Apr 2008) | 3 lines Issue 2034: Fix $FORTRANMODDIR values like ${TARGET.dir} in the default $_FORTANMODFLAG expansion. (Stefano) ........ r2927 | stevenknight | 2008-04-29 11:34:02 -0700 (Tue, 29 Apr 2008) | 3 lines Restore the SCons.Options package (maintains backwards compatibility of importing submodules) to setup.py. ........
Diffstat (limited to 'test/Parallel')
-rw-r--r--test/Parallel/multiple-parents.py89
1 files changed, 79 insertions, 10 deletions
diff --git a/test/Parallel/multiple-parents.py b/test/Parallel/multiple-parents.py
index f81adf59..d9c414ff 100644
--- a/test/Parallel/multiple-parents.py
+++ b/test/Parallel/multiple-parents.py
@@ -50,34 +50,69 @@ test = TestSCons.TestSCons()
# c) Some targets succeed building
# d) Some children are ignored
# e) Some children are pre-requesites
-# f) Some sources are missing
+# f) Some children have side-effects
+# g) Some sources are missing
+# h) Builds that are interrupted
test.write('SConstruct', """
+opts = Options()
+opts.Add( BoolOption('interrupt', 'Interrupt the build.', 0 ) )
+optEnv = Environment(options=opts)
+
def fail_action(target = None, source = None, env = None):
return 2
+def simulate_keyboard_interrupt(target = None, source = None, env = None):
+ # Directly invoked the SIGINT handler to simulate a
+ # KeyboardInterrupt. This hack is necessary because there is no
+ # easy way to get access to the current Job/Taskmaster object.
+ import signal
+ handler = signal.getsignal(signal.SIGINT)
+ handler(signal.SIGINT, None)
+ return 0
+
+interrupt = Command(target='interrupt', source='', action=simulate_keyboard_interrupt)
+
+
failed0 = Command(target='failed00', source='', action=fail_action)
-ok0 = Command(target='ok00', source='', action=Touch('${TARGET}'))
+ok0 = Command(target=['ok00a', 'ok00b', 'ok00c'],
+ source='',
+ action=[Touch('${TARGETS[0]}'), Touch('${TARGETS[1]}'), Touch('${TARGETS[2]}')])
prereq0 = Command(target='prereq00', source='', action=Touch('${TARGET}'))
ignore0 = Command(target='ignore00', source='', action=Touch('${TARGET}'))
igreq0 = Command(target='igreq00', source='', action=Touch('${TARGET}'))
missing0 = Command(target='missing00', source='MissingSrc', action=Touch('${TARGET}'))
+withSE0 = Command(target=['withSE00a', 'withSE00b', 'withSE00c'],
+ source='',
+ action=[Touch('${TARGETS[0]}'), Touch('${TARGETS[1]}'), Touch('${TARGETS[2]}'),
+ Touch('side_effect')])
+SideEffect('side_effect', withSE0)
-prev_level = failed0 + ok0 + ignore0
+prev_level = failed0 + ok0 + ignore0 + missing0 + withSE0
prev_prereq = prereq0
prev_ignore = ignore0
prev_igreq = igreq0
+if optEnv['interrupt']:
+ prev_level = prev_level + interrupt
+
for i in range(1,20):
failed = Command(target='failed%02d' % i, source='', action=fail_action)
- ok = Command(target='ok%02d' % i, source='', action=Touch('${TARGET}'))
+ ok = Command(target=['ok%02da' % i, 'ok%02db' % i, 'ok%02dc' % i],
+ source='',
+ action=[Touch('${TARGETS[0]}'), Touch('${TARGETS[1]}'), Touch('${TARGETS[2]}')])
prereq = Command(target='prereq%02d' % i, source='', action=Touch('${TARGET}'))
ignore = Command(target='ignore%02d' % i, source='', action=Touch('${TARGET}'))
igreq = Command(target='igreq%02d' % i, source='', action=Touch('${TARGET}'))
missing = Command(target='missing%02d' %i, source='MissingSrc', action=Touch('${TARGET}'))
+ withSE = Command(target=['withSE%02da' % i, 'withSE%02db' % i, 'withSE%02dc' % i],
+ source='',
+ action=[Touch('${TARGETS[0]}'), Touch('${TARGETS[1]}'), Touch('${TARGETS[2]}'),
+ Touch('side_effect')])
+ SideEffect('side_effect', withSE)
- next_level = failed + ok + ignore + igreq + missing
+ next_level = failed + ok + ignore + igreq + missing + withSE
for j in range(1,10):
a = Alias('a%02d%02d' % (i,j), prev_level)
@@ -108,7 +143,8 @@ Default(all)
re_error = """\
(scons: \\*\\*\\* \\[failed\\d+] Error 2\\n)|\
-(scons: \\*\\*\\* Source `MissingSrc' not found, needed by target `missing\\d+'\\.( Stop\\.)?\\n)\
+(scons: \\*\\*\\* Source `MissingSrc' not found, needed by target `missing\\d+'\\.( Stop\\.)?\\n)|\
+(scons: \\*\\*\\* \\[\\w+] Build interrupted\.\\n)\
"""
re_errors = "(" + re_error + ")+"
@@ -116,7 +152,17 @@ re_errors = "(" + re_error + ")+"
test.run(arguments = 'all',
status = 2,
stderr = "scons: *** [failed19] Error 2\n")
-test.must_not_exist(test.workpath('ok'))
+test.must_not_exist(test.workpath('side_effect'))
+for i in range(20):
+ test.must_not_exist(test.workpath('ok%02da' % i))
+ test.must_not_exist(test.workpath('ok%02db' % i))
+ test.must_not_exist(test.workpath('ok%02dc' % i))
+ test.must_not_exist(test.workpath('prereq%02d' % i))
+ test.must_not_exist(test.workpath('ignore%02d' % i))
+ test.must_not_exist(test.workpath('igreq%02d' % i))
+ test.must_not_exist(test.workpath('withSE%02da' % i))
+ test.must_not_exist(test.workpath('withSE%02db' % i))
+ test.must_not_exist(test.workpath('withSE%02dc' % i))
for i in range(5):
@@ -135,11 +181,17 @@ for i in range(5):
status = 2,
stderr = re_errors,
match=TestSCons.match_re_dotall)
+ test.must_exist(test.workpath('side_effect'))
for i in range(20):
- test.must_exist(test.workpath('ok%02d' % i))
+ test.must_exist(test.workpath('ok%02da' % i))
+ test.must_exist(test.workpath('ok%02db' % i))
+ test.must_exist(test.workpath('ok%02dc' % i))
test.must_exist(test.workpath('prereq%02d' % i))
test.must_not_exist(test.workpath('ignore%02d' % i))
test.must_exist(test.workpath('igreq%02d' % i))
+ test.must_exist(test.workpath('withSE%02da' % i))
+ test.must_exist(test.workpath('withSE%02db' % i))
+ test.must_exist(test.workpath('withSE%02dc' % i))
for i in range(5):
@@ -147,7 +199,7 @@ for i in range(5):
status = 2,
stderr = re_errors,
match=TestSCons.match_re_dotall)
- test.must_not_exist(test.workpath('ok'))
+
for i in range(5):
test.run(arguments = '-c all')
@@ -157,6 +209,7 @@ for i in range(5):
stderr = re_errors,
match=TestSCons.match_re_dotall)
+
for i in range(5):
test.run(arguments = '-c all')
@@ -164,10 +217,26 @@ for i in range(5):
status = 2,
stderr = re_errors,
match=TestSCons.match_re_dotall)
+ test.must_exist(test.workpath('side_effect'))
for i in range(20):
- test.must_exist(test.workpath('ok%02d' % i))
+ test.must_exist(test.workpath('ok%02da' % i))
+ test.must_exist(test.workpath('ok%02db' % i))
+ test.must_exist(test.workpath('ok%02dc' % i))
test.must_exist(test.workpath('prereq%02d' % i))
test.must_not_exist(test.workpath('ignore%02d' % i))
test.must_exist(test.workpath('igreq%02d' % i))
+ test.must_exist(test.workpath('withSE%02da' % i))
+ test.must_exist(test.workpath('withSE%02db' % i))
+ test.must_exist(test.workpath('withSE%02dc' % i))
+
+
+for i in range(5):
+ test.run(arguments = '-c all')
+
+ test.run(arguments = '-j 8 -k --random interrupt=yes all',
+ status = 2,
+ stderr = re_errors,
+ match=TestSCons.match_re_dotall)
+
test.pass_test()