summaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
...
* [SV 63516] [DOS] Support include files with drivespecsPaul Smith2022-12-181-1/+11
| | | | | | | | | * src/makeint.h (HAVE_DRIVESPEC): Create a macro to check. * src/main.c (.FEATURES): Add "dospaths" as a feature. * src/read.c (eval_makefile) [DOS]: If the included makefile name starts with a drivespec, don't search the include directories. * doc/make.texi (Include): Document this behavior. * tests/scripts/features/include: Add a test.
* [SV 63417] Ensure global .NOTINTERMEDIATE disables all intermediatesDmitry Goncharov2022-11-281-3/+0
| | | | | | | | | | | | | | | | | Fix .NOTINTERMEDIATE without prerequisites to disable intermediate status for all targets. * src/makeint.h: Declare extern no_intermediates. * src/main.c: Add global definition of no_intermediates. * src/file.c: Remove static no_intermediates to use global variable. (remove_intermediates): Check no_intermediates. * src/implicit.c (pattern_search): For a file found by implicit search set file->notintermediate if no_intermediates is set. * src/remake.c (update_file_1): Don't set file->secondary for a pre-existing file if no_intermediates is set. The check for no_intermediates here is redundant, but won't hurt: keep it in case things change so that it matters. * tests/scripts/targets/NOTINTERMEDIATE: Fix a test.
* [SV 63347] Always add command line variable assignments to MAKEFLAGSDmitry Goncharov2022-11-281-30/+243
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit introduces two visible changes: 1. Keep command line variable assignments in MAKEFLAGS at all times, even while parsing makefiles. 2. Define makeflags immediately when a makefile modifies MAKEFLAGS. The new MAKEFLAGS and MAKEOVERRIDES initialization procedure: 1. decode_switches (argc, argv, o_command) is called to parse command line variable assignments. 2. Command line variable assignments go through quote_for_env. Initialize -*-command-variables-*- to the quoted values. 3. MAKEOVERRIDES is initialized to refer to -*-command-variables-*- with origin o_env to keep the definitions in the database intact. 4. define_makeflags() is called which adds MAKEOVERRIDES to MAKEFLAGS. 5. Makefiles are parsed. If a makefile modifies MAKEFLAGS, the new value of MAKEFLAGS is defined right away. 6. Env switches are decoded again as o_env. The definitions set by decode_switches at step 1 stay intact, as o_command beats o_env. We must preserve the original intact definitions in order to detect failure cases; for example: $ cat makefile all:; $(hello) $ make hello='$(world' makefile:1: *** unterminated variable reference. Stop. * src/makeint.h: Declare enum variable_origin, struct variable and define_makeflags(). Add parameter origin to decode_env_switches(). * src/main.c (define_makeflags): Remove "all". If a variable is assigned on the command line then append MAKEOVERRIDES to MAKEFLAGS. (decode_env_switches): Replace parameter env with origin. (decode_switches): Replace parameter env with origin. Treat origin == o_command as env == 0. (handle_non_switch_argument): Replace parameter env with origin. Treat origin == o_command as env == 0. (main): Call decode_switches() with origin==o_command before parsing makefiles. Call decode_switches() with origin==o_env after parsing makefiles. * src/variable.c (set_special_var): Define makeflags at parse time, each time a makefile modifies MAKEFLAGS. (do_variable_definition): Strip command line variable assignments from MAKEFLAGS before appending extra flags. set_special_var() adds them back. * tests/scripts/variables/MAKEFLAGS: Add tests.
* Add specific hints for errors due to invalid conditionalsPaul Smith2022-11-152-20/+49
| | | | | | | | * src/read.c (eval): If "missing separator" appears to be due to missing space after ifeq/ifneq, give a hint about the error. * tests/scripts/misc/failure: Check for these types of failures. * tests/scripts/variables/special: Move error checking unrelated to special variables, to misc/failure.
* [SV 63333] Be more lenient when failing to create temporary filesDmitry Goncharov2022-11-132-1/+32
| | | | | | | | | | | | | | | | | | | | | | If make cannot create a temporary lock file for output sync, continue without output sync enabled rather than dying. However, if make cannot store a makefile from stdin to a temporary file that is still a fatal error. * misc.c (get_tmppath): Keep running on failure to generate a temporary file name. (get_tmpfd): Keep running on failure to get a temporary file. (get_tmpfile): Keep running on failure to open a temporary file. Ensure memory is freed if we return an error. * posixos.c (os_anontmp): Keep running on failure to open an anonymous temporary file. * output.c (setup_tmpfile): Print an error on failure to create an output sync lock file. * main.c (main): Die on failure to store makefile from stdin to a temporary file. * tests/scripts/features/output-sync: Add tests. * tests/scripts/features/temp_stdin: Ditto.
* Keep going if we can't connect to the jobserverPaul Smith2022-11-131-14/+19
| | | | | | | | * src/posixos.c (jobserver_parse_auth): Don't invoke fatal() if we can't connect to an existing jobserver: just keep going without it. * src/w32/w32os.c (jobserver_parse_auth): Ditto. * tests/scripts/features/jobserver: Add a test for invalid FIFO auth files.
* [SV 63315] tests: Simplify TERM signalingPaul Smith2022-11-133-2/+9
| | | | | | | | | | | | Tests that try to kill the make process were not behaving as expected on OpenBSD: the signal was sent from make to its children but the sleep didn't die. Something odd about the way the shell treats TERM. To reduce platform dependencies add "term" to the helper tool and run that instead of kill / sleep. * tests/thelp.pl: Add a new operation "term" that takes a PID. * tests/scripts/features/output-sync: Use it. * tests/scripts/features/temp_stdin: Ditto.
* tests: Don't convert \ to / when checking regex'sPaul Smith2022-11-132-154/+133
| | | | | | | | | | | | | When tests compare the output they will try converting backslashes to slashes to see if that works. When we compare using regex's, we can't do that because backslashes can escape special characters. * tests/test_driver.pl (compare_output): Clean up this function. (compare_answer_vms) [VMS]: Comparing answers on VMS is complex; move all of it into its own function returning 0/1. (compare_answer): A new function to compare answers: return 0/1. Remember the CRLF->LF conversion forever; only check \ -> / when we compare strings, not regex's.
* Fix issues found by ASAN and CoverityPaul Smith2022-10-314-14/+7
| | | | | | | | | | | | | | | * tests/test_driver.pl: Preserve the LSAN_OPTIONS variable. * tests/scripts/targets/ONESHELL: Don't set a local variable. * tests/scripts/functions/let: Test empty let variable. * src/posixos.c (osync_parse_mutex): Free existing osync_tmpfile. * src/misc.c (get_tmpfd): Set umask() before invoking mkstemp(). * src/ar.c (ar_parse_name): Check invalid name (shouldn't happen). * src/function.c (define_new_function): Free previous function entry when replacing it with a new one. * src/job.c (child_execute_job): Initialize pid for safety. (construct_command_argv_internal): In oneshell mode ensure that the returned argv has the right format (0th element is a pointer to the entire buffer).
* tests: Avoid the use of File::Temp->newdir()Paul Smith2022-10-293-33/+30
| | | | | | | | | | This was added in Perl 5.8 but some systems still only provide older versions such as Perl 5.6. We don't really need it anyway. Paul Eggert <eggert@cs.ucla.edu> reported this issue. * tests/README: Update this to be a bit more modern. * tests/test_driver.pl: Delete the $TEMPDIR variable. * tests/scripts/features/temp_stdin: Use $temppath not $TEMPDIR.
* [SV 62174] Force locale to be "C" before retrieving error messagesPaul Smith2022-10-281-72/+75
| | | | | | | | | | | We attempt to do this with POSIX::setlocale() but apparently on some systems (AIX) this isn't sufficient. So, in addition force the LC environment variables to use "C". Reported by Dmitry Goncharov <dgoncharov@users.sf.net>. * tests/run_make_tests.pl: Move the global setup into set_default(). Force the %ENV locale variables to use the ones we'll use when running make, then reset them back again after we find error messages.
* * tests/scripts/features/exec: Don't test with C shellsPaul Smith2022-10-251-3/+11
| | | | | | Using C shells (csh, tcsh) with make is known to be problematic due to incorrect ways it handles open file descriptors, at least. If the user's shell is *csh then don't try it during exec tests.
* Increase the test framework timeout from 5s to 60sPaul Smith2022-10-253-6/+6
| | | | | | | | | | | | It seems that some of the test environments hit the 5s timeout on some tests. Since it doesn't really matter, as long as we don't hang forever, increase the timeout to 60s. * tests/test_driver.pl: Increase $test_timout to 60. We don't need to handle VMS timeouts specially anymore. * tests/scripts/features/parallelism: We don't need to override the default timeout anymore. * tests/scripts/features/patternrules: Remove confusing comment.
* [SV 63098] Enhance detection of missing peer also-make targetsPaul Smith2022-10-221-0/+22
| | | | | | | | | | | | The previous attempt to detect missing peer targets for implicit rules had some holes. Move the detection to notice_finished_file(). * src/remake.c (check_also_make): If we don't have the current mtime for the file, obtain it. (update_goal_chain): Don't call check_also_make() here. (check_dep): Ditto. (notice_finished_file): If we finished running an implicit rule that has also_make targets, invoke check_also_make().
* Collect failure details when the regression tests failPaul Smith2022-10-221-0/+1
| | | | | | * README.in: Add a section on running regression tests. * Makefile.am (check-regression): Capture the test run output, and on failure collect configure and test results into a tar file.
* Enhance tests to work on different systemsPaul Smith2022-10-229-28/+35
| | | | | | | | | | | | | | | | | | | | | The GNU platform testers reported a number of test errors on different systems; try to address them. * tests/thelp.pl: A number of tests timed out with a 4-second timeout. Increase the default timeout to 10 seconds. * tests/run_make_tests.pl: Executing directories on cygwin behaves differently in Perl than make so skip these tests there. * tests/scripts/options/symlinks: Check for the symlink feature in make, rather than whether the system supports them. * tests/scripts/features/implicit_search: On some systems "false" exits with a different exit code. Use the helper instead. * tests/scripts/features/loadapi: Ditto. * tests/scripts/features/output-sync: Sleep before make -f bar in the first test as well as the second one. * tests/scripts/features/exec: Skip on cygwin, which seems to be "UNIX" but where scripts don't run normally. * tests/scripts/misc/fopen-fail: Skip on cygwin, where make eventually exits with exit code 0 and no error messages.
* [SV 63243] tests: Avoid SIGTERM racing with make error messagesDmitry Goncharov2022-10-222-4/+8
| | | | | | | | | Original patch from Frank Heckenbach <f.heckenbach@fh-soft.de>. * tests/scripts/features/output-sync: Introduce a sleep to let make write its error message. Some systems use different names for SIGTERM so match with a regex. * tests/scripts/features/temp_stdin: Ditto.
* * NEWS: Use GNU Make instead of GNU makePaul Smith2022-10-181-1/+1
| | | | | | | | | | | | * README.git: Ditto. * README.Amiga: Ditto. * README.DOS: Ditto. * README.OS2: Ditto. * README.VMS: Ditto. * README.W32: Ditto. * README.customs: Ditto. * make-gdb.py: Ditto. * tests/run_make_tests.pl: Ditto.
* Update URLs to use https rather than httpPaul Smith2022-10-188-11/+11
| | | | | * (all): Change http:// to https:// * README.W32: Remove invalid link to mingw.org.
* Add preprocessor macros for different platformsPaul Smith2022-10-181-1/+1
| | | | | | | | | | | | | * configure.ac: MK_CONFIGURE shows config.h was generated by configure. * src/config.ami: Define MK_AMIGAOS. * src/config.h-vms: Define MK_VMS. * src/configh.dos: Define MK_DJGPP. * src/config.h.W32: Define MK_W32 and WINDOWS32. * src/build_w32.bat: Let WINDOWS32 be defined by config.h. Remove unused setting of WIN32. * src/job.c: Clean up use of WIN32. * src/main.c: Ditto. * tests/scripts/features/default_names: Ditto.
* [SV 63157] Ensure temporary files are removed when signaledPaul Smith2022-10-152-0/+52
| | | | | | | | | | | | | | | | | | Original patch from Dmitry Goncharov <dgoncharov@users.sf.net>. When handling a fatal signal ensure the temporary files for stdin and the jobserver fifo (if in use) are deleted. * src/makeint.h (temp_stdin_unlink): Declare a new method. * src/main.c (temp_stdin_unlink): Delete the stdin temporary file if it exists. If the unlink fails and we're not handling a signal then show an error. (main): Call temp_stdin_unlink() instead of unlinking by hand. * src/commands.c (fatal_error_signal): Invoke cleanup methods if we're handling a fatal signal. * tests/scripts/features/output-sync: Test signal handling during output sync and jobserver with FIFO. * tests/scripts/features/temp_stdin: Test signal handling when makefiles are read from stdin.
* * tests/test_driver.pl (toplevel): [SV 63156] Keep temp env varsDmitry Goncharov2022-10-081-3/+3
|
* [SV 63098] Temporarily revert the change to pattern rule behaviorPaul Smith2022-10-021-4/+23
| | | | | | | | | | | | | | | | | | | | The fix for SV 12078 caused a backward-compatibility issue with some makefiles. In order to allow users to resolve this issue, revert that change for this release cycle: it will be reinstated in the next release cycle. Introduce a warning if we detect that the recipe of a multi-target pattern rule doesn't create all the targets. * NEWS: Announce the future backward-incompatibility. * doc/make.texi (Pattern Intro): Describe the behavior and that it will change in the future. * src/remake.c (check_also_make): Check for also_make targets that were not created and generate a warning. (update_goal_chain): Call the new function. (check_dep): Ditto. (update_file_1): Defer implicit rule detection until after we check all the also_make files (as it used to be). * tests/scripts/features/patternrules: Add tests of the new warning. Skip the tests for SV 12078.
* [SV 63100] Set the floc of every created goal dep structDmitry Goncharov2022-09-251-4/+23
| | | | | | * src/read.c (eval): Initialize the goaldep floc pointer. * tests/scripts/features/loadapi: Verify that the floc is set after unloading and reloading dynamic objects.
* Always restore global environ if we use vforkPaul Smith2022-09-251-0/+9
| | | | | | | | | | | | | | | | We may change the global environ variable in the child; when using vfork() this also sets it in the parent. Preserve the parent's environ in child_execute_job() so it takes effect for all callers. Reported by Denis Excoffier <bug-tar@Denis-Excoffier.org> Root cause found by Martin Dorey <Martin.Dorey@hitachivantara.com> * src/job.c (start_job_command): Remove save/restore of the parent environment. (child_execute_job): Add save/restore of the parent environment, if we use vfork(). * tests/scripts/functions/shell: Add a test the crashes if we don't reset environ after we run $(shell ...).
* * tests/script/parallelism: Force wait to remove races.Paul Smith2022-09-201-46/+43
|
* [SV 56301] Fail if an included makefile can't be builtDmitry Goncharov2022-09-202-31/+35
| | | | | | | | | | Fail if a mandatory include file fails to be built even if it's built as part of a grouped target where the other include file is optional. * src/main.c (main): If a makefile doesn't build set any_failed. * tests/scripts/features/include: Add tests. * tests/scripts/options/dash-k: Stop after include build failure.
* * tests/scripts/features/statipattrules: [SV 17374] Add testPaul Smith2022-09-201-1/+12
| | | | Verify that double-colon static pattern rules work.
* [SV 12078, SV 62809] Rebuild grouped targets if any is missingPaul Smith2022-09-207-210/+208
| | | | | | | | | | | | | | | | | | | If any of a set of grouped targets is missing or out of date, even if make is not trying to build that target, rebuild them all. Ensure this is true for explicit grouped targets as well as pattern rule grouped targets. Original patch by Jonathan Gravel <jo@stashed.dev> * src/remake.c (update_file_1): After matching any pattern rules, go through the also_make targets and set noexist as needed. Also compute the oldest this_mtime. * tests/scripts/features/grouped_targets: Add regression tests. * tests/scripts/features/patternrules: Ditto. * tests/features/vpath: Rewrite to use modern run_make_test(). Add a test that we check for VPATH before implicit rule search. Move the tests in vpath2 and vpath3 into this suite. * tests/features/vpathplus: Rewrite to use modern run_make_test().
* * tests/scripts/targets/WAIT: [SV 63071] Remove possible test races.Dmitry Goncharov2022-09-171-8/+8
|
* [SV 13862] Implement the .WAIT special targetPaul Smith2022-09-121-0/+193
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The next version of the POSIX standard defines parallel execution and requires the .WAIT special target as is implemented in some other versions of make. This implementation behaves similarly to others in that it does not create a relationship between targets in the dependency graph, so that the same two targets may be run in parallel if they appear as prerequisites elsewhere without .WAIT between them. Now that we support .WAIT it's trivial to also support prerequisites of the .NOTPARALLEL special target, which forces the prerequisites of those targets to be run serially (as if .WAIT was specified between each one). * NEWS: Announce the new .WAIT and .NOTPARALLEL support. * doc/make.texi (Parallel Disable): A new section to discuss ways in which parallel execution can be controlled. Modify cross-refs to refer to this section. * src/dep.h (struct dep): Add a new wait_here boolean. (parse_file_seq): Add PARSEFS_WAIT to check for .WAIT dependencies. * src/file.c (split_prereqs): Use PARSEFS_WAIT. (snap_deps): If .NOTPARALLEL has prerequisites, set .WAIT between each of _their_ prerequisites. (print_prereqs): Add back in .WAIT when printing prerequisites. * src/implicit.c (struct patdeps): Preserve wait_here. (pattern_search): Ditto. Use PARSEFS_WAIT when parsing prereqs for pattern rule expansion. * src/read.c (check_specials): Don't give up early: remembering to update these options is not worth the rare speedup. (check_special_file): If .WAIT is given as a target show an error-- once--if it has prereqs or commands. (record_files): Call check_special_file on each target. (parse_file_seq): If PARSEFS_WAIT is given, look for .WAIT prereqs. If we see one assume that we are building a struct dep chain and set the wait_here option while not putting it into the list. * src/remake.c (update_file_1): If wait_here is set and we are still running, then stop trying to build this target's prerequisites. * src/rule.c (get_rule_defn): Add .WAIT to the prerequisite list. * src/shuffle.c (shuffle_deps): Don't shuffle the prerequisite list if .WAIT appears anywhere in it. * tests/scripts/targets/WAIT: Add a test suite for this feature.
* [SV 63045] Reload each intact unloaded shared objectDmitry Goncharov2022-09-121-0/+82
| | | | | | | | | | | | | | | | | | | | | | | If makefile rules do not update an unloaded shared object, load it again. Avoid double loading of the same object if the setup function returns -1. * src/filedef.h (struct file): Add "unloaded" flag. * src/makeint.h (load_file): Take struct file *. (unload_file): Return int. * src/main.c (main): Reload unloaded shared objects if they weren't updated. * src/commands.c (execute_file_commands): Set "unloaded" and reset "loaded" when a shared object is unloaded. * src/read.c (eval): Set "loaded" and reset "unloaded" when a shared object is loaded. Add successfully loaded files to the db. * src/load.c (load_file): Check "loaded" to avoid double loading the same object. Fix a memory leak of string loaded. Return -1, rather than 1, if the object is already loaded. This fixes double loading of the same object when the setup routine returns -1. (load_object): Add a log message. (unload_file): Return an error on dlclose failure. Log a message. * tests/scripts/features/loadapi: Add new tests.
* [SV 63047] Fix shuffle of SECONDEXPANSION prerequisitesSergei Trofimovich2022-09-121-0/+9
| | | | | | | | | | | | | | Commit 07eea3aa4 `make --shuffle` prevented shuffling prerequisites that use .SECONDEXPANSION, since shuffle happens before expansion. This has two problems: 1. No shuffling happens for such prerequisites. 2. Use-after-free when outdated '->shuf' links are used. Add a reshuffle into expansion phase right after dependency changes. * src/file.c (expand_deps): Add reshuffle if dependencies change. * src/shuffle.c (identity_shuffle_array): Fix comment typo. * tests/scripts/options/shuffle: Add new SECONDEXPANSION test.
* Support the MAKE_TMPDIR environment variablePaul Smith2022-09-111-0/+10
| | | | | | | | | | | Allow build systems to choose an alternative location for make to store its temporary files. * NEWS: Announce the new environment variable. * doc/make.texi (Temporary Files): Provide documentation. * src/misc.c (get_tmpdir): Split into a new function. Compute the temporary directory and store it in a static location. * tests/scripts/features/jobserver: Add a test of MAKE_TMPDIR.
* [SV 63044] load: Update .LOADED if the setup function returns -1Dmitry Goncharov2022-09-101-5/+34
| | | | | | | | * src/load.c (load_file): Update .LOADED if setup returns non-0. * tests/scripts/features/load: Change the return value based on an environment variable. Ensure that returning -1 still adds to .LOADED. Also add a test that verifies that make doesn't try to rebuild the loaded file if -1 is returned.
* [SV 63037] Fix tests for newer OS / compilerPaul Smith2022-09-102-9/+12
| | | | | | * tests/scripts/features/load: Add function prototypes. * tests/scripts/features/temp_stdin: Only set user execute bits. Some configurations don't allow changing the group bits...??
* [SV 63040] shell: Fall back to the callers environmentPaul Smith2022-09-101-0/+16
| | | | | | | | | | | | | | | | | | | If we detect a recursive variable reference when constructing the environment for the shell function, return the original value from the caller's environment. Other options such as failing, returning the empty string, or returning the unexpanded make variable value have been shown to not behave well in real-world environments. If the variable doesn't exist in the caller's environment, return the empty string. Found by Sergei Trofimovich <slyich@gmail.com> when testing older versions of autoconf. * NEWS: Clarify this behavior. * doc/make.texi (Shell Function): Ditto. Also add info about !=. * src/expand.c (recursively_expand_for_file): Search the caller's environment if we detect a recursive variable expansion. * tests/scripts/functions/shell: Add tests for this behavior.
* [SV 63016] Don't fail exporting to $(shell ...)Paul Smith2022-09-081-0/+10
| | | | | | | | | | | | | | | | The fix for SV 10593 caused recursive expansion errors when exporting a variable that contains a $(shell ...) invocation. If we see this type of recursion, ignore it and expand to the empty string rather than failing. * src/variable.h (env_recursion): New global variable. * src/variable.c (target_environment): If creating the environment for a $(shell ...) function increment env_recursion. Remove the check for expansion in a shell function context. * src/expand.c (recursively_expand_for_file): Check for recursive expansion in a $(shell ...) environment context and if present, show the verbose message and return the empty string. * tests/scripts/functions/shell: Add a test for this situation.
* Fix tests for MacOS and WindowsPaul Smith2022-08-305-33/+33
| | | | | | | | | | | * maintMakefile: Remove the template headers as prerequisites. * tests/scripts/features/jobserver: Only test fifo if enabled. * tests/scripts/variables/INCLUDE_DIRS: On MacOS none of the default directories exist so .INCLUDE_DIRS is empty by default. * tests/scripts/features/se_explicit: Fail via exit. cp will show different error messages on different systems. * tests/scripts/features/se_implicit: Ditto. * tests/scripts/features/se_statpat: Ditto.
* Rework output sync to lock a temp file on POSIXPaul Smith2022-08-301-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some POSIX systems do not allow locks to be taken on non-files, such as pipes. This is a problem since very often make is invoked with its stdout redirected to a pipe. Also, if stdout is redirected to a file that already has a lock on it for some other reason (perhaps a shared file such as /dev/null) it can cause a hang. This means our previous method of locking stdout, although it had some nice advantages, is not portable enough. Instead, use a temporary file and take the lock on that. We pass the name of the file to child make processes. On Windows we continue to use a shared mutex for output sync. Remove POSIX emulation functions like fcntl from Windows; instead follow the lead of the jobserver and create an interface in os.h for output sync, and move the OS-specific content to posixos.c and w32os.c. * NEWS: Add a note. * src/makeint.h (ALL_SET): Check that all bits are set. * src/os.h: Add bits for checking the state of stdin/stdout/stderr. Add prototypes for OS-specific output sync methods. * src/posixos.c (check_io_state): Determine the status of stdin, stdout, stderr an return a suite of bits describing them. (osync_enabled): If the global variable holding the FD of the lock file (osync_handle) is valid return true. (osync_setup): Create a temporary file and remember its name in a global variable (osync_tmpfile), and set osync_handle. (osync_get_mutex): If output sync is enabled, return the filename of the lock file prefixed with "fnm:" to denote a filename. (osync_parse_mutex): If the provided filename has the wrong format disable output sync. Else open the lock file and set osync_handle. (osync_clear): Close osync_handle. If we're the parent make, then also unlink the temporary file. (osync_acquire): Take a lock on the osync_handle descriptor. (osync_release): Release the lock on the osync_handle descriptor. (fd_set_append): Add APPEND mode to a file descriptor. * src/w32/w32os.c: Perform the same actions as posixos.c, copying the details from src/w32/compat/posixfcn.c. Use a mutex rather than locking a temporary file. * src/output.h: Remove all the OS-specific content. * src/output.c: Remove all the OS-specific content. (set_append_mode): Remove and replace with fd_set_append(). (sync_init): Remove and replace with check_io_state(). (acquire_semaphore): Remove and replace with osync_acquire(). (release_semaphore): Remove and replace with osync_release(). (setup_tmpfile): If the IO state is not obtained, get it. If stdout and/or stderr are valid, set up a tempfile to capture them. (output_init): Set io_state if not set already, and check it when deciding whether to close stdout on exit. * src/main.c (main): If we're syncing, set up the mutex using the new osync_setup() / osync_parse_mutex() methods. (prepare_mutex_handl_string): Replace with osync_parse_mutex(). (die): Call osync_clear(). * src/w32/compat/posixfcn.c: Remove implementations of fcntl(), record_sync_mutex(), create_mutex(), and same_stream().
* test_driver: check for leftover temp files after each testPaul Smith2022-08-292-36/+52
| | | | | | | | | | | | | | | | | | | | Reset the temp directory for every test to a local directory, then after each test see if any new temp files were created and not deleted: if they were then fail the test. Rather than delete the temp files we leave them there and avoid reporting files that were seen before, so the user can investigate them. Rewrite the temp_stdin tests to rely on this built-in behavior rather than implementing the checks directly. * tests/test_driver.pl: Create a $TEMPDIR variable pointing to a temporary directory outside the test temp directory. (toplevel) Before starting any tests create a temp directory and set the POSIX and Windows temp directory environment variables to use it. (compare_output) Check the contents of the temp directory. If any new files have appeared, fail the test. * tests/scripts/features/temp_stdin: Remove check_tempfile() and all users of it, plus setting of temp environment variables.
* [SV 62908] Don't free fifo_name twice during error handlingDmitry Goncharov2022-08-221-0/+15
| | | | | * src/posixos.c (jobserver_setup): Set fifo_name to NULL after free. * tests/scripts/features/jobserver: Add tests.
* Fix bootstrap.bat for bootstrapping on WindowsPaul Smith2022-08-222-2/+4
| | | | | | | | | | | | | | | | * README.git: Clarify that these methods are lightly tested. * build_w32.bat: Don't support any config step: fail if not completed. Move the config steps into bootstrap.bat. Don't print compile lines by default and add a --verbose option to show them. * bootstrap.bat: Ensure we have curl and sed before we do anything. Pull the latest necessary files from gnulib. Create a convert.sed script that can update the various template files, and update Basic.mk, config.h.W32, and gmk-default.h. * tests/run_make_tests.pl: Remove CRLF rather than using chop. If we run perl in Git for Bash it seems to handle newlines differently. * tests/scripts/features/temp_stdin: Remove the make copy and close STDIN so we can delete the temp file on Windows. * .gitignore: Ignore the convert.sed script.
* Support implementing the jobserver using named pipesPaul Smith2022-08-022-9/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using anonymous pipes for jobserver support has some advantages: for example there is nothing on disk that needs to be cleaned up. However it has many obscure problems, related to the fact that in order for it to work we need to ensure these resources are properly passed through to child processes that want to use the jobserver. At the same time we don't want to pass the pipe to process which DON'T know about the jobserver. Other processes can open file descriptors which we then think are our jobserver, but aren't. And, we open the pipe file descriptors in blocking mode which doesn't work for all users. See issues such as SV 57178, SV 57242, and SV 62397 To avoid these issues, use named pipes (on systems where they are available) instead of anonoymous pipes. This simplifies many things: we never need to pass open file descriptors to our children; they can open the jobserver named pipe. We don't need to worry about recursive vs. non-recursive children. Users don't have to "pass through" the resources if they are invoking sub-makes. Each child can open its own file descriptor and set blocking as needed. The downside is the named pipe exists on disk and so must be cleaned up when the "top-level" make instance exits. In order to allow make to continue to be used in build systems where older versions of GNU make, or other tools that want to use the jobserver, but don't understand named pipes, introduce a new option --jobserver-style that allows the user to choose anonymous pipes. * NEWS: Announce the change and the --jobserver-style option. * doc/make.1: Add --jobserver-style documentation. * doc/make.texi (Special Variables): Add missing items to .FEATURES. (Options Summary): Add --jobserver-style. (POSIX Jobserver): Named pipes, changes to --jobserver-auth, and the --jobserver-style option. (Windows Jobserver): Document --jobserver-style for Windows. * configure.ac: Check for mkfifo. * src/config.h-vms.template: Undefined HAVE_MKFIFO. * src/config.h.W32.template: Ditto. * src/main.c: Add jobserver-style as a new command line option. (main): Add jobserver-fifo to .FEATURES if supported. Pass the style option to jobserver_setup(). * src/os.h (jobserver_setup): Accept a style string option. * src/posixos.c (enum js_type): Enumeration of the jobserver style. (js_type): Which style we are currently using. (fifo_name): The path to the named pipe (if in use). (jobserver_setup): If no style is given, or "fifo" is given, set up a named pipe: get a temporary file and use mkfifo() on it, then open it for reading and writing. If something fails fall back to anonymous pipes. (jobserver_parse_auth): Parse jobserver-auth to determine the style. If we are using a named pipe, open it. If we're using anonymous pipes ensure they're valid as before. (jobserver_get_invalid_auth): Don't invalidate the jobserver when using named pipes. (jobserver_clear): Clean up memory used for named pipes. (jobserver_acquire_all): Unlink the named pipe when done. * src/w32/w32os.c (jobserver_setup): Check the style argument. * tests/scripts/features/jobserver: Use --jobserver-style to test the anonymous pipe behavior, and also test named pipe/semaphore behavior. Check invalid jobserver-style options. * tests/scripts/functions/shell: Use --jobserver-style to test the anonymous pipe behavior, and also test named pipe/semaphore behavior.
* [SV 62706] Only second-expand targets that might be builtDmitry Goncharov2022-07-303-7/+390
| | | | | | | | | | | | | | | | | | | | | Second-expand only the prerequisites of the targets being built. Defer second-expanding the prerequisites of targets until we need to decide if they should be built. * NEWS: Mention the change in behavior. * doc/make.texi (Secondary Expansion): Document the new behavior. * src/filedef.h (struct file): Add flag snapped. (expand_deps): Declare a function to second expand the prerequisites of a target. * src/file.c (rehash_file): Merge flag snapped. (expand_deps): Remove qualifier static. Check flag snapped. (snap_deps): Remove the loop which performed second expansion for all targets. * src/remake.c (update_file_1): Second expand the prerequisites of the considered target. * tests/scripts/features/se_explicit: Add tests. * tests/scripts/features/se_implicit: Ditto. * tests/scripts/features/se_statpat: Ditto.
* Disable the jobserver in non-recursive childrenPaul Smith2022-07-301-5/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Savannah issues such as SV 57242 and SV 62397 show how passing references to closed file descriptors via the --jobserver-auth option in MAKEFLAGS can lead to problematic outcomes. When computing the child environment for a non-recursive shell, add an extra option to MAKEFLAGS to disable the file descriptors for the jobserver. Unfortunately this doesn't modify the value of the make variable MAKEFLAGS, it only modifies the value of the sub-shell environment variable MAKEFLAGS. This can lead to confusion if the user is not considering the distinction. * src/makeint.h: Publish the jobserver-auth value. Add a global definition of the name of the command line option. * src/os.h (jobserver_get_invalid_auth): New function to return a string invalidating the jobserver-auth option. * src/w32/w32os.c (jobserver_get_invaid_auth): Implement it. On Windows we use a semaphore so there's no need to invalidate. * src/posixos.c (jobserver_parse_auth): If we parse the invalid auth string, don't set up the jobserver. (jobserver_get_invalid_auth): Return an invalid option. * src/variable.h (target_environment): Specify if the target environment is for a recursive shell or non-recursive shell. * src/variable.c (target_environment): Move checking for MAKELEVEL into the loop rather than doing it at the end. Along with this, check for MAKEFLAGS and MFLAGS, and update them based on whether we're invoking a recursive or non-recursive child, and also on whether it's necessary to invalidate the jobserver. * src/function.c (func_shell_base): Shell functions can never be recursive to pass 0 to target_environment(). * src/job.c (start_job_command): Specify whether the child is recursive when calling target_environment(). * src/main.c: Export jobserver_auth. sync_mutex doesn't need to be exported. Use the global definition for the option name. * tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
* Ensure that MAKEFLAGS is set when invoking $(shell ...)Paul Smith2022-07-302-0/+25
| | | | | | | | | * src/main.c (main): Don't reset the jobserver if the number of slots has not changed. (define_makeflags): Add all normal flags even when ALL is not set. * tests/scripts/functions/shell: Test invoking make in $(shell ...). * tests/scripts/variables/MAKEFLAGS: Test the value of MAKEFLAGS in $(shell ...).
* * tests/run_make_tests.pl: Exit 1 if we detect an error.Paul Smith2022-07-091-3/+3
|
* [SV 10593] Export variables to $(shell ...) commandsPaul Smith2022-07-091-4/+51
| | | | | | | | | | | | | | | | | | | | | | Export all variables, including exported makefile variables, when invoking a shell for the $(shell ...) function. If we detect a recursive variable expansion, silently ignore that variable and do not export it. We do print a debug message. * NEWS: Announce the potential backward-incompatibility. * doc/make.texi (Shell Function): Document the export behavior. * src/main.c (main): Add "shell-export" to .FEATURES. * src/job.h: New function to free struct childbase. * src/job.c (free_childbase): Implement it; call from free_child. * src/function.c (func_shell_base): Use target_environment() to obtain the proper environment for the shell function. Use free_childbase() to free memory. (windows32_openpipe): Don't reset the environment: the caller already provided a proper PATH variable in envp. * src/variable.c (target_environment): If we detect a recursive expansion and we're called from func_shell, ignore the variable. (sync_Path_environment): Simplify and reduce memory allocation. * tests/scripts/functions/shell: Add tests for this.
* [SV 62650] Show correct warning when combining targetsDmitry Goncharov2022-07-092-8/+18
| | | | | | | * src/file.c (rehash_file): Fix warning message. (rehash_file): Fix comment to match the behavior. * tests/scripts/features/se_explicit: Fix test. * tests/scripts/features/mult_rules: Add a new test.
View Lorry Controller Status