summaryrefslogtreecommitdiff
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* Check for recipe line count overflow before it overflowsPaul Smith2022-10-181-34/+30
| | | | | | | | | | | | | | | awk 'BEGIN { print "x:" for (i = 0; i < 65536; i++) printf "\techo %d\n", i} ' | make -f - Outputs only "make: 'x' is up to date." Larger values run only the lines above 65536. Reported by Paul Eggert <eggert@cs.ucla.edu>. * src/commands.c (chop_commands): Check the line count before it has a chance to overflow. Use size_t for max count so it can't overflow. Remove stray 'd' in diagnostic.
* Remove template files to simplify distribution creationPaul Smith2022-10-186-66/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The README templates were not useful since the replacement step didn't have anything to replace: rename them. Rather than creating template files for the config variants, create mkconfig.h.in containg PACKAGE_* variables to be replaced, and have config variant header files include it. Note on POSIX we don't use this, and continue to generate a single config.h.in file. Use config.status to convert the README.in and mkconfig.h.in files during distribution creation. Modify all users of VERSION to use PACKAGE_VERSION instead. * configure.ac: Use GNU Make not GNU make as the package name. * README.in: Use GNU Make not GNU make. * README.git: Remove references to README.W32.template. * .gitignore: Update for new behavior. * Basic.mk.template: Remove unused posix_SOURCES and VERSION, and references to unshipped mk/Posix.mk * Makefile.am: Add src/mkconfig.h as an extra dist file. * bootstrap.bat: Rewrite mkconfig.h.in to mkconfig.h * maintMakefile: Remove obsolete template files; add mkconfig.h.in. * prepare_vms.com: Rewrite mkconfig.h.in to mkconfig.h * mk/VMS.mk: Fix incorrect header file prerequisite. * src/mkconfig.h.in: New file containing PACKAGE variables. * src/config.ami: Include mkconfig.h. * src/config.h.W32: Ditto. * src/configh.dos: Ditto. * src/config.h-vms: Ditto. * src/version.c: Use PACKAGE_VERSION not VERSION.
* Support building with DJGPP 2.05Juan M. Guerrero2022-10-161-2/+7
| | | | | | | | * builddos.bat: Use env var settings for paths. Fix a typo in expand.o. Add the missing load.o compilation and link. Enable Guile support. Copy Basic.mk from the correct location. * src/configh.dos.templage: DJGPP supports strtoll() and ssize_t given new enough versions. Set preprocessor variables to 1 not 0.
* * src/job.c: [SV 63185] Include sys/loadavg.h if needed.Paul Smith2022-10-161-4/+9
|
* [SV 63215] Remember the random seed for shuffle modePaul Smith2022-10-163-6/+10
| | | | | | | | | | Basic fix provided by James Hilliard <james.hilliard1@gmail.com>. Ensure we remember and propagate the random seed we generate during shuffle mode. Also add a debug statement displaying the seed. * src/shuffle.c (shuffle_set_mode): Init and save the randoms seed. * src/misc.c (make_rand): Code cleanups. * src/main.c (main): Show a debug message containing the seed.
* Use (void) rather than () in function declarationsPaul Smith2022-10-155-36/+35
| | | | | | | | | | | In C, a function declaration with () allows any set of arguments. Use (void) to mean "no arguments". * src/dep.h: Switch () to (void) for functions with no arguments. * src/makeint.h: Ditto. * src/os.h: Ditto. * src/shuffle.h: Ditto. * src/variable.h: Ditto.
* [SV 63157] Ensure temporary files are removed when signaledPaul Smith2022-10-153-19/+26
| | | | | | | | | | | | | | | | | | Original patch from Dmitry Goncharov <dgoncharov@users.sf.net>. When handling a fatal signal ensure the temporary files for stdin and the jobserver fifo (if in use) are deleted. * src/makeint.h (temp_stdin_unlink): Declare a new method. * src/main.c (temp_stdin_unlink): Delete the stdin temporary file if it exists. If the unlink fails and we're not handling a signal then show an error. (main): Call temp_stdin_unlink() instead of unlinking by hand. * src/commands.c (fatal_error_signal): Invoke cleanup methods if we're handling a fatal signal. * tests/scripts/features/output-sync: Test signal handling during output sync and jobserver with FIFO. * tests/scripts/features/temp_stdin: Test signal handling when makefiles are read from stdin.
* * src/posixos.c (job_root): Remember root jobserver instancesPaul Smith2022-10-152-13/+36
| | | | | | | | (jobserver_setup): Set job_root to true. (jobserver_clear): if we're the root instance and we're using a FIFO, unlink it. If we're not in a signal handler, free memory. (jobserver_acquire_all): Call jobserver_clear(). (sync_root): Rename from sync_parent for consistency.
* Set the proper type for variables set in signal handlersPaul Smith2022-10-152-2/+2
| | | | | | | * bootstrap.conf: Get gnulib's sig_atomic_t type checking M4 macro. * configure.ac: Invoke it. * src/makeint.h (handling_fatal_signal): Set the type correctly. * src/commands.c (handling_fatal_signal): Ditto.
* Clean up some warnings on Windows buildsPaul Smith2022-10-035-24/+44
| | | | | | | | | | | | | | * src/arscan.c (parse_int): Avoid conversion from int to char and check for overflow given a max value. (ar_scan): Check intmax sizes then cast to the proper type. (ar_member_touch): Get proper return type from ar_scan and cast it to off_t. * src/function.c (a_word_hash_cmp): Don't cast from size_t to int. (func_filter_filterout): Count using unsigned long to avoid casts. (construct_include_path): Explicitly cast to void*. * src/shuffle.c (reverse_shuffle_array): Use size_t index. * src/w32/subproc/sub_proc.c (process_wait_for_multiple_objects): Initialize the return value in case the count is 0.
* * src/misc.c (make_lltoa): Use printf format macro from makeint.hPaul Smith2022-10-031-10/+4
| | | | (make_ulltoa): Ditto.
* * src/arscan.c (ar_scan): Avoid sign comparison warningsPaul Smith2022-10-031-2/+2
| | | | | | Pacify 12.2.1 20220819 (Red Hat 12.2.1-2) -Wsign-compare by checking the readbuf() return explicitly for errors then casting from ssize_t. Initial patch provided by Paul Eggert <eggert@cs.ucla.edu>.
* Support systems with 32-bit long and 64-bit time_tPaul Eggert2022-10-034-35/+59
| | | | | | | | | | | | | | | | | | | | | | | | | Don't assume that time_t fits in long, as some hosts (e.g., glibc x86 -D_TIME_BITS=64) have 32-bit long and 64-bit time_t. * bootstrap.conf (gnulib_modules): Add largefile, to support files with timestamps after Y2038 on hosts with 32-bit long. * configure.ac: Do not call AC_SYS_LARGEFILE, as the largefile module does that for us. * src/makeint.h (PRIdMAX, PRIuMAX, SCNdMAX): Define if not already defined (taken from gnulib). * src/ar.c: Include intprops.h, for TYPE_MAXIMUM, as INTEGER_TYPE_MAXIMUM does not work on time_t without issuing a bunch of warnings. (ar_member_date): Check that result is in time_t range. (ar_member_date_1): Use intmax_t to hold the date. (ar_glob_match): Ditto. * src/arscan.c (VMS_function, VMS_function_ret, ar_scan) (parse_int, ar_scan, ar_member_pos, ar_member_touch) (describe_member): Convert long int to intmax_t. * src/file.c (file_timestamp_sprintf): Use intmax_t/uintmax_t instead of long/unsigned long for values that might be time_t. * src/arscan.c (ar_member_touch): Fix buffer overrun if the timestamp is too large.
* Add support for intmax_tPaul Smith2022-10-034-9/+39
| | | | | | | | * configure.ac: Ask autoconf to detect it. * src/config.ami.template: Add #define intmax_t for AmigaOS. * src/config.h-vms.template: Add #define intmax_t for VMS. * src/config.h.W32.template: Add #define intmax_t for Windows. * src/configh.dos.template: Add #define intmax_t for MS-DOS/DJPP.
* [SV 63098] Temporarily revert the change to pattern rule behaviorPaul Smith2022-10-021-19/+42
| | | | | | | | | | | | | | | | | | | | The fix for SV 12078 caused a backward-compatibility issue with some makefiles. In order to allow users to resolve this issue, revert that change for this release cycle: it will be reinstated in the next release cycle. Introduce a warning if we detect that the recipe of a multi-target pattern rule doesn't create all the targets. * NEWS: Announce the future backward-incompatibility. * doc/make.texi (Pattern Intro): Describe the behavior and that it will change in the future. * src/remake.c (check_also_make): Check for also_make targets that were not created and generate a warning. (update_goal_chain): Call the new function. (check_dep): Ditto. (update_file_1): Defer implicit rule detection until after we check all the also_make files (as it used to be). * tests/scripts/features/patternrules: Add tests of the new warning. Skip the tests for SV 12078.
* * src/filedef.h (is_ordinary_mtime): Check for "ordinary" mod time.Paul Smith2022-10-022-2/+3
| | | | * src/remake.c (update_file_1): Use the new macro.
* * src/remake.c (update_file_1): Return valid enum element, not 0.Paul Smith2022-10-021-6/+6
|
* * src/main.c (main): Add debug output for jobserver and osync.Paul Smith2022-10-022-8/+8
| | | | | * src/posixos.c (jobserver_setup): Remove redundant debug output. (jobserver_parse_auth): Ditto.
* [SV 63111] Ensure output-sync is cleaned on re-exec and recursionAndreas Schwab2022-10-022-2/+9
| | | | | | | | | | | | Now that output-sync is using a separate file as a mutex rather than stdout, ensure the new file descriptor is closed before re-exec and not inherited by children. * src/main.c (main): Call osync_clear() before re-exec. * src/posixos.c (osync_setup): Reset output-sync handle inheritance. (osync_parse_mutex): Ditto. Copyright-paperwork-exempt: yes
* * src/function.c: Change int to uint to align with gmk_func_ptrPaul Smith2022-10-011-8/+8
|
* Implement a simple xorshift 32bit random number generatorPaul Smith2022-10-013-5/+34
| | | | | | | | | | | | | | | | Avoid relying on the system random number generator for our random shuffle, so that the same seed gives the same results on all systems. This generator doesn't need to be amazing, just pretty good, so don't bother with xorshift* or xorshift+, etc. * src/makeint.h: Declare make_seed() and make_rand(). * src/misc.c (make_seed): Set the seed value for the RNG. (make_rand): Return the next random number. If the seed was not set initialize it first. * src/shuffle.c (shuffle_set_mode): If we don't get a seed from the user just leave it unset (0). (shuffle_deps_recursive): Use make_seed() not srand(). (random_shuffle_array): Use make_rand() not rand().
* [SV 63100] Set the floc of every created goal dep structDmitry Goncharov2022-09-251-0/+1
| | | | | | * src/read.c (eval): Initialize the goaldep floc pointer. * tests/scripts/features/loadapi: Verify that the floc is set after unloading and reloading dynamic objects.
* Provide new functions to convert long long to stringPaul Smith2022-09-254-39/+50
| | | | | | | | | | | | | | | | | | | | The previous attempt to use PRI* macros to avoid compiler-specific printf format specifiers didn't work because we are using raw long long type, not the uintX_t types. On systems where long and long long are the same size, uint64_t might be type "long" and PRId64 is just "ld". Instead write new functions that convert [unsigned] long long to a string and call those instead. * src/makeint.h: Declare make_lltoa() and make_ulltoa(). * src/misc.c (make_lltoa): New function that writes a long long value into a provided buffer. Return the buffer for ease-of-use. (make_ulltoa): Ditto, for unsigned long long. * src/function.c (func_wordlist): Call these new methods. Also rework the error strings so we share the translated string. * src/dir.c (print_dir_data_base): Call the new methods instead of using MSVC macros.
* Always restore global environ if we use vforkPaul Smith2022-09-251-12/+15
| | | | | | | | | | | | | | | | We may change the global environ variable in the child; when using vfork() this also sets it in the parent. Preserve the parent's environ in child_execute_job() so it takes effect for all callers. Reported by Denis Excoffier <bug-tar@Denis-Excoffier.org> Root cause found by Martin Dorey <Martin.Dorey@hitachivantara.com> * src/job.c (start_job_command): Remove save/restore of the parent environment. (child_execute_job): Add save/restore of the parent environment, if we use vfork(). * tests/scripts/functions/shell: Add a test the crashes if we don't reset environ after we run $(shell ...).
* * src/posixos.c (osync_clear): Don't close invalid FDs.Paul Smith2022-09-251-1/+1
|
* * src/misc.c (strncasecmp): Use size_t for length to match std.Paul Smith2022-09-252-2/+2
| | | | * src/misc.h (strncasecmp): Fix the declaration.
* Fix compiler warnings in the MS-Windows build.Eli Zaretskii2022-09-254-6/+16
| | | | | | | | | | | * src/w32/w32os.c (osync_get_mutex, osync_parse_mutex): Cast to DWORD_PTR when converting integers to HANDLEs and vice versa. * src/w32/pathstuff.c (w32ify): Pacify compiler warnings about 'strncpy'. * src/makeint.h (PRId64) [!HAVE_INTTYPES_H]: Define if undefined. * src/function.c (func_wordlist): Use PRId64 instead of %lld.
* Avoid compilation warningsEli Zaretskii2022-09-242-3/+3
| | | | | | * src/implicit.c: (pattern_search): * src/main.c: (main): Avoid compilation warnings for variables only used when compiling a debug version with assertions.
* [SV 56301] Fail if an included makefile can't be builtDmitry Goncharov2022-09-201-0/+1
| | | | | | | | | | Fail if a mandatory include file fails to be built even if it's built as part of a grouped target where the other include file is optional. * src/main.c (main): If a makefile doesn't build set any_failed. * tests/scripts/features/include: Add tests. * tests/scripts/options/dash-k: Stop after include build failure.
* [SV 12078, SV 62809] Rebuild grouped targets if any is missingPaul Smith2022-09-202-7/+25
| | | | | | | | | | | | | | | | | | | If any of a set of grouped targets is missing or out of date, even if make is not trying to build that target, rebuild them all. Ensure this is true for explicit grouped targets as well as pattern rule grouped targets. Original patch by Jonathan Gravel <jo@stashed.dev> * src/remake.c (update_file_1): After matching any pattern rules, go through the also_make targets and set noexist as needed. Also compute the oldest this_mtime. * tests/scripts/features/grouped_targets: Add regression tests. * tests/scripts/features/patternrules: Ditto. * tests/features/vpath: Rewrite to use modern run_make_test(). Add a test that we check for VPATH before implicit rule search. Move the tests in vpath2 and vpath3 into this suite. * tests/features/vpathplus: Rewrite to use modern run_make_test().
* [SV 13862] Implement the .WAIT special targetPaul Smith2022-09-127-36/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The next version of the POSIX standard defines parallel execution and requires the .WAIT special target as is implemented in some other versions of make. This implementation behaves similarly to others in that it does not create a relationship between targets in the dependency graph, so that the same two targets may be run in parallel if they appear as prerequisites elsewhere without .WAIT between them. Now that we support .WAIT it's trivial to also support prerequisites of the .NOTPARALLEL special target, which forces the prerequisites of those targets to be run serially (as if .WAIT was specified between each one). * NEWS: Announce the new .WAIT and .NOTPARALLEL support. * doc/make.texi (Parallel Disable): A new section to discuss ways in which parallel execution can be controlled. Modify cross-refs to refer to this section. * src/dep.h (struct dep): Add a new wait_here boolean. (parse_file_seq): Add PARSEFS_WAIT to check for .WAIT dependencies. * src/file.c (split_prereqs): Use PARSEFS_WAIT. (snap_deps): If .NOTPARALLEL has prerequisites, set .WAIT between each of _their_ prerequisites. (print_prereqs): Add back in .WAIT when printing prerequisites. * src/implicit.c (struct patdeps): Preserve wait_here. (pattern_search): Ditto. Use PARSEFS_WAIT when parsing prereqs for pattern rule expansion. * src/read.c (check_specials): Don't give up early: remembering to update these options is not worth the rare speedup. (check_special_file): If .WAIT is given as a target show an error-- once--if it has prereqs or commands. (record_files): Call check_special_file on each target. (parse_file_seq): If PARSEFS_WAIT is given, look for .WAIT prereqs. If we see one assume that we are building a struct dep chain and set the wait_here option while not putting it into the list. * src/remake.c (update_file_1): If wait_here is set and we are still running, then stop trying to build this target's prerequisites. * src/rule.c (get_rule_defn): Add .WAIT to the prerequisite list. * src/shuffle.c (shuffle_deps): Don't shuffle the prerequisite list if .WAIT appears anywhere in it. * tests/scripts/targets/WAIT: Add a test suite for this feature.
* [SV 63045] Reload each intact unloaded shared objectDmitry Goncharov2022-09-126-58/+87
| | | | | | | | | | | | | | | | | | | | | | | If makefile rules do not update an unloaded shared object, load it again. Avoid double loading of the same object if the setup function returns -1. * src/filedef.h (struct file): Add "unloaded" flag. * src/makeint.h (load_file): Take struct file *. (unload_file): Return int. * src/main.c (main): Reload unloaded shared objects if they weren't updated. * src/commands.c (execute_file_commands): Set "unloaded" and reset "loaded" when a shared object is unloaded. * src/read.c (eval): Set "loaded" and reset "unloaded" when a shared object is loaded. Add successfully loaded files to the db. * src/load.c (load_file): Check "loaded" to avoid double loading the same object. Fix a memory leak of string loaded. Return -1, rather than 1, if the object is already loaded. This fixes double loading of the same object when the setup routine returns -1. (load_object): Add a log message. (unload_file): Return an error on dlclose failure. Log a message. * tests/scripts/features/loadapi: Add new tests.
* [SV 63047] Fix shuffle of SECONDEXPANSION prerequisitesSergei Trofimovich2022-09-122-1/+11
| | | | | | | | | | | | | | Commit 07eea3aa4 `make --shuffle` prevented shuffling prerequisites that use .SECONDEXPANSION, since shuffle happens before expansion. This has two problems: 1. No shuffling happens for such prerequisites. 2. Use-after-free when outdated '->shuf' links are used. Add a reshuffle into expansion phase right after dependency changes. * src/file.c (expand_deps): Add reshuffle if dependencies change. * src/shuffle.c (identity_shuffle_array): Fix comment typo. * tests/scripts/options/shuffle: Add new SECONDEXPANSION test.
* * src/shuffle.c: Minor coding changes.Paul Smith2022-09-111-33/+25
|
* Support the MAKE_TMPDIR environment variablePaul Smith2022-09-111-13/+25
| | | | | | | | | | | Allow build systems to choose an alternative location for make to store its temporary files. * NEWS: Announce the new environment variable. * doc/make.texi (Temporary Files): Provide documentation. * src/misc.c (get_tmpdir): Split into a new function. Compute the temporary directory and store it in a static location. * tests/scripts/features/jobserver: Add a test of MAKE_TMPDIR.
* [SV 63044] load: Update .LOADED if the setup function returns -1Dmitry Goncharov2022-09-101-2/+3
| | | | | | | | * src/load.c (load_file): Update .LOADED if setup returns non-0. * tests/scripts/features/load: Change the return value based on an environment variable. Ensure that returning -1 still adds to .LOADED. Also add a test that verifies that make doesn't try to rebuild the loaded file if -1 is returned.
* [SV 63040] shell: Fall back to the callers environmentPaul Smith2022-09-101-1/+14
| | | | | | | | | | | | | | | | | | | If we detect a recursive variable reference when constructing the environment for the shell function, return the original value from the caller's environment. Other options such as failing, returning the empty string, or returning the unexpanded make variable value have been shown to not behave well in real-world environments. If the variable doesn't exist in the caller's environment, return the empty string. Found by Sergei Trofimovich <slyich@gmail.com> when testing older versions of autoconf. * NEWS: Clarify this behavior. * doc/make.texi (Shell Function): Ditto. Also add info about !=. * src/expand.c (recursively_expand_for_file): Search the caller's environment if we detect a recursive variable expansion. * tests/scripts/functions/shell: Add tests for this behavior.
* [SV 63016] Don't fail exporting to $(shell ...)Paul Smith2022-09-083-19/+30
| | | | | | | | | | | | | | | | The fix for SV 10593 caused recursive expansion errors when exporting a variable that contains a $(shell ...) invocation. If we see this type of recursion, ignore it and expand to the empty string rather than failing. * src/variable.h (env_recursion): New global variable. * src/variable.c (target_environment): If creating the environment for a $(shell ...) function increment env_recursion. Remove the check for expansion in a shell function context. * src/expand.c (recursively_expand_for_file): Check for recursive expansion in a $(shell ...) environment context and if present, show the verbose message and return the empty string. * tests/scripts/functions/shell: Add a test for this situation.
* Ensure debug output ends in newlinePaul Smith2022-09-083-9/+8
| | | | | | * src/load.c (load_object): Debug output strings must include \n. * src/variable.c (target_environment): Ditto. * src/posixos.c: Remove extra newlines.
* [SV 62840] Don't change IO buffering before printing versionPaul Smith2022-08-302-105/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If users run 'make --version | head -n1' they expect make to exit with a success (0) code. This works because the pipe forces the default buffering on stdout to be fully buffered so all the output is printed to the pipe in a single write(2) and won't fail. However due to output sync we forcibly set stdout to line buffered, which means if the reader closes the pipe fast enough make will exit with an error code because the write to stdout failed. Move the setup done in output_init() back into main() where it can be done in a proper order. Rework the order of operations during startup so that we check for help and version flags before we change the buffering. Clean up the behavior of print_usage(). Original changes from Dmitry Goncharov <dgoncharov@users.sf.net>. * src/main.c (switches): Don't send --version in the environment. (print_usage): Add a blank line after the version before the usage. Move the die() into this function since we always die() afterward. Note the die will flush so no need to do it explicitly. (print_version): The caller will fflush when appropriate. (close_stdout): Move from output.c so it is installed early. (decode_switches): Only call print_usage on error, not for --help. (main): Install the close_stdout handler immediately after start. Move the calls to print_usage() due to --help and --version to be called immediately after we decode the switches. Move the buffer set here from output_init(), immediately after we know we'll be running. * src/output.c (output_init): Move buffer setting to main(). (close_stdout): Move to main().
* Rework output sync to lock a temp file on POSIXPaul Smith2022-08-3010-469/+485
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some POSIX systems do not allow locks to be taken on non-files, such as pipes. This is a problem since very often make is invoked with its stdout redirected to a pipe. Also, if stdout is redirected to a file that already has a lock on it for some other reason (perhaps a shared file such as /dev/null) it can cause a hang. This means our previous method of locking stdout, although it had some nice advantages, is not portable enough. Instead, use a temporary file and take the lock on that. We pass the name of the file to child make processes. On Windows we continue to use a shared mutex for output sync. Remove POSIX emulation functions like fcntl from Windows; instead follow the lead of the jobserver and create an interface in os.h for output sync, and move the OS-specific content to posixos.c and w32os.c. * NEWS: Add a note. * src/makeint.h (ALL_SET): Check that all bits are set. * src/os.h: Add bits for checking the state of stdin/stdout/stderr. Add prototypes for OS-specific output sync methods. * src/posixos.c (check_io_state): Determine the status of stdin, stdout, stderr an return a suite of bits describing them. (osync_enabled): If the global variable holding the FD of the lock file (osync_handle) is valid return true. (osync_setup): Create a temporary file and remember its name in a global variable (osync_tmpfile), and set osync_handle. (osync_get_mutex): If output sync is enabled, return the filename of the lock file prefixed with "fnm:" to denote a filename. (osync_parse_mutex): If the provided filename has the wrong format disable output sync. Else open the lock file and set osync_handle. (osync_clear): Close osync_handle. If we're the parent make, then also unlink the temporary file. (osync_acquire): Take a lock on the osync_handle descriptor. (osync_release): Release the lock on the osync_handle descriptor. (fd_set_append): Add APPEND mode to a file descriptor. * src/w32/w32os.c: Perform the same actions as posixos.c, copying the details from src/w32/compat/posixfcn.c. Use a mutex rather than locking a temporary file. * src/output.h: Remove all the OS-specific content. * src/output.c: Remove all the OS-specific content. (set_append_mode): Remove and replace with fd_set_append(). (sync_init): Remove and replace with check_io_state(). (acquire_semaphore): Remove and replace with osync_acquire(). (release_semaphore): Remove and replace with osync_release(). (setup_tmpfile): If the IO state is not obtained, get it. If stdout and/or stderr are valid, set up a tempfile to capture them. (output_init): Set io_state if not set already, and check it when deciding whether to close stdout on exit. * src/main.c (main): If we're syncing, set up the mutex using the new osync_setup() / osync_parse_mutex() methods. (prepare_mutex_handl_string): Replace with osync_parse_mutex(). (die): Call osync_clear(). * src/w32/compat/posixfcn.c: Remove implementations of fcntl(), record_sync_mutex(), create_mutex(), and same_stream().
* Add get_tmpfd() and allow anonymous temp filesPaul Smith2022-08-306-143/+164
| | | | | | | | | | | | | | | | | | | | | | | | The output sync feature wants a file descriptor not a FILE*. We were using tmpfile() but this returns FILE* which means we needed to dup() the descriptor then fclose() the original, which is just unnecessary overhead for every command we run. Create a get_tmpfd() method that returns a file descriptor directly by using mkstemp() if available, else do the best we can. Also allow anonymous temp files if the filename pointer is NULL. This causes the file to be unlinked. On Windows this requires a special open so add an os_anontmp() method to handle this. * src/makeint.h: Add prototype for get_tmpfd(). * src/misc.c (get_tmpfd): If we have mkstemp() use that, else just open(2). If we don't want to keep the filename, unlink the file. (get_tmpfile): Use get_tmpfd() if we have fdopen(), else use fopen(). * src/output.c (output_tmpfd): Call get_tmpfd() with NULL. * src/os.h (os_anontmp): On Windows make this a function, else fails. * src/w32/compat/posixcfn.c (tmpfile): Move to w32os.c:os_anontmp(). * src/w32/w32os.c (os_anontmp): Create a temp file that will be deleted when the process exits, and return a file descriptor to it.
* [SV 62908] Don't free fifo_name twice during error handlingDmitry Goncharov2022-08-221-0/+1
| | | | | * src/posixos.c (jobserver_setup): Set fifo_name to NULL after free. * tests/scripts/features/jobserver: Add tests.
* Support implementing the jobserver using named pipesPaul Smith2022-08-026-58/+194
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using anonymous pipes for jobserver support has some advantages: for example there is nothing on disk that needs to be cleaned up. However it has many obscure problems, related to the fact that in order for it to work we need to ensure these resources are properly passed through to child processes that want to use the jobserver. At the same time we don't want to pass the pipe to process which DON'T know about the jobserver. Other processes can open file descriptors which we then think are our jobserver, but aren't. And, we open the pipe file descriptors in blocking mode which doesn't work for all users. See issues such as SV 57178, SV 57242, and SV 62397 To avoid these issues, use named pipes (on systems where they are available) instead of anonoymous pipes. This simplifies many things: we never need to pass open file descriptors to our children; they can open the jobserver named pipe. We don't need to worry about recursive vs. non-recursive children. Users don't have to "pass through" the resources if they are invoking sub-makes. Each child can open its own file descriptor and set blocking as needed. The downside is the named pipe exists on disk and so must be cleaned up when the "top-level" make instance exits. In order to allow make to continue to be used in build systems where older versions of GNU make, or other tools that want to use the jobserver, but don't understand named pipes, introduce a new option --jobserver-style that allows the user to choose anonymous pipes. * NEWS: Announce the change and the --jobserver-style option. * doc/make.1: Add --jobserver-style documentation. * doc/make.texi (Special Variables): Add missing items to .FEATURES. (Options Summary): Add --jobserver-style. (POSIX Jobserver): Named pipes, changes to --jobserver-auth, and the --jobserver-style option. (Windows Jobserver): Document --jobserver-style for Windows. * configure.ac: Check for mkfifo. * src/config.h-vms.template: Undefined HAVE_MKFIFO. * src/config.h.W32.template: Ditto. * src/main.c: Add jobserver-style as a new command line option. (main): Add jobserver-fifo to .FEATURES if supported. Pass the style option to jobserver_setup(). * src/os.h (jobserver_setup): Accept a style string option. * src/posixos.c (enum js_type): Enumeration of the jobserver style. (js_type): Which style we are currently using. (fifo_name): The path to the named pipe (if in use). (jobserver_setup): If no style is given, or "fifo" is given, set up a named pipe: get a temporary file and use mkfifo() on it, then open it for reading and writing. If something fails fall back to anonymous pipes. (jobserver_parse_auth): Parse jobserver-auth to determine the style. If we are using a named pipe, open it. If we're using anonymous pipes ensure they're valid as before. (jobserver_get_invalid_auth): Don't invalidate the jobserver when using named pipes. (jobserver_clear): Clean up memory used for named pipes. (jobserver_acquire_all): Unlink the named pipe when done. * src/w32/w32os.c (jobserver_setup): Check the style argument. * tests/scripts/features/jobserver: Use --jobserver-style to test the anonymous pipe behavior, and also test named pipe/semaphore behavior. Check invalid jobserver-style options. * tests/scripts/functions/shell: Use --jobserver-style to test the anonymous pipe behavior, and also test named pipe/semaphore behavior.
* Enhance get_tmpfile() and add get_tmppath()Paul Smith2022-08-024-55/+77
| | | | | | | | | | | | | | | | | | Move all the logic on creating temporary files into misc.c, and add a new function get_tmppath() that returns the pathname to a temporary file without creating or opening it. * src/makeint.h: Add a declaration for get_tmppath(). Remove the template argument from get_tmpfile(): it will compute its own. * src/main.c (main): Remove the logic for computing templates. * src/vmsjobs.c (child_execute_job): Ditto. * src/misc.c (get_tmptemplate): New function to return an allocated template string for use with various mktemp-style functions. (get_tmppath): Return an allocated path to a temporary file, but do not create it. Generally this should be avoided due to TOCTOU issues. (get_tmpfile): Use get_tmptemplate() to generate a template rather than using one passed in. If we don't have mkstemp() then use get_tmppath() to compute the path of a temp file.
* [SV 62706] Only second-expand targets that might be builtDmitry Goncharov2022-07-303-38/+25
| | | | | | | | | | | | | | | | | | | | | Second-expand only the prerequisites of the targets being built. Defer second-expanding the prerequisites of targets until we need to decide if they should be built. * NEWS: Mention the change in behavior. * doc/make.texi (Secondary Expansion): Document the new behavior. * src/filedef.h (struct file): Add flag snapped. (expand_deps): Declare a function to second expand the prerequisites of a target. * src/file.c (rehash_file): Merge flag snapped. (expand_deps): Remove qualifier static. Check flag snapped. (snap_deps): Remove the loop which performed second expansion for all targets. * src/remake.c (update_file_1): Second expand the prerequisites of the considered target. * tests/scripts/features/se_explicit: Add tests. * tests/scripts/features/se_implicit: Ditto. * tests/scripts/features/se_statpat: Ditto.
* Disable the jobserver in non-recursive childrenPaul Smith2022-07-309-45/+146
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Savannah issues such as SV 57242 and SV 62397 show how passing references to closed file descriptors via the --jobserver-auth option in MAKEFLAGS can lead to problematic outcomes. When computing the child environment for a non-recursive shell, add an extra option to MAKEFLAGS to disable the file descriptors for the jobserver. Unfortunately this doesn't modify the value of the make variable MAKEFLAGS, it only modifies the value of the sub-shell environment variable MAKEFLAGS. This can lead to confusion if the user is not considering the distinction. * src/makeint.h: Publish the jobserver-auth value. Add a global definition of the name of the command line option. * src/os.h (jobserver_get_invalid_auth): New function to return a string invalidating the jobserver-auth option. * src/w32/w32os.c (jobserver_get_invaid_auth): Implement it. On Windows we use a semaphore so there's no need to invalidate. * src/posixos.c (jobserver_parse_auth): If we parse the invalid auth string, don't set up the jobserver. (jobserver_get_invalid_auth): Return an invalid option. * src/variable.h (target_environment): Specify if the target environment is for a recursive shell or non-recursive shell. * src/variable.c (target_environment): Move checking for MAKELEVEL into the loop rather than doing it at the end. Along with this, check for MAKEFLAGS and MFLAGS, and update them based on whether we're invoking a recursive or non-recursive child, and also on whether it's necessary to invalidate the jobserver. * src/function.c (func_shell_base): Shell functions can never be recursive to pass 0 to target_environment(). * src/job.c (start_job_command): Specify whether the child is recursive when calling target_environment(). * src/main.c: Export jobserver_auth. sync_mutex doesn't need to be exported. Use the global definition for the option name. * tests/scripts/variables/MAKEFLAGS: Add tests for $MAKEFLAGS.
* Ensure that MAKEFLAGS is set when invoking $(shell ...)Paul Smith2022-07-301-23/+17
| | | | | | | | | * src/main.c (main): Don't reset the jobserver if the number of slots has not changed. (define_makeflags): Add all normal flags even when ALL is not set. * tests/scripts/functions/shell: Test invoking make in $(shell ...). * tests/scripts/variables/MAKEFLAGS: Test the value of MAKEFLAGS in $(shell ...).
* Take advantage of mempcpy() and stpcpy()Paul Smith2022-07-3011-76/+50
| | | | | | | | | | | | | | | | | | | | | | | * src/makeint.h (stpcpy): Add missing declaration. * src/amiga.c (MyExecute): Avoid extra strlen using stpcpy. * src/function.c (func_shell_base): Ditto. (func_error): Use memcpy() not strcpy() when we know the length. * src/job.c (construct_command_argv_internal): Use stpcpy(). * src/main.c (main): Ditto. (define_makeflags): Ditto. * src/variable.c (print_target_variables): Use memcpy() when we know the length. * src/commands.c (set_file_variables): Use mempcpy(). * src/expand.c (variable_buffer_output): Ditto. * src/file.c (expand_deps): Ditto. * src/function.c (abspath): Ditto. (handle_function): Ditto. * src/implicit.c (pattern_search): Ditto. * src/job.c (construct_command_argv_internal): Use mempcpy() and don't add multiple spaces when there are no shell flags. * src/main.c (decode_env_switches): Use mempcpy() to simplify. (define_makeflags): Ditto. * src/variable.c (selective_vpath_search): Ditto.
* Convert HAVE_GETLOADAVG to HAVE_DECL_GETLOADAVGPaul Smith2022-07-094-44/+126
| | | | | | | | | | | We want to know if getloadavg() is declared, not if we have it. * configure.ac: Add it as a define (why is this not the default?) * src/job.c: Conditionalize declaration on HAVE_DECL_GETLOADAVG. * config.ami.template: Rename HAVE_GETLOADAVG to HAVE_DECL_GETLOADAVG. * config.h-vms.template: Ditto. * config.h.W32.template: Ditto. Clean up and add new elements from latest config.h.
View Lorry Controller Status