summaryrefslogtreecommitdiff
path: root/tests/test_driver.pl
Commit message (Collapse)AuthorAgeFilesLines
* * tests/test_driver.pl: Show info about failed testsPaul Smith2023-05-071-26/+36
| | | | | Remove working directories for skipped tests. We use different directories per test so don't keep rmfiles.
* tests: Run each file in a separate directoryPaul Smith2023-04-021-51/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid cross-contamination between test files by creating a new working directory for each file, and setting it as the current directory before starting the tests in that file. Rename the test output as tNNN.{base,log,diff,mk} where NNN is a test number starting with 001 for the first test. It is slightly more annoying to find diff files since you can't use autocomplete directly but it is simpler to match things. Detect the source directory as the location of the test_driver.pl script, so remove the separate -srcdir option. * Makefile.am: Remove hacks to create symlinks when building out-of-tree, and remove -srcdir option from run_make_tests. * tests/test_driver.pl: Locate $srcpath based on __FILE__, then compute $toppath as its parent. Set $scriptpath under $srcpath and $workpath under the current directory. Toss $*_filename and modify get_logfile() etc. to use the suffix directly. Add a chdir() around the invocation of the test. * tests/run_make_tests.pl: Throw out the -srcdir option and use $srcpath set in test_driver.pl. The #WORK# helper is no longer useful so remove it. Set #PWD# to the current working dir. Always search the local directory and $srcpath for config-flags.pm. Use $srcpath for finding the thelp.pl script. * tests/scripts/features/vpath: Don't put things in work/ as it is no longer a subdirectory. * tests/scripts/features/vpathgpath: Ditto. * tests/scripts/features/vpathplus: Ditto. * tests/scripts/misc/general1: Ditto. * tests/scripts/misc/general2: Ditto. * tests/scripts/options/dash-k: Ditto. * tests/scripts/options/symlinks: Use $testpath as the working directory. * tests/scripts/variables/GNUMAKEFLAGS: Use the test helper to display env var values (grepping for GNUMAKEFLAGS finds extra things now that it is our current working directory).
* Adjust output strings to be alignedPaul Smith2023-04-021-6/+6
| | | | | | | | Change error and fatal messages to start with lowercase and not end with a period. Note a few very common messages were left as-is, just in case some other tools parse them. Also modify the test known-good-output to satisfy the messages.
* * tests/scripts/misc/general4: Only test empty PATH on UNIXPaul Smith2023-02-201-1/+0
|
* * tests/test_driver.pl: Convert z/OS answers for regex outputPaul Smith2023-01-151-7/+10
| | | | | | Allow z/OS customizations to apply to regex output matching. * tests/scripts/features/output-sync: Use a regex for Terminated. * tests/scripts/features/temp_stdin: Ditto.
* [SV 17448] Ignore whitespace around $(file ...) namesPaul Smith2023-01-081-3/+3
| | | | | | | | | | The other issues related to whitespace reported in this bug are not addressed by this change. * src/functions.c (func_file): Strip whitespace from the start and end of the filename provided to the $(file ...) function. * tests/scripts/functions/file: Add tests for this. * tests/test_driver.pl: Use 3-arg version of open().
* [SV 62654] Support GNU Make on z/OSPaul Smith2023-01-081-10/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Original patches provided by Igor Todorovski <itodorov@ca.ibm.com> Reworked by Paul Smith <psmith@gnu.org>. Thanks to IBM for providing a test system. * NEWS: Announce support. * AUTHORS: Ditto. * README.zOS: Provide details on building GNU Make on z/OS. * build.sh (get_mk_var): z/OS sh has a strange bug which causes it to generate extra lines of output: rework the function to print output as we compute it instead of collecting it into a variable, which works around this bug. * src/makeint.h: Declare MK_OS_ZOS if we're building for z/OS. * src/arscan.c: Don't include <ar.h> on z/OS. * src/job.c: We can't change environ in ASCII mode on z/OS. * src/main.c: Ditto. Also we can't use pselect() on z/OS. * src/posixos.c: pselect() seems to hang on z/OS: don't use it. * tests/run_make_tests.pl: Handle different exit codes on z/OS. * tests/test_driver.pl: Preserve some special z/OS env.vars. Add special checks to output comparisons when on z/OS. * tests/scripts/features/archives: Don't validate names. Don't try to compile empty files as IBM compilers complain. * tests/scripts/features/shell_assignment: Fix octal value of #. * tests/scripts/features/temp_stdin: Don't print "term". * tests/scripts/functions/shell: Handle shell exit codes. * tests/scripts/targets/ONESHELL: Ditto. * tests/scripts/targets/POSIX: sh -x prints differently. * tests/scripts/variables/SHELL: Ditto.
* Update the copyright year on all filesPaul Smith2023-01-011-1/+1
|
* Apply spelling fixes discovered by CodespellPaul Smith2022-12-311-1/+1
| | | | | | | | | | | | | | | | | | * maintMakefile: Apply spelling fixes. * src/file.c: Ditto. * src/misc.c: Ditto. * src/remake.c: Ditto. * src/vmsjobs.c: Ditto. * src/w32/pathstuff.c: Ditto. * tests/test_driver.pl: Ditto. * tests/run_make_tests.com: Ditto * tests/scripts/features/implicit_search: Ditto * tests/scripts/features/output-sync: Ditto * tests/scripts/features/patternrules: Ditto * tests/scripts/features/se_explicit: Ditto * tests/scripts/features/statipattrules: Ditto * tests/scripts/functions/foreach: Ditto * tests/scripts/variables/MAKEFLAGS: Ditto
* tests [WINDOWS32]: Support Strawberry Perl on WindowsPaul Smith2022-12-201-10/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Strawberry Perl has some different behaviors from ActiveState Perl which impact the test suite: - Avoid Perl's chomp() as it may not remove CRs; chomp() may remove only the final NL but not the CR in a CRNL line ending. - Strawberry Perl doesn't support ActiveState's system(1, ...) form. - Strawberry Perl (or msys?) does something weird with "/tmp" when provided to exec(), replacing it with the user's %TEMP%. - Strawberry Perl uses msys paths like /c/foo instead of C:\foo. * tests/test_driver.pl (get_osname): Strawberry Perl uses 'msys' as its $^O so if we see that use a port of 'W32'. (_run_with_timeout): Strawberry Perl doesn't support the special system(1, ...) form of system() so use POSIX standard fork/exec. (compare_answer): Paths generated by Strawberry Perl use msys path format (e.g., /c/foo instead of C:\foo); check for those differences and compare RE against both the unmodified and modified log. * tests/run_make_tests.pl (set_defaults): Switch from chomp to s/// to remove CRNL and NL line endings. * tests/scripts/features/errors: Executing directories on Strawberry will give an error; translate it to Windows error output format. * tests/scripts/features/output-sync: Ditto. * tests/scripts/features/temp_stdin: Ditto. * tests/scripts/functions/realpath: Ditto. * tests/scripts/options/dash-I: Ditto. * tests/scripts/variables/INCLUDE_DIRS: Ditto. * tests/scripts/misc/close_stdout: /dev/full is reported as existing on Strawberry Perl, but it doesn't do anything. Skip the test. * tests/scripts/variables/MAKEFLAGS: When an argument containing /tmp is passed to a program via exec(), something replaces it with the expansion of the %TEMP% variable. Instead of using /tmp create a local directory to use.
* * tests/test_driver.pl: Remember if something failed and report itPaul Smith2022-12-201-0/+12
|
* tests: Don't convert \ to / when checking regex'sPaul Smith2022-11-131-153/+133
| | | | | | | | | | | | | When tests compare the output they will try converting backslashes to slashes to see if that works. When we compare using regex's, we can't do that because backslashes can escape special characters. * tests/test_driver.pl (compare_output): Clean up this function. (compare_answer_vms) [VMS]: Comparing answers on VMS is complex; move all of it into its own function returning 0/1. (compare_answer): A new function to compare answers: return 0/1. Remember the CRLF->LF conversion forever; only check \ -> / when we compare strings, not regex's.
* Fix issues found by ASAN and CoverityPaul Smith2022-10-311-1/+1
| | | | | | | | | | | | | | | * tests/test_driver.pl: Preserve the LSAN_OPTIONS variable. * tests/scripts/targets/ONESHELL: Don't set a local variable. * tests/scripts/functions/let: Test empty let variable. * src/posixos.c (osync_parse_mutex): Free existing osync_tmpfile. * src/misc.c (get_tmpfd): Set umask() before invoking mkstemp(). * src/ar.c (ar_parse_name): Check invalid name (shouldn't happen). * src/function.c (define_new_function): Free previous function entry when replacing it with a new one. * src/job.c (child_execute_job): Initialize pid for safety. (construct_command_argv_internal): In oneshell mode ensure that the returned argv has the right format (0th element is a pointer to the entire buffer).
* tests: Avoid the use of File::Temp->newdir()Paul Smith2022-10-291-4/+0
| | | | | | | | | | This was added in Perl 5.8 but some systems still only provide older versions such as Perl 5.6. We don't really need it anyway. Paul Eggert <eggert@cs.ucla.edu> reported this issue. * tests/README: Update this to be a bit more modern. * tests/test_driver.pl: Delete the $TEMPDIR variable. * tests/scripts/features/temp_stdin: Use $temppath not $TEMPDIR.
* Increase the test framework timeout from 5s to 60sPaul Smith2022-10-251-3/+4
| | | | | | | | | | | | It seems that some of the test environments hit the 5s timeout on some tests. Since it doesn't really matter, as long as we don't hang forever, increase the timeout to 60s. * tests/test_driver.pl: Increase $test_timout to 60. We don't need to handle VMS timeouts specially anymore. * tests/scripts/features/parallelism: We don't need to override the default timeout anymore. * tests/scripts/features/patternrules: Remove confusing comment.
* Update URLs to use https rather than httpPaul Smith2022-10-181-1/+1
| | | | | * (all): Change http:// to https:// * README.W32: Remove invalid link to mingw.org.
* * tests/test_driver.pl (toplevel): [SV 63156] Keep temp env varsDmitry Goncharov2022-10-081-3/+3
|
* test_driver: check for leftover temp files after each testPaul Smith2022-08-291-7/+47
| | | | | | | | | | | | | | | | | | | | Reset the temp directory for every test to a local directory, then after each test see if any new temp files were created and not deleted: if they were then fail the test. Rather than delete the temp files we leave them there and avoid reporting files that were seen before, so the user can investigate them. Rewrite the temp_stdin tests to rely on this built-in behavior rather than implementing the checks directly. * tests/test_driver.pl: Create a $TEMPDIR variable pointing to a temporary directory outside the test temp directory. (toplevel) Before starting any tests create a temp directory and set the POSIX and Windows temp directory environment variables to use it. (compare_output) Check the contents of the temp directory. If any new files have appeared, fail the test. * tests/scripts/features/temp_stdin: Remove check_tempfile() and all users of it, plus setting of temp environment variables.
* * tests/test_driver.pl: Don't freak if diff can't be foundPaul Smith2022-06-181-24/+49
|
* * tests/test_driver.pl (run_all_tests): Keep one copy of STDINPaul Smith2022-04-241-24/+28
|
* * tests/test_driver.pl (toplevel): [SV 62174] Enforce the C localePaul Smith2022-04-241-0/+2
|
* [SV 62173] Fix archive tests for non-GNU systemsPaul Smith2022-04-241-7/+8
| | | | | | | | | | | | | Original patch from Dmitry Goncharov <dgoncharov@users.sf.net> Fix the ar flags to match the default values in make. If the U option is supported, add it to the end not the beginning, since ARFLAGS now starts with a "-". * tests/test_driver.pl: Add defaults for global variables. (get_osname): Preserve $osname if it's set from $^O * tests/scripts/features/archive: Set the default arflags. If "U" is available add it to the end of the flags.
* tests: Preserve Windows temp environment variablesPaul Smith2022-02-271-5/+5
| | | | | | | * src/main.c (main): Show the temp filename on error. Also on Windows prefer TMP, then TEMP, and only lastly TMPDIR. * tests/test_driver.pl: Add TMP, TEMP, and USERPROFILE to the list of environment variables to preserve.
* * tests/test_driver.pl: Support STDIN redirection.Paul Smith2022-02-261-6/+14
| | | | | | Before running tests, duplicate STDIN then reset it after the test completes. Also when -keep is provided, preserve the base and run files.
* * <all>: Update copyright notices.Paul Smith2022-02-101-1/+1
|
* tests: Enhance output to contain filename/line number infoPaul Smith2021-11-271-5/+26
| | | | | | | | * tests/scripts/options/dash-I: Use string comparison operator. * tests/test_driver.pl (cmd2str): New method to convert a command line to a string with proper quoting. * tests/run_make_tests.pl (create_command): Call cmd2str() to format the command. Add the filename/line number to the output.
* tests: Simplify customization of %ENVPaul Smith2020-04-011-15/+4
| | | | | | | | | | | | | | | | | | | | | | Rather than having an %extraENV that is added to the default %ENV and resetting %ENV _before_ each test, allow the test setup to modify %ENV directly as needed then reset %ENV _after_ each test. * tests/test_driver.pl: Remove unused %extraENV. (resetENV): Don't add in %extraENV. (_run_command): Reset after we run the command rather than before. * tests/scripts/features/export: Convert %extraENV to %ENV * tests/scripts/features/jobserver: Ditto * tests/scripts/features/parallelism: Ditto * tests/scripts/features/targetvars: Ditto * tests/scripts/functions/eval: Ditto * tests/scripts/functions/foreach: Ditto * tests/scripts/functions/origin: Ditto * tests/scripts/misc/general4: Ditto * tests/scripts/options/dash-e: Ditto * tests/scripts/targets/POSIX: Ditto * tests/scripts/variables/GNUMAKEFLAGS: Ditto * tests/scripts/variables/SHELL: Ditto
* Apply spelling corrections from Fossies spellcheckPaul Smith2020-01-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | See https://fossies.org/features.html#codespell Spelling issues in Git commit messages or lib/* source are not applied. * README.OS2.template: Apply spelling corrections. * README.VMS: Ditto. * src/commands.c: Ditto. * src/config.ami.template: Ditto. * src/configh.dos.template: Ditto. * src/job.c: Ditto. * src/job.h: Ditto. * src/read.c: Ditto. * src/variable.c: Ditto. * src/vms_exit.c: Ditto. * src/vms_export_symbol.c: Ditto. * src/vms_progname.c: Ditto. * src/vmsfunctions.c: Ditto. * src/vmsjobs.c: Ditto. * src/w32/pathstuff.c: Ditto. * tests/scripts/variables/automatic: Ditto. * tests/test_driver.pl: Ditto.
* * tests/test_driver.pl: Enhance error messagesDmitry Goncharov2020-01-051-5/+12
|
* Update copyright statements for 2020Paul Smith2020-01-031-1/+1
|
* * tests/test_driver.pl: Compute full path to Perl interpreterPaul Smith2020-01-031-1/+27
|
* * test/test_driver.el (_run_with_timeout): Show error message.Paul Smith2019-09-221-1/+1
|
* Refresh the test suite framework implementation.Paul Smith2019-09-161-697/+519
| | | | | | | | | | | | | | | | | | | | | | | | | | Go through both run_make_tests.pl and test_driver.pl and slightly modernize the Perl and clean up indentation etc. Fix a number of warnings in the test scripts detected by running with -w. * tests/test_driver.pl: Move make error string detection out of the base test driver. (run_all_tests): Ensure that we always look for tests in the cwd. * tests/run_make_tests.pl: Use File::Spec for path manipulations. Correctly use setlocale() when detecting error strings. Get configuration from the config-flags.pm file not config.status. * tests/scripts/features/archives: Use new $cwddir variable. * tests/scripts/features/reinvoke: Add missing semicolon. * tests/scripts/features/vpath2: Avoid non-existent variable. * tests/scripts/functions/foreach: Escape variables. * tests/scripts/misc/bs-nl: Remove non-existing \v escape sequence. * tests/scripts/misc/general4: Use handy create_file(). * tests/scripts/options/dash-C: Use Cwd/$cwddir. * tests/scripts/options/dash-I: Use subst_make_string() and #PWD#. * tests/scripts/options/symlinks: Use File::Spec. * tests/scripts/targets/DEFAULT: Use create_file and run_make_test. * tests/scripts/variables/CURDIR: Use run_make_test. * tests/scripts/variables/automatic: Remove extraneous "\". * tests/scripts/vms/library: Remove extra "my" and extraneous "\".
* * tests/test_driver.pl: Remember error for "running" a directory.Paul Smith2019-09-161-0/+8
| | | | * tests/scripts/features/errors: Check errors "running" a directory.
* * src/job.c (child_execute_job): Use errno from find_in_given_path().Paul Smith2019-09-161-0/+8
| | | | | * tests/test_driver.pl: Determine non-executable error message. * tests/scripts/features/errors: Use the local non-executable error.
* * src/hash.c (jash_string): [SV 54980] Avoid ASAN errorPaul Smith2019-05-191-0/+2
|
* Update copyright statements for 2019Paul Smith2019-05-191-1/+1
|
* * all: Update Copyright statements for 2018Paul Smith2018-07-011-1/+1
|
* Update copyright statements for 2017.Paul Smith2017-07-091-1/+1
|
* * tests/test_driver.pl: Useful error if given an invalid test name.Paul Smith2017-06-041-0/+2
|
* Add test suite support to WindowsPaul Smith2017-06-041-49/+135
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * main.c (main): Sanitize program name detection on Windows. * makeint.h: 'program' is a const string on all platforms now. * tests/run_make_tests.bat: Windows bat file to invoke tests * tests/test_driver.pl: Obtain system-specific error messages. (get_osname): Compute the $port_type here. Add more $osname checks for different Windows Perl ports. (_run_command): Rewrite the timeout capability to work properly with Windows. Don't use Perl fork/exec; instead use system(1,...) which allows a more reliable/proper kill operation. Also, allow options to be given as a list instead of a string, to allow more complex quoting of command-line arguments. * tests/run_make_tests.pl (run_make_with_options): Allow options to be provided as a list in addition to a simple string. (set_more_defaults): Write sample makefiles and run make on them instead of trying to run echo and invoking make with -f-, to avoid relying on shell and echo to get basic configuration values. Also create a $sh_name variable instead of hard-coding /bin/sh. * tests/scripts/features/archives: Skip on Windows. * tests/scripts/features/escape: Use list method for passing options. * tests/scripts/features/include: Use system-specific error messages. * tests/scripts/features/output-sync: "Command not found" errors generate very different / odd output on Windows. This needs to be addressed but for now disable these tests on Windows. * tests/scripts/functions/abspath: Disable on Windows. * tests/scripts/functions/file: Use system-specific error messages. * tests/scripts/functions/shell: "Command not found" errors generate very different / odd output on Windows. This needs to be addressed but for now disable these tests on Windows. * tests/scripts/misc/close_stdout: Disable on Windows. * tests/scripts/options/dash-k: Use system-specific error messages. * tests/scripts/options/dash-l: Disable on Windows. * tests/scripts/options/eval: Use list method for passing options. * tests/scripts/options/general: Skip some non-portable tests. * tests/scripts/targets/ONESHELL: Skip some non-portable tests. * tests/scripts/targets/POSIX: Skip some non-portable tests. * tests/scripts/variables/MAKEFILES: Skip some non-portable tests. * tests/scripts/variables/SHELL: Use a makefile not -f- for testing.
* Portability changes for the test suite.Paul Smith2016-12-261-0/+24
| | | | | | * tests/test_driver.pl: Save error strings for later comparison. * tests/run_make_tests.pl: Create portable commands for later use. * tests/*: Use these new variables.
* Update Copyright statements for the new year.Paul Smith2016-02-281-1/+1
|
* Update Copyright statements for 2014.Paul Smith2014-09-301-1/+1
|
* Update the regression test harness to support VMS.John Malmberg2014-07-071-18/+248
| | | | | | | | | | | | | | | | | * config_flags_pm.com, test_make.com: set up and run the regression test environment on VMS. * tests/run_make_tests.pl [VMS]: Use an alternate rmdir() implementation on VMS. (run_make_with_options) [VMS]: Provide VMS-specific quoting and shell invocations. (set_more_defaults) [VMS]: Set default values when running on VMS. * tests/test_driver.pl (vms_get_process_logicals) [VMS]: Retrieve the proper values from %ENV on VMS. (resetENV) [VMS]: Use it. (toplevel) [VMS]: Fix a bug with opendir() on some logical_devices. (compare_output) [VMS]: Convert VMS test output to a "standard" format. (_run_command) [VMS]: Handle signals and exit codes the VMS way. (remove_directory_tree_inner) [VMS]: Unlink all versions of the file.
* GNU Make release 4.0.4.0Paul Smith2013-10-091-5/+5
|
* Clean up some tests for use on Windows.Paul Smith2013-10-051-107/+111
|
* Portability enhancements for Mac OSX.Paul Smith2013-09-221-1/+2
| | | | | | Don't dup stdout into stderr in the test suite. Don't rely on $port_type eq "UNIX" to mean case-preserving. Check against the real output of ar when creating archives.
* [SV 27374] Fatal immediately on unrecoverable fopen() errors.Paul Smith2013-09-151-25/+28
|
* Update source file format: remove TABs, use GNU coding styles.Paul Smith2013-05-171-8/+8
|
View Lorry Controller Status