summaryrefslogtreecommitdiff
path: root/tests/defs
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'master' into testsuite-workStefano Lattarini2011-12-231-52/+3
|\ | | | | | | | | | | | | * tests/gettext-macros.test: In the generated `get.sh' file, use `skip_all_', not `skip_'. * tests/libtool-macros.test: Likewise. * tests/list-of-tests.mk: Update.
| * Merge branch 'maint' into master.Stefano Lattarini2011-12-221-49/+3
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * m4/regex.m4: Bump serial number by some notches, for good measure. Prefer the deprecation message from maint over that from master. * m4/python.m4: Bump serial number. * tests/Makefile.am (canon6.log): Depend on `libtool-macros.log'. (canon7.log): Likewise. (extradep2.log): Likewise. (ar-lib4.log): Likewise. (ar-lib6a.log): Likewise. (ar-lib6b.log): Likewise. (vartypo2): Likewise. (posixsubst-ltlibraries.log): Likewise. * tests/defs: Adjust the path of the included `get.sh' scripts generated by `libtool-macros.test' and `gettext-macros.test'. * aclocal.m4: Delete, it's auto-generated now. * configure: Likewise. * Makefile.in: Likewise. * doc/Makefile.in: Likewise. * m4/Makefile.in: Likewise. * tests/Makefile.in: Likewise. * lib/Makefile.in: Likewise. * lib/am/Makefile.in: Likewise. * lib/Automake/Makefile.in: Likewise. * lib/Automake/tests/Makefile.in: Likewise. * .gitignore: Correctly ignored the autogenerated files that are not committed anymore.
* | \ Merge branch 'master' into testsuite-workStefano Lattarini2011-12-101-3/+5
|\ \ \ | |/ /
| * | Merge branch 'maint'Stefano Lattarini2011-11-101-3/+5
| |\ \ | | |/ | | | | | | | | | | | | | | | | | | | | | * maint: tests: avoid a spurious failure of 'ltinit.test' MinGW tests: testsuite is now safe to run with dmake in parallel mode tests: fix spurious failures w.r.t. parallel make and colorization tests: fix spurious failure in 'distcheck-override-infodir.test' tests: avoid another failure of 'uninstall-fail.test' on Solaris
* | | Merge branch 'master' into testsuite-workStefano Lattarini2011-11-051-0/+48
|\ \ \ | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commits merged from master: - test defs: new functions to analyze configure help screen - warnings: fix buglets for portability warnings - tests: extend tests on 'extra-portability' warning category Extra edits to the testsuite: * tests/extra-portability.test: Use `$am_original_AUTOMAKE' instead of the obsolete `$original_AUTOMAKE'. * tests/extra-portability3.test: Likewise. * tests/list-of-tests.mk: Update.
| * | test defs: new functions to analyze configure help screenStefano Lattarini2011-11-041-0/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs (extract_configure_help, grep_configure_help): New functions. * tests/maintmode-configure-msg.test: Use them, reducing code duplication and test brittleness. * tests/help-depend.test: Likewise. * tests/help-depend2.test: Likewise. * tests/help-dmalloc.test: Likewise. * tests/help-lispdir.test: Likewise. * tests/help-multilib.test: Likewise. * tests/help-python.test: Likewise. * tests/help-regex.test: Likewise. * tests/help-silent.test: Likewise. * tests/help-upc.test: Likewise. * tests/help-init.test: Make grepping of configure help screen slightly stricter. * tests/self-check-configure-help.test: New self test. * tests/Makefile.am (TESTS): Add it. From a report by Jim Meyering.
* | | Merge branch 'tw-merge-master' into testsuite-workStefano Lattarini2011-11-031-0/+8
|\ \ \
| * \ \ Merge branch 'master' into testsuite-workStefano Lattarini2011-11-031-0/+8
| |\ \ \ | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commits merged from master: - tests: various minor tweakings, mostly related to AM_PROG_AR - maint-mode: fix botched configure messages - fix: regenerate Makefiles - tests: few improvements to some `ar-lib' related tests - info: allow user to inhibit creation/update of '${infodir}/dir' - warnings: new 'extra-portability' category, for AM_PROG_AR - ar-lib: new 'AM_PROG_AR' macro, triggering the 'ar-lib' script Extra testsuite adjustments: * tests/instspc.tap: Adjust to new portability requirements due to the new AM_PROG_AR macro. * tests/lex-lib.test: Likewise. * tests/extra-portability2.test: Use `$am_original_AUTOMAKE' instead of the obsolete `$original_AUTOMAKE'. * tests/list-of-tests.mk: Update.
| | * | Merge branch 'msvc' into masterStefano Lattarini2011-11-031-0/+8
| | |\ \ | | | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commits merged from msvc: - tests: few improvements to some `ar-lib' related tests - warnings: new 'extra-portability' category, for AM_PROG_AR - ar-lib: new 'AM_PROG_AR' macro, triggering the 'ar-lib' script Extra changes required by the testsuite: * tests/instspc-tests.sh: Adjust to new portability requirements due to the new AM_PROG_AR macro. * tests/libobj-basic.test: Likewise. * tests/libobj2.test: Likewise. * tests/libobj15a.test: Likewise. * tests/libobj15b.test: Likewise. * tests/libobj15c.test: Likewise. * tests/libobj16a.test: Likewise. * tests/libobj16b.test: Likewise. * tests/libobj17.test: Likewise. * tests/libobj18.test: Likewise. * tests/libobj19.test: Likewise. * tests/libobj20a.test: Likewise. * tests/libobj20b.test: Likewise. * tests/libobj20c.test: Likewise. * tests/canon6.test: Likewise. * tests/canon6.test: Likewise. * tests/canon7.tests: Likewise. * tests/extra9.test: Likewise. * tests/extradep.test: Likewise. * tests/extradep2.test: Likewise. * tests/posixsubst-ldadd.test: Likewise. * tests/posixsubst-libraries.test: Likewise. * tests/posixsubst-ltlibraries.test: Likewise. * tests/python-virtualenv.test: Likewise. * tests/vartypos.test: Likewise. * tests/vartypo2.test: Likewise. * tests/suffix.test: Update to take into account previous master-only changes. * tests/suffix2.test: Likewise. * tests/libobj7.test: Call automake with the `--add-missing' option, instead of creating a dummy `ar-lib' file, since this test now also runs "./configure" and "make". * tests/suffix5.test: Copy the real `ar-lib' script file, instead of creating a dummy one, since this test now also runs "./configure" and "make". * tests/extra-portability2.test: Fix this test not to rely on the older, faulty semantics of "strictness specification always reset warning level", which has been fixed in commit v1.11-623-g1609491 (see also automake bug#7669 a.k.a. PR/547). Since we are at it, throw inf few other minor unrelated improvements. * tests/extra-portability.test: Explicitly pass `-Wall' to automake calls, for clarity.
* | | | test defs: new function 'am_keeping_testdirs', for better APIsStefano Lattarini2011-10-251-6/+11
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change is meant to make it easier for TAP-based test scripts to remove files left over by completed (sub)tests, while still honoring the user-provided `$keep_testdirs' setting. * tests/defs (am_keeping_testdirs): New function, telling whether we should keep the test directories around, even in case of success (by default, we don't). (exit trap): Use it. * tests/depmod.tap: Likewise. * tests/instspc.tap: Likewise. Also, fix some pre-existing "copy & paste" bugs, and a typo in comments.
* | | Merge branch 'master' into testsuite-workStefano Lattarini2011-09-181-0/+5
|\ \ \ | |/ /
| * | Merge branch 'java-work'Stefano Lattarini2011-09-181-1/+6
| |\ \ | | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * java-work: fix: remove duplicated ChangeLog entries java: allow both JAVA and nobase_JAVA in the same Makefile.am java: allow both dist_JAVA and nodist_JAVA in the same Makefile.am fix: regenerate tests/Makefile.in tests defs: use `skip_' for skipping java-related tests java tests: use $PATH_SEPARATOR where appropriate java coverage: test JAVACFLAGS and AM_JAVACFLAGS java coverage: test rebuild rules for java java coverage: try to build and run a java program test defs: new requirement for the default java interpreter java tests: tweak and make stricter a couple of tests java coverage: add test on uninstall with JAVA primary java tests: require java compiler more properly java: allow both JAVA and nobase_JAVA in the same Makefile.am java: allow both dist_JAVA and nodist_JAVA in the same Makefile.am
* | | * tests/defs [$sh_errexit_works = yes]: Output 'SIGINT' for signal 2.Peter Rosin2011-09-131-1/+1
| | | | | | | | | | | | Signed-off-by: Peter Rosin <peda@lysator.liu.se>
* | | tests: also trap SIGQUIT; other fixesStefano Lattarini2011-09-131-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some shells seems to just ignore SIGQUIT under some circumstances, even when it is not blocked; however, if the signal it trapped, the trap gets correctly executed. * tests/defs [$sh_errexit_works = yes]: Also trap SIGQUIT. Add extensive explanation of why we do so. * tests/self-check-cleanup.tap: Improve and fix to ensure that an earlier failure does not prevent the rest of the test to execute. Problem revealed by a failure in `self-check-cleanup.tap'.
* | | test defs: work around weird ksh behaviour w.r.t. signal handlingStefano Lattarini2011-09-131-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs (is_blocked_signal): Use perl to determine whether a signal is trapped, since trying to do it portably within the shell means opening a nasty can of worms. For more information and background, see: <http://lists.gnu.org/archive/html/bug-autoconf/2011-09/msg00004.html> <http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2009-February/004121.html> <http://www.cons.org/cracauer/sigint.html>
* | | Merge branch 'prove' into testsuite-workStefano Lattarini2011-09-101-4/+12
|\ \ \ | | | | | | | | | | | | | | | | | | | | * prove: prove: avoid weird signal-related bug tests: allow use of `prove' to run automake testsuite
| * | | prove: avoid weird signal-related bugStefano Lattarini2011-09-101-4/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs (trap): In the trap code dealing with SIGPIPE signal, ignore further signals of the same kind. See also Test::Harness issue [rt.cpan.org #70855], archived at <https://rt.cpan.org/Ticket/Display.html?id=70855>
* | | | tests defs: more namespace-safenessStefano Lattarini2011-09-101-17/+17
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ($using_tap): Renamed ... ($am_using_tap): ... to this. ($parallel_tests): Renamed ... ($am_parallel_tests): ... to this. ($test_prefer_config_shell): Renamed ... ($am_test_prefer_config_shell): ... to this. ($original_AUTOMAKE): Renamed ... ($am_original_AUTOMAKE): ... to this. ($original_ACLOCAL): Renamed ... ($am_original_ACLOCAL): ... to this. * tests/defs, tests/defs-static.in, tests/gen-testsuite-part, tests/README, and all callers: Adjusted. * tests/Makefile.am (AM_TESTS_ENVIRONMENT): Adjust, and rewrite to make future adjustments easier. * tests/defs (funcs_file_): Renamed ... (am_funcs_file): ... to this. * Makefile.am (sc_tests_obsolete_variables): New maintainer check. (syntax_check_rules): Add it.
* | | test defs: more environment cleanupStefano Lattarini2011-09-091-1/+11
| | | | | | | | | | | | | | | | | | * tests/defs: Also unset the XFAIL_TESTS, and various LOG_DRIVER, LOG_COMPILER variables (and corresponding flags variable).
* | | installcheck: support it in our own testsuiteStefano Lattarini2011-09-081-17/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * NEWS: Update. * tests/defs-static.in ($am_running_installcheck): New variable. Initialize it to a default of "no", and sanity-check its value. ($am_datadir, $am_docdir, $am_automake_acdir, $am_system_acdir, $am_bindir, $am_scriptdir, $am_pkgvdir): New variables, initialized from AC_SUBST'ed values which are adaptively chosen depending on whether the testsuite is running in "simple" mode or in "installcheck" mode. ($MISSING): Remove, it's not really required by the testsuite. ($PATH): Update using `$am_bindir', not `$testbuilddir'. * Makefile.am (installcheck-local): New target. * lib/Automake/tests/Makefile.am (installcheck-local): New target. (AM_TESTS_ENVIRONMENT, AM_PL_LOG_FLAGS): Adaptively choose the `-I' flags for perl based on whether the tests are being run by "make check" or "make installcheck". * tests/Makefile.am (installcheck-local): New target. (defs-static): Generate it at make time, not at configure time, in order to correctly resolve all the indirections in the AC_SUBST'd variables; done with the help of ... (do_subst): ... this new macro. (EXTRA_DIST): Explicitly distribute `defs-static.in'. (CLEANFILES): Remove `defs-static'. * configure.ac (AC_CONFIG_FILES): Remove `tests/defs-static'. (AC_SUBST): Substitute also `automake_acdir', `system_acdir', `scriptdir' and `amdir' to proper values. * lib/Makefile.am (scriptdir): Don't explicitly define anymore. * lib/am/Makefile.am (amdir): Likewise. * m4/acdir/dirlist: Remove, it's not needed anymore. Instead use ... * m4/acdir/.placeholder: ... this file. * m4/Makefile.am (EXTRA_DIST): Update. (automake_acdir, system_acdir): Don't explicitly define anymore. * tests/aclocal-print-acdir.test: Use new `$am_...dir' variables, instead of hard-coding values with $testsrcdir and $testbuilddir, to test more faithfully under "make installcheck". * tests/amhello-binpkg.test: Likewise. * tests/amhello-cflags.test: Likewise. * tests/amhello-cross-compile.test: Likewise. * tests/ansi2knr-no-more.test: Likewise. * tests/autodist-stamp-vti.test: Likewise. * tests/auxdir.test: Likewise. * tests/check2.test: Likewise. * tests/copy.test: Likewise. * tests/multlib.test: Likewise. * tests/obsolete.test: Likewise. * tests/parallel-tests-interrupt.test: Likewise. * tests/repeated-options.test: Likewise. * tests/suffix5.test: Likewise. * tests/tap-doc2.test: Likewise. * tests/txinfo22.test: Likewise. * tests/vtexi3.test: Likewise. * tests/defs: Likewise. Also, print the values of `$using_tap' and `$am_running_installcheck', to simplify debugging, and do some related code movings and tweakings. * tests/aclocal3.test: Update to avoid possible spurious failures when running under "make installcheck". * tests/warnings-unknown.test: Likewise. * tests/location.test: Likewise, and improve debugging input since we are at it. * tests/acloca10.test: Adapt and tweak to work also during a "make installcheck", and not to require a pre-populated `dirlist' file. * tests/acloca18.test: Likewise. * tests/dirlist2.test: Likewise. * tests/dirlist.test: Likewise, and remove an obsolete comment. * tests/README: Update. * tests/dirlist-abspath.test: New test. * tests/list-of-tests.mk: Update.
* | | Merge branch 'test-protocols' into testsuite-workStefano Lattarini2011-09-061-0/+9
|\ \ \ | | | | | | | | | | | | | | | | * test-protocols: tests: fix spurious failures due to ignored signals
| * \ \ Merge branch 'master' into test-protocolsStefano Lattarini2011-09-061-0/+9
| |\ \ \ | | |/ / | | | | | | | | | | | | * master: tests: fix spurious failures due to ignored signals
| | * | tests: fix spurious failures due to ignored signalsStefano Lattarini2011-09-061-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs (is_blocked_signal): New function. * tests/parallel-tests-interrupt.test: Do not try to use/trap signals that are ignored by the parent shell: they will be ignored by all the child processes too. * tests/self-check-exit.test: Likewise. * tests/self-check-cleanup.test: Likewise, and do few minor improvements and extensions since we are at it.
* | | | Merge branch 'test-protocols' into testsuite-workStefano Lattarini2011-08-291-1/+1
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * test-protocols: tests: prefer AM_... variables to user-reserved ones in Makefile tests: some minor fixlets w.r.t. TAP-based tests tests: avoid re-running a test with the 'parallel-tests' option tests: defaults to awk+shell driver in tests on TAP support With minor manual edits to 'tests/Makefile.am' to complete the TAP_LOG_DRIVER_FLAGS -> AM_TAP_LOG_DRIVER_FLAGS rename.
| * | | tests: defaults to awk+shell driver in tests on TAP supportStefano Lattarini2011-08-291-1/+1
| | | | | | | | | | | | | | | | | | | | * tests/defs ($am_tap_implementation): Default to "shell", not to "perl".
* | | | Merge branch 'test-protocols' into testsuite-workStefano Lattarini2011-08-191-0/+24
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * test-protocols: testsuite: fix some redundant autotools calls in tests on TAP * THANKS (Daniel Richard G.): Update e-mail address. java: avoid compilation errors when CLASSPATH is empty parallel-tests: no more spurious successes for FreeBSD make tap: improve granularity for tests on problematic TAP messages tap: correctly handle string "0" in TAP messages tap: a minor simplification in the perl TAP driver parallel-tests: fix help screen for test driver scripts tap: add a dummy TAP driver script implemented in shell + awk testsuite: refactor tests on TAP support in view of future changes fix: regenerate tests/Makefile.in hacking: we now require autoconf 2.68
| * | | tap: add a dummy TAP driver script implemented in shell + awkStefano Lattarini2011-08-171-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The user can also now decide which implementation of the TAP driver to use in the testsuite by defining the `$am_tap_implementation' variable to either "perl" or "shell". Future enhancements will allow the testsuite to automatically run the test scripts on TAP support with both the TAP driver implementations, to improve coverage. * tests/defs (fetch_tap_driver): Honor the `$am_tap_implementation' variable to decide which implementation of the TAP driver to fetch. ($am_tap_implementation): Default to "perl". * tests/tap-common-setup.test: Do not fetch the TAP driver, the code in tap-setup.sh does that already (and respecting runtime overriding of `$am_tap_implementation'). * lib/tap-driver: Renamed ... * lib/tap-driver.pl: ... to this, and ... ($ME): ... adjusted this. * doc/automake.texi: Adjust to the renaming. * tests/Makefile.am (TAP_LOG_DRIVER): Likewise. * tests/tap-doc2.test: Likewise. * lib/tap-driver.sh: New script, still mostly dummy. * lib/Makefile.am (dist_script_DATA): Update, and since we are at it, rewrite it to make it easier to add new entries in the future.
| * | | testsuite: refactor tests on TAP support in view of future changesStefano Lattarini2011-08-171-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs (fetch_tap_driver): New subroutine; it fetches the automake-provided TAP driver from the `lib/' directory into the current directory, and edits its shebang line so that it will be run with the perl interpreter determined at configure time. * tests/tap-setup.sh: Use it. * tests/tap-common-setup.test: There's no need to AC_SUBST `PERL' anymore, nor to use it in the Makefile to run the TAP driver. Also, use the `fetch_tap_driver' function instead of copying the `tap-driver' auxiliary script directly. * tests/tap-bad-prog.tap: Likewise. * tests/tap-diagnostic-custom.test: Likewise. * tests/tap-doc.test: Likewise. * tests/tap-merge-stdout-stderr.test: Likewise. * tests/tap-more.test: Likewise. * tests/tap-more2.test: Likewise. * tests/tap-recheck.test: Likewise. * tests/tap-summary-aux.sh: Likewise. * tests/tap-basic.test: Likewise, and fix a grammaro in comments since we are at it.
* | | | test defs: fix typo in commentsStefano Lattarini2011-08-171-1/+1
| | | | | | | | | | | | | | | | | | | | * tests/defs (get_shell_script): Fix typo in the function description.
* | | | Merge branch 'test-protocols' into testsuite-workStefano Lattarini2011-08-141-0/+1
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * test-protocols: coverage: missing tap plan and non-zero exit status testsuite: fix an unportable use of sed testsuite: avoid spurious failure if rst2html program is missing fix: two recent ChangeLog entries tap: improve diagnosing and reporting of plan mismatches tap: log all TAP stream, even after a "Bail out!" coverage: TAP diagnostics after "Bail out!" aren't reported coverage: TAP errors after a "Bail out!" directive aren't reported testsuite: reorganize tests on TAP plans coverage: more TAP driver consistency w.r.t. 'prove' tap: improve some comments in the TAP driver tap: non-zero exit status after "Bail out!" should not be reported tap: a "plan with SKIP" after test results is an error test harness: avoid possible fork bomb coverage: possible infinite recursion in the test harness coverage: possible infinite recursion in the test harness test defs: yet more environment cleanup
| * | | Merge branch 'master' into test-protocolsStefano Lattarini2011-08-111-0/+1
| |\ \ \ | | |/ / | | | | | | | | | | | | | | | | * master: coverage: possible infinite recursion in the test harness test defs: yet more environment cleanup
| | * | test defs: yet more environment cleanupStefano Lattarini2011-08-101-0/+1
| | | | | | | | | | | | | | | | * tests/defs: Also unset the TEST_SUITE_LOG variable.
* | | | testsuite: avoid confusing SKIP messages for errored testsStefano Lattarini2011-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs [$use_tap] (exit trap): Don't print the test plan in case the client script exited with a non-zero status. This avoids the display of redundant "SKIP" messages for tests with lazy plans that error out before any test is run.
* | | | testsuite: more granular count of test results in our TAP libraryStefano Lattarini2011-08-131-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/tap-functions.sh ($tap_bad_count_): Removed, superseded by ... ($tap_fail_count_, $tap_xpass_count_): ... these new variables, which keep more granular counts. ($tap_pass_count_: New variable. * tests/defs (exit trap): Adjust and simplify accordingly.
* | | | testsuite: improvements to our shell TAP library and APIsStefano Lattarini2011-08-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These improvements impact only the Automake's own testsuite. * tests/tap-functions.sh (plan_): Allow new special arguments "unknown" a.k.a. "later" and "lazy" a.k.a. "now". Perform sanity check the given arguments. Adjust incorrect comments. (lazy_plan_): Removed, superseded by the new version of `plan_'. (not): New function, run a command and inverts its exit status. (ok_, not_ok_, skip_): These are just shorthands for common `result_' usages. (xfail_, xpass_, reset_test_count_): Removed, they are not used, nor does a legitimate use for them seem in sight. (not): New function, to run a command inverting its exit status. (command_not_ok_): Removed, superseded by `command_ok_' + `not'. (command_ok_): Extended to accept TAP directives. * tests/defs (trap): Don't use `late_plan_' anymore, use `plan_ "now"' instead. * tests/ac-output-old.tap: Update to take advantage of the new APIs. * tests/self-check-env-sanitize.tap: Likewise. * tests/suffix-chain.tap: Likewise. * tests/suffix10.tap: Likewise. * tests/suffix11.tap: Likewise. * tests/suffix3.tap: Likewise. * tests/suffix8.tap: Likewise. * tests/tap-bad-prog.tap: Likewise. * tests/self-check-exit.tap: Likewise, plus a small improvement in the name of a variable.
* | | | Merge branch 'test-protocols' into tap-testsuite-workStefano Lattarini2011-08-091-1/+35
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | * test-protocols: tests: don't let a known Solaris make bug poison too many tests gitignore: more use of anchors
| * | | Merge branch 'master' into test-protocolsStefano Lattarini2011-08-091-0/+35
| |\ \ \ | | |/ / | | | | | | | | | | | | | | | | * master: tests: don't let a known Solaris make bug poison too many tests gitignore: more use of anchors
| | * | tests: don't let a known Solaris make bug poison too many testsStefano Lattarini2011-08-091-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See automake bug#7670 and bug#7824. * tests/defs (make_can_chain_suffix_rules): New function, tell whether the make implementation in use can chain suffix rules automatically or not. * tests/suffix3.test: Use it to avoid issuing calls to make that are unportable to make implementations that are not smart enough to chain suffix rules automatically. * tests/suffix8.test: Use it to avoid issuing calls to make that * tests/suffix10.test: Use it to avoid issuing calls to make that * tests/suffix11.test: Use it to avoid issuing calls to make that * tests/suffix-chain.test: New test, exposes the limitation that we have papered over in the tests above.
* | | | test defs: few more sanity checksStefano Lattarini2011-08-091-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | * tests/defs: Catch some more (very unlikely) set-up or internal errors, and diagnose them explicitly. This is intended mostly to be a theoretic-only improvement.
* | | | test defs: really make them "set -e" cleanStefano Lattarini2011-08-091-15/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Motivated by a spurious failure of test `instsh3-w.shtst' on Solaris 10. * tests/defs: Make really "set -e" clean, to avoid spurious failures in case this file is sourced by a test script that has already set the `errexit' flag. To be sure we don't regress, move the `set -e' call much earlier, and make the comment telling about the expected "set -e" cleanliness more loud and outstanding.
* | | | test defs: test directory creation more configurableStefano Lattarini2011-08-091-29/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After this change, the individual tests will be able to tell `./defs' whether or not to create a temporary test directory for them, and if yes, whether or not to pre-populate it. * tests/defs: Only create the temporary test directory if the variable `$am_create_testdir' is not set to "no". In this case, only pre-populate it if that variable is not set to `$empty'. * tests/defs-static.in: Check that `$am_create_testdir' is not exported in the environment. * tests/self-check-env-sanitize.test: Update. * tests/self-check-dir.test: Extend, and make deeper and more robust. * tests/README (Section "Writing test cases" subsection "Do"): Update. * tests/autodist.test: Set `$am_create_testdir' instead of removing by hand files created by ./defs. * tests/tap-doc2.test: Likewise.
* | | | test defs: small cleanups and tweakingsStefano Lattarini2011-08-081-15/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * tests/defs: Where possible, prefer `framework_failure_' or `fatal_' to `Exit 99'. Try to catch more possible hard errors and setup failures. Do not hide expected error messages. Avoid a couple of forks when copying files from `$(top_srcdir)/lib'. ($signal): Remove, no more needed.
* | | | testsuite: improve interface for giving early TAP planStefano Lattarini2011-08-081-5/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since commit `v1.11-976-g4740f56', an early TAP plan can also be specified after input lines have already been seen, as long as no one of them is a TAP result line. We can now take advantage of this in our own testsuite, allowing the TAP-based scripts to specify their earlier plan after having included `./defs'. In particular, this offers the benefit of allowing those scripts to issue a "skip all" plan (e.g., "1..0 # SKIP reason ...") after the inclusion of `./defs', which wasn't possible with the older interface (introduced in commit `v1.11-1114-g0286308'). * tests/defs: Don't try to call `plan_' if `$planned' is defined, the calling scripts can do that directly now. * tests/ac-output-old.tap: Use `plan_' directly instead of setting `$planned' before including `./defs'. * tests/automake-cmdline.tap: Likewise. * tests/depmod.tap: Likewise. * tests/instspc.tap: Likewise. * tests/tap-bad-prog.tap: Likewise. * tests/add-missing.tap: Don't set `$planned'.
* | | | test defs: fix skip directives to work also with TAP-based testsStefano Lattarini2011-08-081-43/+43
| | | | | | | | | | | | | | | | | | | | * test/defs: Use `skip_all_', not `skip_', to issue a skip of the whole calling test script.
* | | | Merge branch 'test-protocols' into tap-testsuite-workStefano Lattarini2011-08-081-1/+2
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | * test-protocols: test defs: more environment cleanup test harness: more minor VPATH-rewrite issues test harness: "make -n check" works correctly with Solaris make
| * | | Merge branch 'master' into test-protocolsStefano Lattarini2011-08-081-1/+2
| |\ \ \ | | |/ / | | | | | | | | | | | | * master: test defs: more environment cleanup
| | * | Merge branch 'maint'Stefano Lattarini2011-08-081-1/+2
| | |\ \ | | | |/ | | | | | | | | | | | | * maint: test defs: more environment cleanup
| * | | Merge branch 'master' into test-protocolsStefano Lattarini2011-08-031-1/+1
| |\ \ \ | | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * master: maintcheck: fix maintainer-check failures, both real and spurious docs: how to use '-I' option in AM_CPPFLAGS for best VPATH support tests: relax a test on amhello examples to cater to Solaris tar test defs: function 'is_newest' now works also with directories docs: add references between the 2 sections on java support Sync auxiliary files from upstream. fix typo in recent ChangeLog entry docs, tests: synchronize examples on silent-rules from config.site tests: fix weakness in 'tests-environment-backcompat.test' tests: portability fixes in tests on amhello examples remake: add test ensuring that slower remakes don't hang remake: fix outdated comment in configure.am docs, tests: synchronize examples from docs to tests coverage: new test on parallel-tests TESTS runtime overriding cosmetics: fix typos in recent ChangeLog entries
* | | | test defs: get name of TAP-using tests correctlyStefano Lattarini2011-08-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change fixes the names of the temporary directories used by the TAP-based tests (e.g., test `foo.tap' now uses `foo.dir' rather than `foo.tap.dir'); more importantly, this make the variable `$destdir' defined in `tests/defs' usable also in the TAP tests. ($me): Also strip the any `.tap' suffix from the name of the test.
* | | | testsuite: TAP tests properly decide when to remove tempdirsStefano Lattarini2011-08-071-26/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, the TAP tests in the Automake testsuite were removing the temporary test directory even when they failed or were skipped, thus making debugging more difficult. * tests/tap-functions.sh (incr_tap_count): Removed, superseded by ... (incr_): ... this function, which can increment the value of any variable passed to it. (result_): Updated to use `incr_' instead of the now-removed `incr_tap_count_'. Keep count of failures, xfailures, xpasses, and skips, using ... ($tap_skip_count_, $tap_bad_count, _$tap_xfail_count_): ... these new variables. * tests/defs (trap): Try to use their values to decide whether the temporary directory being used by the test script should be removed or not. Other code reorganizations. And move the code for the removal of the temporary directory out to ... (rm_rf_): ... this new subroutine. (Main code): Use that instead of duplicating the code.