| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The use 'run_make' with the -E, -O and -M option, it is more
idiomatic now. Also, this way, centralized fixes and improvements
done in 'run_make' will automatically propagate through most of
the testsuite.
* syntax-checks.mk (sc_tests_no_run_make_redirect): Also check against
'$MAKE' invocations that uses output redirections (and not only against
'run_make' invocation that do so).
* Several tests: Adjust (and few minor cosmetic changes as well, while
at it).
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Instead, properly use the -E, -O, or -M options of run_make.
The occurrences of the 'run_make' invocations using output
redirection has been found by the recently-introduced maintainer
check 'sc_tests_no_run_make_redirect'.
* t/ax/tap-summary-aux.sh: Adjust.
* t/ax/testsuite-summary-checks.sh: Likewise.
* Several other tests: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We now do so with a new wrapper shell function 'run_make()', which
relies on the use of AM_MAKEFLAGS, avoiding the use of the '-e' make
option. The use of that option (that causes the environment variables
to take precedence over the macro definitions in the Makefile) has
proved to be quite brittle in the past, causing annoying and sometimes
problematic spurious failures. This has worsened in some still
unpublished work going on in development branches. It's time to solve
the issue once and for all.
Note that we do not convert all uses of $MAKE in the testsuite right
away; we might do so in follow-up changes, with leisure, to avoid a
"patch bomb" effect (this commit is already too much of a bomb itself).
What we do in this commit is to get rid of all "$MAKE -e" invocations.
We admit that the implementation and feature-set of 'run_make()' are
far from perfect, but good enough for our current purposes. We'll
improve 'run_make()' if and when the need arises.
* syntax-checks.mk (sc_tests_no_make_e): New syntax check, guard against
the use of "$MAKE -e".
(syntax_check_rules): Add it.
(sc_tests_overriding_macros_on_cmdline): Adjust.
(lint): New, alias for 'maintainer-check', for lazy typists. Idea
backported from the 'maint' branch (Automake 1.13a).
* t/ax/am-test-lib.sh (run_make): New function. Run $MAKE with the
given command-line arguments, handling command-line override of variable
definitions in a smart way (using AM_MAKEFLAGS if a non-GNU make
implementation is detected to be in use).
(useless_vpath_rebuild): Adjust to use 'run_make', to avoid a spurious
maintainer check failure.
(yl_distcheck): Use 'run_make' rather than bare '$MAKE'.
(single_quote, append_single_quoted, is_valid_varname): New auxiliary
function, used, directly or indirectly, by it.
* Many tests: Adjust to avoid the use of "$MAKE -e", and prefer the
use of 'run_make' in few other contexts as well, where it makes sense.
Other minor fixlets while at it.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In some shells (e.g., Solaris 10 /bin/ksh, or NetBSD 5.1 /bin/sh),
"unset VAR" returns a non-zero exit status in case the VAR variable
is already unset. This doesn't interact well with our usage of
"set -e" in the testsuite. So far, we've avoided spurious failures
by either explicitly ignoring the exit status from unset:
unset VAR || :
or explicitly ensuring that a variable is set, before trying to
unset it:
VAR=; unset VAR
But we can do better, by aliasing the 'unset' command to a custom
function that will take care of these details for us. This will
avoid us annoying spurious failures in the future, failures that
have already bitten us too much times. For an example, refer to
commit 'v1.12.2-88-g5b1dae5' of 2012-08-05 (tests: avoid tons of
spurious failures on NetBSD).
* t/ax/test-lib.sh (_am_unset): New function.
(unset): New alias to it.
(_am_exit): Adjust comments.
* t/ax/am-test-lib.sh: No need to temporary disable the 'errexit'
shell flag when unsetting variables that are potentially already
unset.
(am_process_requirements): Adjust to remove a now-useless
workaround related to unset.
* t/aclocal-macrodir.tap: Likewise.
* t/aclocal-macrodirs.tap: Likewise.
* t/auxdir-autodetect.sh: Likewise.
* t/ax/am-test-lib.sh: Likewise.
* t/ax/test-lib.sh: Likewise.
* t/check-tests-in-builddir.sh: Likewise.
* t/dist-formats.tap: Likewise.
* t/distcheck-configure-flags-am.sh: Likewise.
* t/distcheck-configure-flags.sh: Likewise.
* t/java-empty-classpath.sh: Likewise.
* t/javaflags.sh: Likewise.
* t/lflags.sh: Likewise.
* t/lflags2.sh: Likewise.
* t/lisp-flags.sh: Likewise.
* t/lisp6.sh: Likewise.
* t/missing-auxfile-stops-makefiles-creation.sh: Likewise.
* t/parallel-am.sh: Likewise.
* t/parallel-am2.sh: Likewise.
* t/parallel-am3.sh: Likewise.
* t/parallel-tests-log-override-recheck.sh: Likewise.
* t/pkg-config-macros.sh: Likewise.
* t/python-missing.sh: Likewise.
* t/python-too-old.sh: Likewise.
* t/python11.sh: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-report.sh: Likewise.
* t/self-check-seq.tap: Likewise.
* t/silent-configsite.sh: Likewise.
* t/suffix6c.sh: Likewise.
* t/tar-override.sh: Likewise.
* t/tests-environment-and-log-compiler.sh: Likewise.
* t/vala-configure.sh: Likewise.
* t/werror3.sh: Likewise.
* t/yflags-cmdline-override.sh: Likewise.
* t/yflags.sh: Likewise.
* t/yflags2.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It gives the impression that they are directly runnable, as with
"./t/foo.sh", but it has been a while since that was the case. Today,
tests are runnable only through "make check" or "./runtest".
This change is for the 'micro' branch (automake 1.13.2a). It will
soon be followed by similar patches for the 'maint' branch (automake
1.13a) and the 'master' branch (automake 1.99a).
* t/*.sh, t/*.tap: Remove executable bit.
* maint.mk (sc_tests_executable): Remove.
(syntax_check_rules): Adjust.
* gen-testsuite-part: Set permissions of generated tests to
'444' (-r--r--r--), rather than 555 (-r-xr-xr-x).
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
| |
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a follow-up to today's commit v1.12.4-22-g0610fc8,
"tests: prepare to move ./defs to t/ax/test-init.sh"
* All tests: To run the common setup, use the command:
. test-init.sh
instead of the older, "historical" one:
. ./defs || exit 1
Note that the "|| exit 1" wasn't really useful, since the 'errexit'
shell flag is in effect in both './defs' and 'test-init.sh', and all
the known shells that are good enough to run the automake testsuite
do automatically exit with error when a sourced file cannot be found
(at least, they do so in non-interactive mode, which is the only
mode that concerns us in the testsuite).
* t/ax/tap-summary-aux.sh, t/ax/testsuite-summary-checks.sh: Likewise.
* gen-testsuite-part: Do the same in the generated tests.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
| |
* t/parallel-tests-log-override-recheck.sh: Here, add a proper $sleep
before calling "make recheck". This should ideally have been done
in the earlier commit 'v1.12.1-100-g19d84bc', but it somehow slipped
through the cracks.
* t/test-metadata-recheck.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It turns out that, with NetBSD 5.1 make and FreeBSD 9 make, running
"make recheck" two times in a row quickly fails to correctly re-run
the failed tests in the second run.
That issue has been introduced likely introduced in commit
'v1.12.1-95-gd5443e4' of 20102-07-01, "parallel-tests: reimplement
fix for bug#11791".
Anyway, the use case that has been broken is not realistic (who is
going to run "make recheck" two times in one second, without modifying
any of the tests or the tested programs in the meantime?), so we believe
the best fix is to simply work around the issue in the affected test
cases, rather than risking to slow down or uglify the 'recheck' rule.
* t/parallel-tests9.sh: Enhance a little.
* t/tap-recheck.sh: Adjust adding proper '$sleep' calls were required.
* t/parallel-tests-log-override-recheck.sh: Likewise.
* t/test-driver-custom-multitest-recheck.sh: Likewise.
* t/test-driver-custom-multitest-recheck2.sh: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now that we can assume our tests are run by a decent POSIX shell, we
can simplify our workaround aimed at having the exit status propagated
correctly to the code in the exit trap. Unfortunately, we cannot
dispense with such a workaround altogether, because it's still required
by some shells we need to support (at least Solaris 10 /bin/ksh and
/usr/xpg4/bin/sh).
For more information about the need of that workaround, see the entry
about 'trap' in the section "Limitations of Shell Builtins" in the
Autoconf manual:
<http://www.gnu.org/software/autoconf/manual/autoconf.html#trap>
The new workaround has been tested successfully with the following
shells:
- Bash 4.1
- Bash 3.2
- Bash 3.0
- Bash 2.05b
- dash 0.5.5.1
- dash 0.5.2
- AT&T Ksh 93u (from official Debian package)
- MirBSD Korn Shell 40.2 (from official Debian package)
- Solaris 9, 10 and 11 /bin/ksh
- Solaris 9, 10 and 11 /usr/xpg4/bin/sh
- NetBSD 5.1 /bin/sh
- NetBSD 5.1 /bin/ksh
* t/ax/test-init.sh (Exit): Rename ...
(_am_exit): ... like this.
(exit): New alias for '_am_exit'. We cannot simply redefine 'exit'
as a shell function, because some shells (dash 0.5.5.1, Solaris 10
/bin/ksh and /usr/xpg4/bin/sh) do not allow it.
(_am_exit, trap): Add extra escaping for 'exit' calls, to ensure we
really invoke the 'exit' builtin and not our alias with the same
name.
* configure.ac: Check that the shell selected to run our testsuite
supports aliases named like shell builtins.
* t/REAMDE: Adjust.
* All tests: Adjust, by simply using 'exit' instead of 'Exit'.
* t/self-check-explicit-skips.sh: Adjust: the first usage of 'exit'
after it has been redefined as an alias must be on a new line w.r.t.
that where the alias is defined, in order for the redefinition to be
honored.
* syntax-checks.mk (sc_tests_Exit_not_exit): Delete.
(sc_tests_exit_not_Exit): New.
(syntax_check_rules): Adjust.
(sc_tests_automake_fails): Simplify the recipe a little.
* Several tests: Remove now useless spurious quoting once required
to placate the 'sc_tests_Exit_not_exit' maintainer check.
* gen-testsuite-part: Likewise. Also, avoid uses of 'Exit' in the
generated scripts.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This will help our transition from 'serial-tests' to 'parallel-tests'
as the default test suite driver enabled by a TESTS assignment in the
input Makefile.am. Note that that change of default will only take
place in master, though.
* defs: Pass the 'parallel-tests' option to the AM_INIT_AUTOMAKE
invocation in the created 'configure.ac' stub, unless the variable
'am_serial_tests' is set to "yes". Don't pay attention anymore to
the 'am_parallel_tests' variable, that's obsolete now.
* defs-static.in: Warn if the 'am_serial_tests' variable is set in the
environment; conversely, don't warn anymore about 'am_parallel_tests'
being set in the environment.
* Makefile.am (AM_TESTS_ENVIRONMENT): Nullify the 'am_serial_tests'
variable instead of the now-obsolete 'am_parallel_tests' one.
* syntax-checks.mk (sc_tests_obsolete_variables): Also warn against
uses of 'am_parallel_tests', which is now deprecated in favor of
'am_serial_tests'. Similarly, if a use of 'parallel_tests' is seen,
suggest using 'am_serial_tests' instead, not 'am_parallel_tests'.
* gen-testsuite-part: Now that we use the 'parallel-tests' by default
in our tests, we need to completely change the logic and semantics of
generation of sibling tests for those tests that check the Automake
generated testsuite harness itself. Do that, and give a complete
explanation of the new logic and semantics in the relevant comments.
* t/README: Update.
* Lots of test cases: Adjust.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
|
|
When we (soon) convert the Automake testsuite to a non-recursive
make setup, we'll have to fix the entries of $(TESTS) to be
prepended with the subdirectory they are in; this will increase
the length of $(TESTS), and thus increase the possibility of
exceeding the command-line length limits on some systems (most
notably, MinGW/MSYS). See automake bug#7868 for more information.
Thus we rename the 'tests/' subdirectory to 't/', and each 'x.test'
script in there to 'x.sh'; this way, the $(TESTS) entry 'foo.test'
will become 't/foo.sh', which have the same number of characters.
* tests/: Rename ...
* t/: ... to this.
* t/*.test: Rename ...
* t/*.sh: ... to this.
* t/.gitignore: Removed as obsolete.
* t/defs: Adjust.
* t/gen-testsuite-part: Likewise.
* t/list-of-tests.mk: Likewise.
* t/ccnoco.sh: Likewise.
* t/ccnoco3.sh: Likewise.
* t/self-check-cleanup.tap: Likewise.
* t/self-check-dir.tap: Likewise.
* t/self-check-me.tap: Likewise.
* t/self-check-reexec.tap: Likewise.
* README: Likewise.
* bootstrap: Likewise
* configure.ac: Likewise.
* Makefile.am: Likewise.
* .gitignore: Likewise.
* syntax-check.mk: Likewise.
Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
|
