summaryrefslogtreecommitdiff
path: root/testsuite/driver/runtests.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Testsuite: report and error out on unfound testsThomas Miedema2015-10-291-1/+7
| | | | | | | | | | | | | | | | | | | Users are sometimes confused why their test doesn't run. It is usually because of a misspelled testname, for example using 'TEST=1234' instead of 'TEST=T1234'. After this patch it is hopefully more clear what the problem is, showing: ERROR: tests not found: ['1234'] Instead of: 0 total tests, which gave rise to 0 test cases, of which 0 were skipped Reviewed by: austin, bgamari Differential Revision: https://phabricator.haskell.org/D1388
* Testsuite Windows: don't use forward slashes in topdir pathThomas Miedema2015-10-201-1/+4
| | | | | | | | | | | | | | Changing backwards slashes to forward slashes apparently confuses msys2/mingw magic path handling. I don't quite understand why, but this fixes it. Test Plan: on Windows, make sure PATH does not contain 'inplace/mingw/bin' (let the testsuite driver add it), then run: make TEST='ghcilink003 ghcilink006'. Before this patch, it would fail. Reviewed by: Phyx, bgamari, austin Differential Revision: https://phabricator.haskell.org/D1343
* Make Windows linker more robust to unknown sectionsTamar Christina2015-10-031-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Windows Linker has 3 main parts that this patch changes. 1) Identification and classification of sections 2) Adding of symbols to the symbols tables 3) Reallocation of sections 1. Previously section identification used to be done on a whitelisted basis. It was also exclusively being done based on the names of the sections. This meant that there was a bit of a cat and mouse game between `GCC` and `GHC`. Every time `GCC` added new sections there was a good chance `GHC` would break. Luckily this hasn't happened much in the past because the `GCC` versions `GHC` used were largely unchanged. The new code instead treats all new section as `CODE` or `DATA` sections, and changes the classifications based on the `Characteristics` flag in the PE header. By doing so we no longer have the fragility of changing section names. The one exception to this is the `.ctors` section, which has no differentiating flag in the PE header, but we know we need to treat it as initialization data. The check to see if the sections are aligned by `4` has been removed. The reason is that debug sections often time are `1 aligned` but do have relocation symbols. In order to support relocations of `.debug` sections this check needs to be gone. Crucially this assumption doesn't seem to be in the rest of the code. We only check if there are at least 4 bytes to realign further down the road. 2. The second loop is iterating of all the symbols in the file and trying to add them to the symbols table. Because the classification of the sections we did previously are (currently) not available in this phase we still have to exclude the sections by hand. If they don't we will load in symbols from sections we've explicitly ignored the in # 1. This whole part should rewritten to avoid this. But didn't want to do it in this commit. 3. Finally the sections are relocated. But for some reason the PE files contain a Linux relocation constant in them `0x0011` This constant as far as I can tell does not come from GHC (or I couldn't find where it's being set). I believe this is probably a bug in GAS. But because the constant is in the output we have to handle it. I am thus mapping it to the constant I think it should be `0x0003`. Finally, static linking *should* work, but won't. At least not if you want to statically link `libgcc` with exceptions support. Doing so would require you to link `libgcc` and `libstd++` but also `libmingwex`. The problem is that `libmingwex` also defines a lot of symbols that the RTS automatically injects into the symbol table. Presumably because they're symbols that it needs. like `coshf`. The these symbols are not in a section that is declared with weak symbols support. So if we ever want to get this working, we should either a) Ask mingw to declare the section as such, or b) treat all a imported symbols as being weak. Though this doesn't seem like it's a good idea.. Test Plan: Running ./validate for both x86 and x86_64 Also running the specific test case for #10672 make TESTS="T10672_x86 T10672_x64" Reviewed By: ezyang, thomie, austin Differential Revision: https://phabricator.haskell.org/D1244 GHC Trac Issues: #9907, #10672, #10563
* Testsuite: mention the existence of ticket #10510Thomas Miedema2015-06-111-1/+1
| | | | [skip ci]
* Make validate more quietThomas Miedema2015-06-041-6/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * By default use V=0, and call the testsuite with VERBOSE=2, which we did before only with validate --quiet. This disables printing the test commands it runs. * When --quiet is used, call the testsuite with VERBOSE=1. This disables printing the '====> Scanning' lines, and doesn't print which test is being run. So it only prints something when a test accidentally prints to stdout or when it fails. Don't set this option on Travis, as Travis will cancel a build if it doesn't see any output for more than 10 minutes. * When --quiet is used, set the new test option NO_PRINT_SUMMARY, which skips printing the test summary. Only the list of unexpected failures is printed, if there are any. Note that the full summary can still be found in testsuite_summary.txt * When --quiet is used, don't pass the `-v` flag to `ghc-pkg check` * When --quiet is used, don't print the Oops! header. It shoud be clear from the list of failing tests that something is wrong. This is all done to get the most out of 30 lines of logfile. These changes can be disabled later by simply not passing the --quiet flag to validate. Differential Revision: https://phabricator.haskell.org/D942
* Testdriver: do not interfer with MinGW path magic (#10449)Thomas Miedema2015-05-281-2/+7
| | | | | | | | | | | | | | | | | | This should fix the testsuite driver on Windows using the MinGW tools with a native build of Python. MinGW automagically converts MinGW-style paths (e.g. '/c/programs/ghc/bin/ghc') into ordinary Windows paths (e.g. 'C:/programs/ghc/bin/ghc') when a native Windows program is invoked. But it doesn't do so when those paths are wrapped with a pair of escaped double quotes. The fix is to not call `eval` on the paths in Python, which let's us use one less pair of quotes, and makes MinGW happy. Reviewers: Rufflewind, austin Differential Revision: https://phabricator.haskell.org/D911
* Testdriver: don't use os.popen in config/ghcPhil Ruffwind2015-05-281-4/+1
| | | | | | | | | | | | | | | Rewrite config/ghc to use getStdout (which use subprocess.Popen) instead of os.popen, which is deprecated; this also avoids the use of shell Also: * Move getStdout to driver/testutil.py so both config/ghc and driver/runtests.py can use it * Remove support for Python below 2.4, which doesn't have subprocess Reviewed By: thomie Differential Revision: https://phabricator.haskell.org/D908
* Python 3 support, second attempt (Trac #9184)Krzysztof Gogolewski2014-10-191-28/+39
| | | | | | | | | | | | | | | | | | | | | | | | Summary: This is a fixup of https://phabricator.haskell.org/D233 The only difference is in findTFiles (first commit), which previously broke Windows runner; now I translated literally instead attempting to improve it, and checked it works. Test Plan: I used validate under 2,3 on Linux and under 2 on msys2. On Windows I've seen a large number of failures, but they don't seem to be connected with the patch. Reviewers: hvr, simonmar, thomie, austin Reviewed By: austin Subscribers: thomie, carter, ezyang, simonmar Differential Revision: https://phabricator.haskell.org/D310 GHC Trac Issues: #9184
* Extend windows detection in testsuite to recognize MSYS targetGintautas Miliauskas2014-10-111-2/+2
| | | | | | | | | | | | | | Currently, the detection recognizes the following `uname -s` strings: - `CYGWIN_NT-6.3` - `MINGW32_NT-6.3` - `MINGW64_NT_6.3` However, MSYS2 provides an additional target, in which case `uname -s` returns a string such as `MSYS_NT-6.3`. In all these cases, the system ought to be recognized as being a `windows` os by the testsuite runner. See also #9604
* Fallback to `ctypes.cdll` if `ctypes.windll` unavailableGintautas Miliauskas2014-10-111-6/+4
| | | | | | | | | On Windows, we may be using a native build of Python or a mingw/msys build. The former exports `ctypes.windll`, the latter exports `cdll`. Previously the code threw an exception when using the msys Python because it expected `windll` to always be available on Windows. Differential Revision: https://phabricator.haskell.org/D308
* Revert "Basic Python 3 support for testsuite driver (Trac #9184)"Krzysztof Gogolewski2014-10-031-39/+28
| | | | | | This reverts commit 084d241b316bfa12e41fc34cae993ca276bf0730. This is a possible culprit of Windows breakage reported at ghc-devs.
* Basic Python 3 support for testsuite driver (Trac #9184)Krzysztof Gogolewski2014-10-011-28/+39
| | | | | | | | | | | | | | | | | | | | | | | | Summary: Most of the changes is adaptation of old Python 2 only code. My priority was not breaking Python 2, and so I avoided bigger changes to the driver. In particular, under Python 3 the output is a str and buffering cannot be disabled. To test, define PYTHON=python3 in testsuite/mk/boilerplate.mk. Thanks to aspidites <emarshall85@gmail.com> who provided the initial patch. Test Plan: validate under 2 and 3 Reviewers: hvr, simonmar, thomie, austin Reviewed By: thomie, austin Subscribers: aspidites, thomie, simonmar, ezyang, carter Differential Revision: https://phabricator.haskell.org/D233 GHC Trac Issues: #9184
* Revert "Fix a couple test failures encountered when building on Windows"Austin Seipp2014-09-011-4/+1
| | | | | This reverts commit 9711f78f790d10d914e08851544c6fc96f9a030a, as it's causing build phailures in phabricator.
* Fix a couple test failures encountered when building on WindowsAustin Seipp2014-09-011-1/+4
| | | | | | | | | | | | | | Summary: * Adjusts performance tests * Change ghcpkg05.stderr-mingw32 to match ghcpkg05.stderr Test Plan: Ran 'sh validate' and observed fewer test failures afterwards Reviewers: austin Reviewed By: austin Differential Revision: https://phabricator.haskell.org/D191
* New testsuite verbosity level 4Joachim Breitner2014-07-141-2/+2
| | | | | which makes it print performance numbers even when the test succeeds (good for historic analysis)
* Detect msys2 via uname (different to msys)Simon Peyton Jones2013-11-121-1/+4
| | | | | | uname -s msys gives "MINGW32" msys2 gives "MINGW_NT-6.2"
* Flag to test suite: SKIP_PERF_TESTSJoachim Breitner2013-10-051-0/+4
| | | | | | More often than not the output of the performance tests is in the way, rather than helping. This allows the use of `make SKIP_PERF_TESTS=YES` to skip these tests. Fixes #8413
* Test driver: Implement different verbositiesJoachim Breitner2013-09-201-1/+9
| | | | | | | | | Select verbosity with "make VERBOSE=n". Options so far: n=0: No per-test output n=1: Only failing test results n=2: As above, plus progress information (names of all tests) n=3: As aobve, plus commands called. Default currently is n=3, although n=2 might be a nicer default.
* Test driver: Print total time spent at the endJoachim Breitner2013-09-201-5/+2
|
* Tweak the brokens list to include the directory the test is inIan Lynagh2013-02-111-1/+1
|
* Print a warning if we get framework failures when listing brokensIan Lynagh2013-02-111-0/+6
|
* Add 'make list_brokens'Ian Lynagh2013-02-111-20/+25
| | | | | | Gives a list of tickets that the testsuite thinks are broken, and what bug it thinks is the reason. This can then be pasted into trac and 'previewed', which will show any closed tickets with strikeout.
* Pass the test name to the test optionsIan Lynagh2013-02-071-3/+0
| | | | | | | | This allows them to give framework failures. I also had to change how setTestOpts works. Now, rather than applying the options to the directory's "default options", it just stores the options to be applied for each test (i.e. once we know the test name).
* When the testsuite is ^Ced, print the summary anywayIan Lynagh2013-02-021-0/+4
|
* Revert "fix runtests to set LD_LIBRARY_PATH environment variable."Ian Lynagh2013-01-251-26/+22
| | | | | | This reverts commit d262089127c54bfe81963628ac70a309f8133492. We shouldn't need to do this on Linux.
* fix runtests to set LD_LIBRARY_PATH environment variable.David Terei2013-01-241-22/+26
| | | | Patch from Karel Gardas <karel.gardas@centrum.cz>.
* Run all the 'alone' tests at the end of the testsuite runIan Lynagh2012-10-111-1/+4
| | | | | This is more efficient than having to wait for all threads to finish before each 'alone' test.
* Fix a testsuite driver failure on cygwinIan Lynagh2012-05-221-0/+11
|
* Change how we detect whether the terminal is cygwin or msysIan Lynagh2012-05-011-9/+9
| | | | sys.platform now always returns 'win32' for me.
* Disable threads with python 2.7.2.Paolo Capriotti2012-03-061-0/+6
|
* cleaning of testsuiteDavid Terei2011-11-151-4/+0
|
* Driver: define config.libdir for tests to useIan Lynagh2011-10-051-1/+1
| | | | | We also use it internally, which saves us running "ghc --print-libdir" on Windows.
* Add support for checking whether files are written by more than one testIan Lynagh2011-08-071-0/+5
| | | | | | | | | | | | | | | | | | | | Work in progress, but largely works. Known issues: * only supported when using the timeout program * 'test.strace' files aren't cleaned, as they end up in the root directory rather than the test's directory * Doesn't yet track what the current directory is, so finds several files like "A.o" being written by multiple tests (and conversely, may be missing writes to the same file from different directories) * Lots of tests write to $HOME/.ghc/ghci_history. We should probably be passing ghci a flag to stop this from happening. * Some .strace lines aren't understood yet, causing framework failures * One .strace file can cause muiltiple framework failures, if it contains lots of lines that aren't understood Performance: Threads fast testsuite time fast testsuite time with checks 1 16:36.14 25:16.07 5 5:33.95 8:04.05
* Expect 4006 to fail on msysIan Lynagh2011-06-231-0/+8
| | | | | | | In the 65001 codepage, we can't even cat the expected output on msys: $ cat 4006.stdout It works here cat: write error: Permission denied
* speedup testsuite driver startup on Windows/OSXSimon Marlow2010-09-151-25/+27
| | | | | | Instead of calling "ghc-pkg list" and "ghc-pkg field" for each package, call "ghc-pkg dump" and grep the output. Saves a few seconds on Windows for 'make TEST=foo'.
* Move the ctypes import in the driverIan Lynagh2010-05-181-1/+1
| | | | | The import is failing on sparky, and we only use it on Windows anyway, so move it inside an "if windows".
* Fix running dyn tests on OS XIan Lynagh2010-05-031-17/+18
|
* Make the dynlib tests work on WindowsIan Lynagh2010-05-031-1/+33
|
* Fix running the testsuite on msysIan Lynagh2010-04-121-2/+10
| | | | | I'm not entirely sure if the cygwin code is actually right (i.e. I'm not sure what calling convention it uses), but it seems to work.
* Use the UTF8 codepage when running on WindowsIan Lynagh2010-03-191-22/+33
| | | | Fixes openTempFile001 for some system codepages, most notably 437 (US).
* Gather all tests at once, rather than doing them directory by directoryIan Lynagh2009-11-281-22/+13
| | | | | This increases the parallelism possible, and allows us to track what progress we are making.
* Fix setting of utf8 locale on Mac OS XManuel M T Chakravarty2009-11-121-1/+1
|
* Don't use threads on WindowsIan Lynagh2009-10-281-17/+22
| | | | It seems to cause some sort of deadlock
* wibbles to setting LC_ALL, trying to fix buildbot test failuresSimon Marlow2009-09-291-2/+3
|
* Fix runtests.py for Python 2.6.1Manuel M T Chakravarty2009-09-171-3/+5
| | | | - This is the version of Python that comes with Snow Leopard
* Look at whether we already have a UTF8 locale, before trying to find oneIan Lynagh2009-06-111-9/+14
| | | | | | This works around a problem on (old?) Fedora systems, where "locale -a" lists every locale that /might/ exist, rather than only those that /do/ exist.
* Try and find a utf8 locale to useIan Lynagh2009-06-101-0/+16
| | | | In other locales we get some test failures for some ghci unicode tests
* Fix trac #3091: the driver was choking on python versions containing lettersIan Lynagh2009-03-201-3/+7
|
* Simplify, and correct, the code for determining whether we should use threadsIan Lynagh2008-06-121-0/+1
|
* Refuse to use threads unless python version >= 2.5.2Ian Lynagh2008-06-111-1/+12
| | | | | According to trac #1558, 2.5.2 should work. It's possible a lower bound, e.g. 2.5, would suffice.
View Lorry Controller Status