summaryrefslogtreecommitdiff
path: root/test/cairo-test-runner.c
Commit message (Collapse)AuthorAgeFilesLines
* Test cairo-ps in CIUli Schlachter2022-12-281-0/+14
| | | | | | | This adds the necessary dependencies for testing cairo-ps to CI and sets up jobs for testing the ps2 and ps3 backends. Signed-off-by: Uli Schlachter <psychon@znc.in>
* svg: Fix create-for-stream test caseUli Schlachter2021-07-241-12/+0
| | | | | | | | | | | | | | | | | | | | | | Commit 2fbd53 added another test to create-for-stream that failed for cairo-pdf. Manual testing shows that this test also fails for cairo-svg. However, this was not noticed because even before this addition to the test, create-for-stream already failed for cairo-svg. Since the introduction on CI was done based on "let's ignore all current failures", this hid the error. This commit applies the trivial fix for cairo-svg to make this test pass. It is basically the same error that cairo-pdf had. Additionally, this removes the hack to ignore create-for-stream failures in CI since it is now no longer necessary. Thanks to Knut Peterson for reporting this [0]. [0]: https://lists.cairographics.org/archives/cairo/2021-July/029291.html Signed-off-by: Uli Schlachter <psychon@znc.in>
* Fix remaining testsAnton Danilkin2021-05-051-51/+10
|
* Merge branch 'ci-test-x11' into 'master'Uli Schlachter2021-04-301-0/+9
|\ | | | | | | | | CI: Check test results for cairo-xcb and cairo-x11 See merge request cairo/cairo!165
| * .gitlab-ci.yml: Ignore current X11 test failuresUli Schlachter2021-04-251-0/+9
| | | | | | | | Signed-off-by: Uli Schlachter <psychon@znc.in>
* | Merge branch 'mac-tests' into 'master'Uli Schlachter2021-04-261-0/+14
|\ \ | |/ |/| | | | | Run the test suite on MacOS See merge request cairo/cairo!160
| * Run the test suite on MacOSUli Schlachter2021-04-221-0/+14
| | | | | | | | | | | | | | This adds the necessary commands to run the test suite on MacOS in CI and to also ignore the current failures. Signed-off-by: Uli Schlachter <psychon@znc.in>
* | Ignore create-for-stream failure with svgUli Schlachter2021-04-181-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The create-for-stream test verifies that writing something to a file and writing it to an in-memory surface produces the same output. The test currently fails when the svg backend is tested with: TEST: create_for_stream TARGET: pdf RESULT: PASS svg: Stream based output differ from file output for output/create-for-stream.out.svg. TEST: create_for_stream TARGET: svg RESULT: FAIL I guess this is because svg uses unique IDs for surfaces, meaning that drawing two times the same thing in the same process can produce different outputs. However, this is just a guess and I didn't investigate further. Signed-off-by: Uli Schlachter <psychon@znc.in>
* | Add hack to ignore a svg test crashesUli Schlachter2021-04-181-0/+16
|/ | | | | | | This adds a special hack to the test suite to ignore the crashes for self-copy and self-copy-overlap for the svg backend in CI. Signed-off-by: Uli Schlachter <psychon@znc.in>
* test-suite: Add a new mechanism for XFAILsUli Schlachter2021-04-171-0/+101
| | | | | | | | | | | | | | | | | | | | | This commit adds a new mechanism to mark tests as expected to fail via an environment variable. For example, if you expect the tests "foo" and "bar" to fail when run under image.argb32, you would set CAIRO_TEST_IGNORE_image_argb32=foo,bar The test suite then expects these tests to fail and treats this as xfail. If they do not fail, this is a failure on its own. This new feature is explicitly not documented much, because it is only used as a stopgap measure to make our CI more useful: Right now the test suite runs on CI, but the result is ignored. This new feature allows to mark the known failures as xfail without too much work. When the situation changes, this will be noticed as a new test suite failure. Thus, these environment variables to not run into the danger of still containing tests that were already fixed. Signed-off-by: Uli Schlachter <psychon@znc.in>
* test suite: fflush() before fork()Uli Schlachter2021-03-081-0/+6
| | | | | | | | | | | | | | | | | | | | | Forking a process also duplicates the buffers of FILE*s. Thus, if there is pending data, both the parent and the child process will write things. This is seldom a good idea. This issue was not noticed so far since by default the test suite already calls fflush() a lot. However, when stdout and stderr are both not a tty (according to isatty(1) and isatty(2)), these flushes are skipped. The result is that the child process repeat the full output from the test suite starting with "Compiled against cairo 1.17.4, running on 1.17.4." To reproduce this problem run: ./cairo-test-suite 2>&1 | cat Fix this by flushing all the files that I managed to find before fork(). Thanks to Pekka Paalanen for helping me figure this out. Signed-off-by: Uli Schlachter <psychon@znc.in>
* Retire dummy cairo-version.h header to fix meson subproject buildTim-Philipp Müller2020-09-291-7/+0
| | | | | | | | | | | | | | It was originally added to make bisecting easier, but has outlived its usefuleness now. Going forward we'll have just a single cairo-version.h header file, the one with the real version numbers. This is needed to fix the case where cairo is being built as a Meson subproject, but also simplifies things in general. Fixes #421
* meson: Fix musl buildGeorge Matsumura2020-09-021-3/+5
| | | | | | | This constitutes few fixes that are necessary to compile correctly and reduce errors when using musl libc. Signed-off-by: George Matsumura <gmmatsumura01@bvsd.org>
* test: fix build when SHOULD_FORK is falseFabrice Fontaine2020-02-031-1/+1
| | | | | | | | | | | | | | | | The code in test/cairo-test-runner.c properly takes into account platforms that do have fork() support, and uses the SHOULD_FORK define to know whether fork is available or not. However, this SHOULD_FORK macro is used to guard the inclusion of <unistd.h>, which is needed to get the prototype of other functions (namely readlink and getppid), that are used in portions of this file not guarded by SHOULD_FORK. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> [Retrieved from: https://git.buildroot.net/buildroot/tree/package/cairo/0001-fix-nofork-build.patch] Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
* test: Free test listAndrea Canciani2014-12-061-0/+1
| | | | | | | | | | | | | | | | When running the cairo-test-suite, valgrind reports each of the registered test as a leak, because they are _list_prepend()'ed, but the tests list is never _list_free()'d. Fixes the following valgrind error: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) cairo_boilerplate_xmalloc (cairo-boilerplate-system.c:47) cairo_test_register (cairo-test-runner.c:131) _cairo_test_runner_register_tests (cairo-test-constructors.c:1112) main (cairo-test-runner.c:714) Signed-off-by: Bryce Harrington <bryce@osg.samsung.com>
* test: Allow CAIRO_TEST_MODE to independently enable extended testingChris Wilson2013-09-051-4/+18
| | | | | | | | | Such as CAIRO_TEST_MODE=scale for testing application of cairo_surface_set_device_offset. CAIRO_TEST_MODE=similar - test rendering through similar surfaces CAIRO_TEST_MODE=offset - test rendering with a device offset CAIRO_TEST_MODE=scale - test rendering with a device scale
* tests: Add device scale test to "full" testrun similar to offsetsAlexander Larsson2013-09-051-30/+36
|
* test: Use fork() on MacOS XAndrea Canciani2011-12-151-4/+1
| | | | | | | | | | MacOS X 10.7 (and maybe some previous versions, too) can fork() processes which use CoreGraphics. This makes it possible for cairo-test-suite to withstand a test crash without killing the whole suite. The old behavior is still available using the '-f' (foreground) option.
* test: Use cairo_test_list_t for the main test listAndrea Canciani2011-11-121-18/+6
| | | | | | | | | | Instead of embedding the pointer in the test structure, consistently use the cairo_test_list_t structure for test lists. This cleans up the code as the reverse-list operation can be reused. Moreover this makes the code clearer, because each test list is now independent and has no way to know about other test lists.
* Improve the documentation of the flagsAndrea Canciani2011-11-121-5/+4
| | | | | Some utilities were providing incorrect or incomplete usage information.
* Sort option flagsAndrea Canciani2011-11-121-8/+8
| | | | | | | | Keep the option flags in alphabetical order. This makes it easier to check for collisions or missing handlers. Avoids an internal error when passing flags -c, -r or -v to cairo-analyse-trace.
* test: Write the individual test logs to output/Chris Wilson2011-09-151-1/+1
| | | | Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
* cairo-test-runner: don't leak argvPaulo Zanoni2011-09-091-1/+7
| | | | | | | | | | | If CAIRO_TESTS is set, argv will be replaced by a xmaloc'ed variable. We need to free it. This can be easily detected by running: CAIRO_TESTS=user-font make check-valgrind TARGETS=image Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
* test: fix append_argv()Paulo Zanoni2011-09-091-4/+10
| | | | | | | | | | When I ran "CAIRO_TESTS=a1-bug make test", no test executed because of a bug in append_argv(). The "olen" variable was assuming that we always only append a single argument to argv and the resulting argc was also wrong. Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
* test: Silence gcc warningsAndrea Canciani2011-03-171-4/+4
| | | | | | | | | warning: cannot optimize possibly infinite loops gcc does not detect that the "infinite" loops are actually just one or two iterations, depending on the has_similar value being FALSE or TRUE. It realizes it if the iteration variable and the iteration stop value are both enum values.
* test: Silence gcc warningAndrea Canciani2011-03-171-4/+4
| | | | cairo-test-runner.c:180: warning: unused variable ‘buf’
* test: Fix gcc complaints about old style definitionsBenjamin Otte2011-02-191-1/+1
|
* test: Make tests run in natural orderAndrea Canciani2010-10-071-0/+13
| | | | | | When tests are registered, they are prepended to a list. Reverting the list, we get a list whose order is the same as that of registration.
* test: correct options in getopt callAndrea Canciani2010-06-171-1/+1
| | | | | 'k' was missing and 't' was present but unhandled. Options have been reordered according to the "USAGE" ordering.
* test: Disable tests that are too slow to run by default.Chris Wilson2010-06-121-2/+15
| | | | | | Introduce a slow mode [-s] to the runner, and add "slow" to the requirements for the very long running tests like the coverage stress tests.
* test: Use exact matching on test names by default.Chris Wilson2010-06-121-23/+22
| | | | | | Specifying individual tests to run is more common than using a group chosen from a keyword, so change the default matching mode and introduce '-k' to select by keyword.
* test: Distinguish tests that throw an error from a normal fail.Chris Wilson2010-06-111-2/+87
| | | | | | | Hitting an error in a test case is almost as bad as crashing, and the severity may be lost amidst "normal" failures. So introduce a new class of ERROR so that we can immediately spot these during a test run, and appropriately log them afterwards.
* test: Force cairo-test-suite to return SUCCESSChris Wilson2010-05-081-0/+10
| | | | | | Set the CAIRO_TEST_FORCE_PASS environment variable to run through the test suite and ignore errors. Useful for forcing distcheck to continue past a broken test suite.
* test: Embed the list pointer into cairo_test_tChris Wilson2010-05-041-16/+15
| | | | | Reduce the amount of output printed by leak-detectors when a test crashes.
* test: Add an option to explicitly specify a test name to the runner.Eric Anholt2010-02-171-3/+11
|
* [test] Attempt to automatically detect running under gdbChris Wilson2009-08-261-0/+32
| | | | | | | | | A common mistake is to forget to pass the foreground mode to cairo-test-suite when launching it under the debugger, resulting in the debugger not attaching to the children and missing the error you were trying to capture. Under linux, we can inspect the path to our parent's executable and if that looks like gdb, we assume it is and disable forking of traces.
* [test] Exit on first failure '-x'Chris Wilson2009-08-261-2/+13
| | | | | | | | Add a command line option to the test suite to cause it to exit after the first failure. The purpose of this is for integration into 'git bisect run', where the failing test is unknown and we are looking for any failure. For example, for use in a regression script to find commits in the midst of as series that need a refresh of a reference image (or fixing!).
* [test] Summarise tests that fail during the preamble.Chris Wilson2009-07-201-24/+85
| | | | | Some tests only run and check during the preamble phase, and those failures were being ignored during the summary.
* [test] Track XFAIL using expected results stored as xfail.pngChris Wilson2009-07-131-119/+42
| | | | | | | | | | | | | | | | Instead of tagging the sources, which is insensitive to changes, track the known failure modes by recording the current fail as an xfail.png reference. (We also introduce a new.png to track a fresh error, so that they are not lost in the noise of the old XFAILs and hopefully do not cause everyone to fret). As we have removed the XFAIL tagging we find, surprise surprise, that some tests are now working -- so review all the reference images (as also some .ref.png now should be .xfail.png). Note: I've only checked image,pdf,ps,svg. The test surfaces report some failures that probably need to addressed in source. I've not correct the changes for win32 and quartz. Nor fixed up the experimental backends.
* [test] Never use gcc __attribute__ magic for constructors.M Joonas Pihlaja2009-06-181-2/+0
| | | | | | | | The configure test for __attribute__((constructor)) succeeds when compiling with Sun Studio 12 on OpenSolaris but the attribute is just ignored and has no actual effect. This causes the test suite to not run any tests at all. With this patch we revert to always using make-cairo-test-constructors.pl.
* [test] Never, ever, blame a crashing test on external factors.Chris Wilson2009-06-131-1/+1
|
* [test] Add a note to "Getting the elusive zero failures"Chris Wilson2009-05-151-0/+12
| | | | | | | The test suite depends upon many external factors and in order to achieve a pass, your system must match that upon which the reference images were generated. Add a note to read test/README in case of failures so that the casual user is not unduly alarmed by cairo reportedly failing.
* [sdl] Remove new backend.M Joonas Pihlaja2009-02-161-4/+0
| | | | | | | | The SDL backend makes invalid assumptions about SDL_Surface locking semantics and doesn't deal correctly with the unpremultiplied pixel format supported by SDL. Removed as per discussion on the mailing list. http://lists.cairographics.org/archives/cairo/2009-February/016595.html
* [test] Add crtdbg.h includeJeff Muizelaar2009-02-101-0/+4
| | | | crtdbg.h is required for _CrtSetReportMode() and _CrtSetReportFile().
* [test] Quartz doesn't like being forkedJeff Muizelaar2008-12-181-4/+8
| | | | | | | When the cairo-test-suite forks CoreFoundation complains with: "The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). When forked so avoid doing it in the test suite for now. In the future we should investigate the possibility of a work around.
* [perf] Fix SDL compilation for MacOS XPaolo Bonzini2008-11-261-0/+4
| | | | | | | | | | | | | The attached patch makes the SDL tests compile under Mac OS X. The problem is: 1) that <SDL_main.h> should be included in files that define the main function for SDL Mac OS X programs (this is not true with the upcoming SDL 1.3 release). 2) that -lSDLmain, because it is statically linked, needs the Cocoa framework in the LDADD of the main program. Again, 1.3 will not require this.
* [test] Support foreground only execution.Chris Wilson2008-11-191-54/+66
| | | | | Add an option to prevent forking - which makes it difficult to valgrind/gdb individual tests.
* [test] Build test suite into single binary.Chris Wilson2008-10-311-0/+907
Avoid calling libtool to link every single test case, by building just one binary from all the sources. This binary is then given the task of choosing tests to run (based on user selection and individual test requirement), forking each test into its own process and accumulating the results.
View Lorry Controller Status